Everyone Talks About It; Few Manage To Do It: Shipping Small Diffs
May 9, 2017 by Armory
Podcast with Isaac and Colin from the Armory tribe where Isaac explains “Ship Small Diffs” and what the value surrounding it is:
During an on-site visit with a client, I saw “Ship Small Diffs” written on a whiteboard. I had a small chuckle, then I thought about it a bit more and said to myself, “Why isn’t shipping small diffs the status quo?”. For instance, it would be odd to see a whiteboard in today’s tech world that says “Make sure to use a source code repository” written on it.
“Release early, release often” comes to mind when we talk about “ship small diffs.” The current process at many organizations is for a dedicated team of DevOps to facilitate and oversee the deployment process, known to developers as a Slack message that probably looks like: “The DevOps team regrets to inform you that your code has been deployed.”
So what is “Ship Small Diffs?” It is the organizational mentality that deploying in the tiniest possible increments is the best form of integrating advancement. Instead of updating in large batches at a slower frequency, “ship small diffs” wants the organization to be deploying in small increments at a faster frequency. Having code be used by real customers and getting early feedback allows you to learn, fail and innovate at a much faster pace.
When merging and deployments are done as larger batches, problems and bugs can be compounded and make the headache of debugging and identifying the issues a resource guzzling nightmare. If you have 30 bugs but you pushed to production once a day, you can potentially isolate it to fixing one bug a day, instead of 30 bugs at the end of the month. Additionally, small diffs means the reviewer doesn’t suffer review fatigue when they glance at a ten lines of code compared to several thousand lines.
Moreover, smaller changes to the live production means you get to learn faster from your users and iterate changes based on what worked and what didn’t. This means quicker chances to pivot once you know you’re going in the wrong direction, thus saving you time building the wrong product. Shipping smaller diffs is a benefit to the bottom line for most organizations.
With that in mind, why do organizations and DevOps teams have difficulty adhering to “Ship Small Diffs?”
This stems from developers having a history of expecting catastrophic bugs. It’s a systemic loop that looks something like: shipped a bundle of features → something broke → okay let’s try and QA more next time → longer QA time means larger batch to ship → OH NO SOMETHING BROKE → MORE QA TIME → repeat ad nauseam.
The difficulties with faulty deployments has created a negative feedback loop. Fortunately, companies like Netflix, Facebook, and Google have realized the value of shipping daily, often thousands of times a day. They ship small diffs.
Armory has a packaged enterprise Spinnaker that helps fast-track an organization to the “ship small diffs” mentality. The longer a DevOps team uses Spinnaker and becomes more confident in their automated deployments, the more willing they are to write code and push it to production. This in turn gives your teams more feedback from live users, and then further gives you more metrics to iterate upon. As an added side effect of deploying smaller diffs, the issues are less compounded together and less likely to cause catastrophic crashes for your users.
That’s not to say ALL organizations should immediately get into this mindset. We recognize that some companies, such as gaming companies, may prefer to use larger launches due to their specific user-base (imagine how confused gamers would be if the game version changed every 30 minutes!). However, for those organizations that are seeing a hesitant team of developers due to large batches of deployments making life difficult, we suggest adopting the “Ship Small Diffs” mindset.
