Note: Engine Yard friend Daragh Curran, Head of Product Engineering, Intercom has graciously let us post this great piece about code deployment on our blog. Check it out on their own blog here.
Software only becomes valuable when you ship it to customers. Before then it’s just a costly accumulation of hard work and assumptions.
Shipping unlocks a feedback loop that confirms or challenges those assumptions. It makes new things possible for your customers, and gives you the opportunity to focus on the next thing.
Shipping brings life to your team, to your product, and to your customers. Shipping is your company’s heartbeat.
Shipping will try to kill you
The scramble to get that one last feature done, the late nights, the compromises, the sinking feeling when we realise something major is broken, the post-mortems… It’s agony, but if it was easy everyone would do it. Shipping exposes mistakes. We’re nervous about it, and our natural reaction is to do it reluctantly and infrequently, which actually carries higher risk, causing more reluctance in the future.
The cost of shipping is approaching zero
Not too long ago, shipping software involved actual ships, disks, and printed manuals. It happened perhaps once a year. Bug fixes weren’t automatic over the internet like today. Everything was slower and more controlled. The cost of shipping was massive, the consequence of a mistake was large. Today, the cost of shipping has approached zero. Most people can deploy in seconds or minutes with a single command or button click. With a little thought you can do that without your customers noticing, and with automated monitoring you’ll find out immediately if something goes wrong.
Despite the cost of shipping approaching zero, many people still ship software guided by very old habits.
Shipping cadence defines your company
The cadence at which you ship defines your company. A yearly cadence results in a very structured approach to the design->build->test cycle. A few months of building, while the rest is spend fixing. Engineers can join and leave before seeing their hard work end up in the hands of customers. The approach to design becomes one of anticipating all possible needs, rather than focusing and iterating on the important ones.
Obstacles downstream propagate upstream
An obstacle downstream propagates upstream. If you’re not allowed to implement new ideas, you stop having them.
- Paul Graham
The right approach to shipping has a positive influence on your company’s productivity and your team’s happiness & job satisfaction. Shipping infrequently is an obstacle. Ship slow, and you’ll introduce challenges that push you to ship even slower. Ship frequently, and see positive effects everywhere in your company. For example, lets examine how behaviour changes along with shipping frequency, while handling a simple request from a customer.
Lets say a customer gets in touch to say “No matter what I do, I cannot save my name correctly, I think it doesn’t like hyphens“. In a company where you ship continuously, you see this and think Simple — I’ll tweak a test and a regex pattern, get a quick code review from my buddy beside me, merge to mainline, and 1 minute later when it’s deployed to production, reply to the customer: “Sorry about this, it’s fixed now, thanks for letting us know“. They’ll reply: “Wow, thanks for fixing so quickly“. High fives all around!
If we stretch the time to production (TTP) out a little, even to 10 minutes, the behaviour changes. You either do the same, but reply saying it’ll be fixed with our next deploy (probably 10 minutes) – or you wait, so that you can communicate with certainty. The waiting is time where you’ll shift focus to something else, but have the baggage of having to follow up. Perhaps you’ll think, I’ll have a quick coffee, then move on to something else afterwards. Even though your deployments are entirely automated, you lose time because of waiting and losing focus.
If TTP is hours, the behaviour changes again. No longer can you say with certainty when the change will be out there, so you’re tempted to batch up with other similar small changes. You postpone replying until you get time to do it, sometimes forgetting about it. You’re less likely to take prompt action, wow’ing the customer, and you pay some mental cost for having it on a todo list. Since getting to production takes hours now, your team will start restricting to morning only deploys, so miss that slot and it’s further delays.
If TTP is days, it exacerbates that further – perhaps you’ll reply “Thanks for letting us know. We’ll fix this in our next sprint”. It gets bundled in with a whole load of other small low, priority items, you spend more time debating estimates, and priorities, than the first guy took to fix it and reply to the customer. Miss the beginning of week deploy window and further slippage. The larger releases bring higher risk, you’ll tell your customer it’s fixed, only to later require rolling back because of a separate change. Your bug database gets bigger and bigger, with little details that you’ll probably never fix.
When TTP is weeks, it exaggerates that even further – perhaps you’ll reply “Sorry about this, I’ll let the development team know” or something equally lame from your customer’s standpoint. Deep down you realise nothing will be fixed, and the job of talking to customers becomes a cost or hassle, rather than an opportunity to improve your product and nurture happy loyal customers.
Better approaches to writing or testing software help us iterate more quickly and confidently, but the benefits are quite local to engineering teams. Continuous shipping on the other hand, touches all parts of your company, as do the benefits, and the behaviours it enables and encourages.
Linkedin’s transition to continuous deployment is linked to their recent financial success.
Good products, are a side effect of combining good people with an idea in an environment that helps those people to kick ass. Your attitude to shipping is a big part of that environment you create.
Shipping breathes life into how we think. The feedback loop helps us learn, gain confidence in making quick decisions, and build momentum. Momentum in product improvements excites and engages our customers. Seeing quickly the benefits of our hard work, motivates us to do more. Building a team where people can work hard and move fast attracts others to join you – hiring gets easier.
Shipping continuously isn’t an achievement you unlock and then move on. You’ve got to constantly obsess about it. If you believe in the benefits it brings, you’ll be driven to shrink 20 minutes down to 1 minute or less, you’ll consider ‘ability to ship‘ as an equal to ‘does it scale‘ when building new systems. And you’ll do that because of all the life it breathes into your company and your product.
Shipping is your company’s heartbeat.