3 Continuous Delivery KPIs

Continuous delivery isn’t just a buzzword: There are concrete business reasons for building a continuous delivery pipeline. Here are some important ways to judge whether or not your CD system is helping you reach the business and IT objectives that led you towards CD in the first place.

Time to Market

Innovations in development practices are not just about shipping better software—it’s about getting features in front of your customers that give your business a competitive edge, and doing so as quickly as possible. Continuous delivery is part of an agile development model that allows you to release new features to customers, get feedback and adjust/innovate based on your customers’ behaviors.

Continuous delivery should dramatically speed up time to market and time to value, so measuring the amount of time it takes from determining a business logic for a specific application or new feature to deploying that app or feature to your customer base is the ultimate measure of how well your continuous delivery system is working.

Bugs—Do You Have Them and How Quickly Are They Fixed?

Move fast and break things, right? Errr, just to be clear, if we’re talking about your customer-facing ecommerce site on Black Friday, actually let’s break things at the QA stage, ok?

Part of a continuous delivery pipeline is recognizing that mistakes happen and things break. But a successful pipeline includes a sufficiently robust QA process that customers aren’t the ones discovering bugs in your software. It also involves testing new releases in an environment that is identical to your production environment, and managing deployments in a repeatable way, so you know that code is deployed to test and production environments in exactly the same way.

You should track how many bugs are caught during the QA testing v.s. how many are caught in production—sometimes called your defect escape rate.

And that’s not all. There are a couple other important bug-related metrics: How long does it take to fix bugs in production, when they happen? How often is a defect so serious it crashes the application?

Ease/Speed of Deployment

When operating a truly continuous delivery pipeline, deployments will likely happen multiple (perhaps even thousands) times per day. The deployment process should be automated and repeatable, to ensure the following:

• There is zero application downtime during deployments
• Deployments to test environments are handled exactly the same way as deployments to production environments
• Developers are not wasting time managing the deployment rather than working on a new application/feature
• It’s safe and easy to roll back a deployment if it’s immediately clear that something is wrong

It’s impossible to speed up the actual deployment process—and ensure it is done safely and repeatably every time—if you’re deploying manually. Using a deployment automation platform like Spinnaker and Armory is the only way to make sure you’re handling deployments quickly and safely.

The goal of continuous delivery is to tighten your feedback loop, constantly improve your software and make deployments a routine, ho-hum, push-of-a-button affair rather than a major event followed by wrap-up party. A successful transition to continuous delivery will usually change your company culture and lead to happier developers—but that’s harder to measure than the points discussed above.

What to learn more about how Armory can help make your CI/CD pipeline a success? Reach out.