The Now and Future State of Software Delivery
Apr 23, 2018 by TC Currie
With the explosion of microservices and distributed systems to manage software where continuous integration/continuous delivery is critical, new processes are required, and nowhere does that show up as clearly as in the Software Development Lifecycle (SDLC). It’s been a wide open playing field with no consensus on standard approaches to the proliferation of containers, microservices, APIs, and other cloud tech because everything is still brand new. Kubernetes is not yet 3 years old and is emerging as an industry standard. Spinnaker, which powers the Core of Armory’s Platform, was open sourced by Netflix only a few years ago. Even industry giant GitHub is only celebrating its 10th anniversary. So if you’re wondering why all of a sudden you don’t feel like you know anything — it’s not just you.
Today, each stage of the software continuous deployment path has its own set of competing vendors each with their own processes, UIs and interaction rules. And invoices.
Who are you? Who? Who?
So who are these modern application software development vendors, and what do they accomplish in the SDLC? And how many of them are required to get your code up and running? Let’s run through a typical high-level SDLC from "idea" to "feature in production, generating revenue." These examples include typical vendors for each stage.
In the Idea phase of the SDLC, you’re likely doing sprint planning (1) in Jira. When the Coding gets going, it’s deposited in a source code repo (2), like GitHub or GitLab. This is where you do QA iterations (3), using, say, RainforestQA. And you, of course, are using code-scanning security tools (4) like Fossa. Then your software code gets packaged, and an artifact gets created.
Moving on to the Releasing phase, you do Continuous Integration (CI) (5), where Jenkins is popular. Then Armory comes in, with Spinnaker powering Continuous Delivery (6), enabling automated Release Management cycles. Your artifact gets delivered to the infrastructure in your cloud targets (7) like containers in GCP, or VMs in AWS (or vice versa!), which require infrastructure & container management systems (8). More and more, Kubernetes is becoming the standard in this arena. Back to Rainforest for QA. Now, security scanning tools (9) are needed, like Checkmarx, Veracode or Stackrox.
Congrats, you’re now ready to move your code into production.
First, you don’t just throw your artifact into production; you put it behind feature flags (10), functionality provided by Launch Darkly. Armory helps you deploy continuously using a canary deployment — behind a feature flag, of course — to limit your customer blast radius while you evaluate how the new version of your application your affects all areas of production. Canaries work in conjunction with monitoring systems (11) like Datadog, New Relic and SignalFX. These companies monitor both your software, infrastructure, and customer experience. Now you turn to your incident response company (12) like Pager Duty, which is able to identify issues and reduce the “mean time to resolution,” meaning how long it generally takes to resolve an issue and restore customer trust. Given that downtime can cost a larger retailer, for example, thousands of dollars every minute the system is down, this is critical. Last but not least is the need to manage all your artifacts (13) which you can do through J-Frog.
So there we have a baker’s dozen products to get your idea into production. No wonder you’re exhausted.
The logical next step is consolidation. Acquisitions and mergers are likely in the coming years as companies either try to build more of the SDLC’s functionality into their existing platforms, or acquire competitors. Many of these vendors are already beginning to expand beyond their initial expertise into adjacent areas on the SDLC path. Gitlab is beginning to offer CI/CD. Atlasssian is offering Bit Bucket, their own code repository rivaling GitHub and Gitlab.
When these companies start thinking about adding additional value, it makes sense for them to look at adjacent categories of tooling that are already plugging into them. Companies will expand along the SDLC pipeline.
Containers Will Become the Artifact
Another thing we’re seeing is source code vendors and source code repository providers beginning to incorporate continuous integration functionality into their tooling. Specifically, CI is going to be dissolving into source code repos, and CD systems like Spinnaker are ingesting this functionality.
Container registries at the cloud provider level are also beginning to compete for that space and trying to dissolve or ingest the CI into their platforms where the artifacts are stored. Soon, we believe, standalone CI will cease to be a separate offering.
With more and more companies moving everything into containers, it looks like they’re going to start using the cloud provider native artifact storage. Since GitHub, GitLab are already storing the source code, they’re going to try to store artifacts.
Then containers become the artifact. And, since cloud providers already providing that functionality, there’s really no reason for you to use an external registry, unless you want to have a multi-cloud setup.
Note that a multi-cloud strategy calls for much more complex technology. For example a company uses Docker, but they don’t want to put all of their eggs in one basket; they don’t want to put all their artifacts into the same provider. If that provider goes down, they could theoretically redeploy to another cloud provider and keep their system up and running. The technology to do this seamlessly is not quite there yet for stateful services. If your service is stateless, this can be done today, but stateful services still need work. But that’s coming.
All this consolidation is good news for the end customers building software. Fewer vendors mean less complexity, less training , smoother delivery, and (hopefully) fewer dollars.
Armory exists to help companies ship better software, faster.
Today, we are focused on solving deployment challenges, because delivering software to users– continuously– is where the wheels typically fall off for most companies (most are using Jenkins for CI + a mis-match of brittle custom scripting on mutable, infrastructure management tooling like Chef, Puppet or Ansible to get software into production — and often companies are trying to support many simultaneous paths to production across multiple application teams). As our Platform Roadmap shows, we see many opportunities to remove inefficiencies across the SDLC, beyond just the CD portion.