The How and Why of Templated Pipelines

Mar 28, 2019 by Armory

In an enterprise setting, it’s likely you’ll have not just more than one pipeline—you’ll have thousands. Netflix, for example, has over 9,000 individual pipelines. Sometimes a single individual is responsible for managing hundreds of pipelines.

Keeping all of those pipelines up-to-date, ensuring configurations are consistently applied and making any change in the pipeline could become a full-time job that doesn’t leave room for anything else. They also make it possible to take advantage of the fact that many pipelines are very similar, if not identical. Reusing pipelines through templates makes sense, saves time and reduces the number of errors that come with configuring a pipeline from scratch every time and/or manually updating or changing hundreds of pipelines.

We talk a lot about ‘paved road,’ or workflows that make it easy for deployments be rolled out in a repeatable, safe way. Having a standard way to manage deployments, one that works for most teams most of the time, makes it easier to ensure the entire organization is following deployment best practices, regardless of the project or the deployment target.

Using pipeline templates also helps manage multi-cloud deployments or even Kubernetes deployments on top of a single cloud. Templates can and should be target-agnostic, and the specifics for each deployment target can be added into the template’s child pipeline.

Once deployment pipelines are standardized using templating, it also becomes possible to manage infrastructure changes through the Spinnaker pipeline—including running testing and canary analysis on resources such as changes in instance types, firewall rules and load balancers. Applying the same process to infrastructure changes that is applied to application changes makes those changes safer.

How does it work?

If you’re using Armory’s Spinnaker installation, you can use Armory Dinghy service. Pipelines can either be created declaratively or can also be created from an existing pipeline.

In a standard Spinnaker installation, you can use the Spinnaker Managed Pipeline Templates feature.

The exact process and file naming conventions differ slightly between the two services. However, in general the steps are:

Once you have a template, you can create child pipelines from that template.

If you modify the parent template, a GitHub or BitBucket webhook will find and update all of that template’s dependent pipelines. All of the dependent templates are automatically updated whenever the template is updated.

Using pipeline templates makes it easier to manage the complexity of thousands of microservices. It creates a paved road that makes handling the deployment process consistently across an organization and across cloud environments possible.

Templates do not necessarily take flexibility away from any part of the team. The child templates can still be altered to add additional stages, to increase wait times between stages or in other ways to meet the needs identified for a specific application. However, cutting down ad-hoc pipeline creation not only saves time but contributes to an overall safer, more repeatable deployment process.

Share this post:

Recently Published Posts

How to Become a Site Reliability Engineer (SRE)

Jun 6, 2023

A site reliability engineer (SRE) bridges the gap between IT operations and software development. They understand coding and the overall task of keeping the system operating.  The SRE role originated to give software developers input into how teams deploy and maintain software and to improve it to increase reliability and performance. Before SREs, the software […]

Read more

Continuous Deployment KPIs

May 31, 2023

Key SDLC Performance Metrics for Engineering Leaders Engineering leaders must have an effective system in place to measure their team’s performance and ensure that they are meeting their goals. One way to do this is by monitoring Continuous Deployment Key Performance Indicators (KPIs).  CD and Automated Tests If you’re not aware, Continuous Deployment, or CD, […]

Read more

What Are the Pros and Cons of Rolling Deployments?

May 26, 2023

Rolling deployments use a software release strategy that delivers new versions of an application in phases to minimize downtime. Anyone who has lived through a failed update knows how painful it can be. If a comprehensive update fails, there are hours of downtime while it is rolled back. Even if the deployment happens after hours, […]

Read more