The How and Why of Templated Pipelines

Mar 28, 2019 by Armory

In an enterprise setting, it’s likely you’ll have not just more than one pipeline—you’ll have thousands. Netflix, for example, has over 9,000 individual pipelines. Sometimes a single individual is responsible for managing hundreds of pipelines.

Keeping all of those pipelines up-to-date, ensuring configurations are consistently applied and making any change in the pipeline could become a full-time job that doesn’t leave room for anything else. They also make it possible to take advantage of the fact that many pipelines are very similar, if not identical. Reusing pipelines through templates makes sense, saves time and reduces the number of errors that come with configuring a pipeline from scratch every time and/or manually updating or changing hundreds of pipelines.

We talk a lot about ‘paved road,’ or workflows that make it easy for deployments be rolled out in a repeatable, safe way. Having a standard way to manage deployments, one that works for most teams most of the time, makes it easier to ensure the entire organization is following deployment best practices, regardless of the project or the deployment target.

Using pipeline templates also helps manage multi-cloud deployments or even Kubernetes deployments on top of a single cloud. Templates can and should be target-agnostic, and the specifics for each deployment target can be added into the template’s child pipeline.

Once deployment pipelines are standardized using templating, it also becomes possible to manage infrastructure changes through the Spinnaker pipeline—including running testing and canary analysis on resources such as changes in instance types, firewall rules and load balancers. Applying the same process to infrastructure changes that is applied to application changes makes those changes safer.

How does it work?

If you’re using Armory’s Spinnaker installation, you can use Armory Dinghy service. Pipelines can either be created declaratively or can also be created from an existing pipeline.

In a standard Spinnaker installation, you can use the Spinnaker Managed Pipeline Templates feature.

The exact process and file naming conventions differ slightly between the two services. However, in general the steps are:

Once you have a template, you can create child pipelines from that template.

If you modify the parent template, a GitHub or BitBucket webhook will find and update all of that template’s dependent pipelines. All of the dependent templates are automatically updated whenever the template is updated.

Using pipeline templates makes it easier to manage the complexity of thousands of microservices. It creates a paved road that makes handling the deployment process consistently across an organization and across cloud environments possible.

Templates do not necessarily take flexibility away from any part of the team. The child templates can still be altered to add additional stages, to increase wait times between stages or in other ways to meet the needs identified for a specific application. However, cutting down ad-hoc pipeline creation not only saves time but contributes to an overall safer, more repeatable deployment process.

Share this post:

Recently Published Posts

Lambda Deployment is now supported by Armory CD-as-a-Service

Nov 28, 2023

Armory simplifies serverless deployment: Armory Continuous Deployment-as-a-Service extends its robust deployment capabilities to AWS Lambda.

Read more

New Feature: Trigger Nodes and Source Context

Sep 29, 2023

The Power of Graphs for Ingesting and Acting on Complex Orchestration Logic We’ve been having deep conversations with customers and peer thought leaders about the challenges presented by executing multi-environment continuous deployment, and have developed an appreciation for the power of using visual tools such as directed acyclic graphs (DAG) to understand and share the […]

Read more

Continuous Deployments meet Continuous Communication

Sep 7, 2023

Automation and the SDLC Automating the software development life cycle has been one of the highest priorities for teams since development became a profession. We know that automation can cut down on burnout and increase efficiency, giving back time to ourselves and our teams to dig in and bust out innovative ideas. If it’s not […]

Read more