At Armory, we spend every day working with our customers and our friends in the open source community to help enterprises unlock the power of Spinnaker. And what we’ve often seen is that while Spinnaker may brilliantly solve 90% of their needs, extending and customizing Spinnaker to solve that last 10%, such as integrating with some in-house tooling, can be cumbersome.
That’s why we are very excited to announce that the Spinnaker Plugin Framework, available as a part of open source Spinnaker, is fully GA. The Plugin Framework represents a significant shift in the future of the Spinnaker project, adopting a lean core / large ecosystem strategy that will keep Spinnaker highly performant while making it much easier to extend, customize, and add new integrations.
Since launching the framework as early access in the summer of 2020, we’ve already seen more than 15 plugins built by Armory and the rest of the community, and we expect to see many more now that the Plugin Framework is GA. A great example of how community members are leveraging the Plugin Framework to extend Spinnaker to meet their software delivery workflow is Ige Adetokunbo Temitayo and the United Arab Emirates team. They were able to quickly build a plugin to integrate Spinnaker with Microsoft Teams. DevOps tooling vendors have also taken advantage of the plugin framework to tap into the large Spinnaker community. For example, the Pulumi team built a plugin to integrate Spinnaker with their infrastructure-as-code platform. Armory is leaning in to plugins as well – check out this on the new, open source, Observability plugin that serves as an upgrade to the Spinnaker Monitoring daemon.
What is the Plugin Framework and Why Did We Build It?
Historically, extending Spinnaker required in-depth knowledge of the various microservices that compose the platform. And when you wanted to build that new functionality or integration, there was not much of a middle ground between building a proprietary service that you were solely responsible for maintaining, and going through the process of getting your code accepted into open source Spinnaker.
Now with the advent of the Plugin Framework, developers have a standardized way of extending Spinnaker by using PF4J extension points that interface directly with Spinnaker’s microservices. This standardized method of extending Spinnaker microservices gives developers the ability to easily share and manage their own integration as an independent member of the Spinnaker ecosystem.
For Spinnaker Users:
- Enables access to robust SDLC integrations through a large Spinnaker ecosystem.
- Introduces a standardized way to integrate Spinnaker with your DevOps tooling.
- Gives you greater control of your Spinnaker instance as more features are implemented as optional plugins.
For Spinnaker Contributors:
- Makes maintaining the ever-scaling Spinnaker project simpler, as more Spinnaker features are converted into plugins.
And for DevOps tooling Vendors:
- Allows you to tap in to the large Spinnaker community.
- Enables a standardized method for integrating your product with Spinnaker.
- Empowers you to have full control over your Spinnaker integration.
Pulumi, an early adopter of the Plugin Framework, utilized it to build a plugin that integrates Spinnaker with their Infrastructure As a Service platform:
“We heard from customers that they wanted to use Spinnaker pipelines to deliver infrastructure as code in addition to application changes. Thanks to the new Plugin Framework, Pulumi was able to team up with Armory to quickly deliver an experience across many clouds, including AWS, Azure, GCP, and Kubernetes, and to automate deployments as well as surface rich operations information like deployment logs and policy violations in the Spinnaker Console Output. This has enabled some of our largest customers to harmonize their development and infrastructure teams and ship faster with confidence!”
– Joe Duffy, CEO of Pulumi
How Does The Plugin Framework Work?
Since Spinnaker is largely comprised of Java microservices, the Plugin Framework was built using PF4J to create extension points within Spinnaker. This allows developers to create plugins that interface with Spinnaker’s respective microservices.
Typically plugins fall within four categories:
Custom Stages
These stages already exist in other, non-plugin forms, such as the custom run job and the custom webhook stage. But none of these enable creating any kind of stage you want (while the custom run job is popular, it is limited to implementation via a K8s job, doesn’t enable several desired stage task features, and doesn’t allow much UI customization in Deck).
When creating a new stage, first familiarize yourself with custom run jobs and custom webhooks in case these methods are sufficient for you. Otherwise, you can create a custom stage plugin.
Some examples:
- pf4jStage (Random Wait stage)
- extends the SimpleStage extension point (the “simpler” custom stage extension point in Orca)
- fairly straightforward, this stage adds a random wait time
- provides a sense of how the front-end of the stage is created and added to the Deck stage registry; check out this file which shows you how this plugin defines a new stage and new components in Deck’s registries
- winstonStage
- extends StageDefinitionBuilder, shows the full range of the Orca stage API
- note this stage only works in a Netflix environment
- enables sophisticated custom stages with more options than SimpleStage, such as enabling multiple tasks in each stage, allowing these tasks to be re-run, and configuring other stages to be run before, after, or based on the result of this stage
- fetchArtifact
- extends SimpleStage
- illustrates how to fetch an artifact from an external source that can be used for any stage executions, such as a CLI package
Custom Resources and Resource Handlers
Resources, for example infrastructure resources or pipeline dependencies, cover essentially everything Spinnaker manages and automates. Plugins can handle your own custom resources:
- Custom Kubernetes CRD handlers for Clouddriver
- these custom handlers enable you to execute custom code when evaluating a manifest’s stability
- there are not yet any official extension points, but at least these Spring-based plugins enable you to add your own CRD handlers without having to fork your own release of Clouddriver
- Custom Resources for Fiat
- Fiat is the authorization server for Spinnaker
- written as a Spring plugin currently, may soon use a Resource extension point that enables serving permissions for a new resource type, such as read, write, and execute permissions for a “File” resource demonstrated in this example
Cloud Providers
Cloud providers, in the context of Clouddriver, refer to target platforms where Spinnaker can deploy workloads, such as Kubernetes, EC2, Cloud Foundry, etc. Before the plugin framework, adding a new cloud provider was a notoriously difficult task. We’re expecting to see more of these cloud providers developed as plugins, as these are easier for those with less familiarity with Spinnaker’s codebase.
An example of how a cloud provider plugin might look, at least for the back-end, is illustrated by this one for Hashi Nomad. Some key components:
- Caching agents, which are how Spinnaker checks for current state of any particular cloud provider resources
- Atomic operators, for updates to these cloud provider resources
- Searchable provider, for finding a specific set of resources
A complete cloud provider plugin will also add custom stages (e.g. a custom Deploy stage) and Deck updates to aid with visualizing new types of cloud resources.
Integrations
Plugins also enable more integration points with other tools. This is an improvement over older means of integrating with these tools, such as the
Armory Jira stage (which only enables auto-updates to Jira from Spinnaker, not the other way around) or the
echo REST webhook, e.g:
- datadogEventListener (which extends EventListener ) showcases how plugins offer more control over events sent to a downstream events aggregator (Datadog, in this case), so you can add filtering and transformation logic
- this observability plugin enables customizing the Micrometer registry and exposes an OpenMetrics endpoint for the Micrometer/Spectator metrics, which allows tools such as Prometheus or the New Relic OpenMetrics integration to work without the Spinnaker Monitoring Daemon
- integrations with notification systems (those which don’t appear in the notifications drop-down in the stage configuration) can be enabled via a plugin like this one
How Do I Get Started Building Or Using A Plugin?
To get started building a plugin, check out this video tutorial on building a plugin using the plugin framework.
Interested in partnering with us on building a plugin? Reach out to us!
If you’re interested in browsing Spinnaker Plugins, check out a few popular ones here.
What’s Next For The Plugin Framework?
Currently Spinnaker 1.20.6 and 1.21+ support plugins via the Plugin Framework, with both server and frontend components.
In the meantime, we are continuing to invest in the Plugin Framework to continue to make lean core Spinnaker a reality. While the Plugin Framework is community-owned, Armory is still dedicated towards enhancing Spinnaker iteratively, and we deeply appreciate your feedback. Issues and PRs are very welcome!
Interested in learning more about the Plugin Framework? Reach out to us here to visit #plugins on Spinnaker Slack – we’d love to chat!
