We’re uncovering the Armory Agent for Kubernetes: a more flexible, scalable, and secure way for Spinnaker to interact with your Kubernetes infrastructure.
Why We Wrote the Agent
Spinnaker monitors your Kubernetes infrastructure constantly so it can display the infrastructure to users, know how to version your services, and more. Spinnaker reaches out to each cluster, retrieves all the Kubernetes objects it is configured to, computes various relationships, and stores the result in its cache.
While powerful, this unobtrusive discovery mechanism has given rise two types of issues when handling a larger number (100s or 1000s) of Kubernetes clusters: performance and account management.
Caching Performance
With a large number of clusters, performance can be degraded as the same data is retrieved over and over – even when nothing has changed. Spinnaker repeats the process for each cluster and each namespace1. The Agent takes a different approach, listening to changes at the cluster level and streaming them back to Spinnaker. Changes to your infrastructure – whether Spinnaker initiated or not – show up in “real time” in Spinnaker’s cache, all over a single TCP connection per cluster.
Using the Cache
The Agent also comes with an optimized storage that allows some nice performance improvements – made possible by adopting Kubernetes and RDBMS-specific storage.
Below are some examples we saw during the development of the agent:
Account Management
In its Agent mode, the Agent simplifies Kubernetes account management within Spinnaker. Instead of a centralized account management model, where all of your Kubernetes service accounts and credentials are stored within Spinnaker, the Agent sits within each Kubernetes cluster and manages accounts from there. This enables individual teams to manage credentials and permissions for their respective clusters, and allows you to add or delete new accounts on the fly without restarting Spinnaker.
Added Bonus: Enhanced Security
The Agent provides an added bonus by enhancing security. It allows you to keep your Kubernetes API servers private from Spinnaker, and restricts the information leaving the cluster to only what Spinnaker needs to operate. Individual teams can also easily rotate or remove Kubernetes configs as needed.
Getting Started with the Agent
The Agent acts as Spinnaker’s gateway to Kubernetes. It comes in two parts: a lightweight Golang program that gives you a lot of deployment flexibility – we’ll get back to that shortly – and a plugin to Spinnaker. Let’s set it up in our Spinnaker installation:
Installing the Agent is straightforward, especially if you already use the Spinnaker Operator. Follow the quick start, and deploy:
- a plugin to Clouddriver: this will be what the Agent will connect to. A new (gRPC) port will be open.
- a
Servicethat points to Clouddriver’s gRPC port. - a
Deploymentof the Agent.
With the above configuration, you’d just define accounts in the Agent – not in Clouddriver anymore. With a simple bit of yaml, a HELM chart, or a kustomize template, teams can inject the Agent into newly provisioned Kubernetes clusters and immediately make them a software deployment targets. Like other parts of Spinnaker, the Agent fully implements secret management so you can store your kubeconfig files in one of the supported secret engines or just provision them via the method of your choice as Kubernetes secrets.
The Agent will also detect configuration changes and reload the list of accounts immediately. So we could keep updating agent accounts in a separate configMap.
Different Topologies
The great thing with cloud providers – or with a dose of infrastructure automation – is that it’s easy to spin up new Kubernetes clusters. We see application teams more and more in charge of their own infrastructure. In this context, deploying with Spinnaker often becomes a game of cat and mouse to get your cluster information into the hands of the ops team.
We’ve seen a lot of tooling developed over the years. What if we could skip that step? Behold! That’s why it’s called Armory Agent for Kubernetes
“for” Kubernetes is because lawyers and trademark.
In this mode, the Agent is installed in the deployment target cluster by the provisioning tool (e.g. Terraform) – alongside your monitoring and logging infrastructure – or separately via scripts.
The Agent is also configured with Spinnaker’s endpoint and the proper authentication (mTLS, JWT, …). When it starts, it will register to Spinnaker and the account will automatically be available for caching and deployment.
Conversely, if the cluster is deleted – or if you just uninstall the Agent, the account will be removed from Spinnaker after a grace period.
Armory Cloud
I’d be remiss if I didn’t mention Armory Cloud, Armory’s new SaaS offering. It is a big reason we doubled down on the Agent. With the Agent and the model above, Armory can help you deploy to your infrastructure (public or private), does not need any credentials, and can be revoked at anytime by customers.
A different strategy is to manage groups of clusters. This can be useful in an environment where the control plane does not have direct access to your cluster but cluster information is still centralized. But it can also be helpful to group clusters geographically close, belonging to the same business unit, or with a similar network topology.
If you’re interested, please reach out. We’re in early release and now is your time to help shape the deployment service of your dreams.
Release / Status
The Agent is currently in Early Release and is being used by select Armory customers. We’re encouraged with our early results and are iterating quickly. The Agent is compatible with Armory Spinnaker as well as open source Spinnaker, so please reach out if you want to take it for a spin. Also, join us for a lunch & learn about the Agent on 11/4 during the Spinnaker Summit and Gardening Days.
Footnotes:
1. Fortunately, Spinnaker is highly instrumented and if you know what metrics to watch for, there is surely a knob to turn to correct it. Note that there is an effort currently underway to improve the situation and we expect Spinnaker caching performance to greatly improve over the next few releases!
