Skip to main content

Introducing the Armory Agent for Kubernetes

We’re uncovering the Armory Agent for Kubernetes: a more flexible, scalable, and secure way for Spinnaker to interact with your Kubernetes infrastructure.

Why We Wrote the Agent

Spinnaker monitors your Kubernetes infrastructure constantly so it can display the infrastructure to users, know how to version your services, and more. Spinnaker reaches out to each cluster, retrieves all the Kubernetes objects it is configured to, computes various relationships, and stores the result in its cache.

While powerful, this unobtrusive discovery mechanism has given rise two types of issues when handling a larger number (100s or 1000s) of Kubernetes clusters: performance and account management.

Caching Performance

With a large number of clusters, performance can be degraded as the same data is retrieved over and over – even when nothing has changed. Spinnaker repeats the process for each cluster and each namespace1. The Agent takes a different approach, listening to changes at the cluster level and streaming them back to Spinnaker. Changes to your infrastructure – whether Spinnaker initiated or not – show up in “real time” in Spinnaker’s cache, all over a single TCP connection per cluster.

Using the Cache

The Agent also comes with an optimized storage that allows some nice performance improvements – made possible by adopting Kubernetes and RDBMS-specific storage.

Below are some examples we saw during the development of the agent:

Account Management

In its Agent mode, the Agent simplifies Kubernetes account management within Spinnaker. Instead of a centralized account management model, where all of your Kubernetes service accounts and credentials are stored within Spinnaker, the Agent sits within each Kubernetes cluster and manages accounts from there. This enables individual teams to manage credentials and permissions for their respective clusters, and allows you to add or delete new accounts on the fly without restarting Spinnaker.

Added Bonus: Enhanced Security

The Agent provides an added bonus by enhancing security. It allows you to keep your Kubernetes API servers private from Spinnaker, and restricts the information leaving the cluster to only what Spinnaker needs to operate. Individual teams can also easily rotate or remove Kubernetes configs as needed.

 

Getting Started with the Agent

The Agent acts as Spinnaker’s gateway to Kubernetes. It comes in two parts: a lightweight Golang program that gives you a lot of deployment flexibility – we’ll get back to that shortly – and a plugin to Spinnaker. Let’s set it up in our Spinnaker installation:

Installing the Agent is straightforward, especially if you already use the Spinnaker Operator. Follow the quick start, and deploy:

  • a plugin to Clouddriver: this will be what the Agent will connect to. A new (gRPC) port will be open.
  • a Service that points to Clouddriver’s gRPC port.
  • a Deployment of the Agent.

With the above configuration, you’d just define accounts in the Agent – not in Clouddriver anymore. With a simple bit of yaml, a HELM chart, or a kustomize template, teams can inject the Agent  into newly provisioned Kubernetes clusters and immediately make them a software deployment targets. Like other parts of Spinnaker, the Agent fully implements secret management so you can store your kubeconfig files in one of the supported secret engines or just provision them via the method of your choice as Kubernetes secrets.

The Agent will also detect configuration changes and reload the list of accounts immediately. So we could keep updating agent accounts in a separate configMap.

Different Topologies

The great thing with cloud providers – or with a dose of infrastructure automation – is that it’s easy to spin up new Kubernetes clusters. We see application teams more and more in charge of their own infrastructure. In this context, deploying with Spinnaker often becomes a game of cat and mouse to get your cluster information into the hands of the ops team.

We’ve seen a lot of tooling developed over the years. What if we could skip that step? Behold! That’s why it’s called Armory Agent for Kubernetes

“for” Kubernetes is because lawyers and trademark.

In this mode, the Agent is installed in the deployment target cluster by the provisioning tool (e.g. Terraform) – alongside your monitoring and logging infrastructure – or separately via scripts.

The Agent is also configured with Spinnaker’s endpoint and the proper authentication (mTLS, JWT, …). When it starts, it will register to Spinnaker and the account will automatically be available for caching and deployment.

Conversely, if the cluster is deleted – or if you just uninstall the Agent, the account will be removed from Spinnaker after a grace period.

Armory Cloud

I’d be remiss if I didn’t mention Armory Cloud, Armory’s new SaaS offering. It is a big reason we doubled down on the Agent. With the Agent and the model above, Armory can help you deploy to your infrastructure (public or private), does not need any credentials, and can be revoked at anytime by customers.

A different strategy is to manage groups of clusters. This can be useful in an environment where the control plane does not have direct access to your cluster but cluster information is still centralized. But it can also be helpful to group clusters geographically close, belonging to the same business unit, or with a similar network topology.

 

If you’re interested, please reach out. We’re in early release and now is your time to help shape the deployment service of your dreams.

Release / Status

The Agent is currently in Early Release and is being used by select Armory customers. We’re encouraged with our early results and are iterating quickly. The Agent is compatible with Armory Spinnaker as well as open source Spinnaker, so please reach out if you want to take it for a spin. Also, join us for a lunch & learn about the Agent on 11/4 during the Spinnaker Summit and Gardening Days.

Footnotes:

1. Fortunately, Spinnaker is highly instrumented and if you know what metrics to watch for, there is surely a knob to turn to correct it. Note that there is an effort currently underway to improve the situation and we expect Spinnaker caching performance to greatly improve over the next few releases!

 

May 11, 2021
|
by Nikema Prophet

#SpeedOfLife First Week Recap

It Begins We started the #SpeedOfLife campaign last week, yay! It has been a bumpy start but we’re happily moving forward. The first person I spoke with about delivering software at the speed of life was our CEO at Armory, DROdio. I posted a snippet of our conversation on Twitter last week and here that […]

Read more

May 4, 2021
|
by Nikema Prophet

What Does It Mean To Deliver Software At The #SpeedOfLife?

In this blog post, I’m excited to announce the #SpeedOfLife campaign that I will be leading for the month of May. I’m looking forward to getting to know more of the awesome people who make up our DevOps community. What is #SpeedOfLife? #SpeedOfLife is a social campaign meant to highlight users and experts in the […]

Read more

April 23, 2021
|
by Chad Tripod

Armory Enterprise install in 10 minutes

I was introduced to Armory and the open-source project Spinnaker a little over 18 months ago.  Up to that point, I’d worked with numerous open-source and proprietary software and was a specialist in application delivery and data analytics.  I immediately found out that Spinnaker was the continuous delivery platform of choice for many large scale […]

Read more