Secrets, Secrets Are No Fun, When You Share with Everyone

Nov 6, 2019 by Kathryn Lewis

Motivation: Spinnaker’s Dirty Little Secret about Secrets

Passwords and tokens (aka “secrets”) are a fact of life in software development, and operating Spinnaker involves managing configs with lots and lots of secrets. Secrets can be a challenge to manage, but tend to be less work than managing a major data breach. Leaked secrets lead to compromises in user privacy, substantial fines, losses in revenue, and damage to public trust and confidence.

More facts of life in software development are version upgrades, changing infrastructure, and the need to maintain version histories of all of these configurations. Source control systems such as Git are excellent solutions for tracking configuration changes and automatically maintaining version histories. However, storing passwords in a GitHub repository is a very dangerous practice, even if it is a private repo. That is, unless you like accidentally sharing your secrets with the entire world…

What not to do…

Until recently, Spinnaker hasn’t had a built-in secrets management system. As a result, users had to construct their own home-brewed solutions for sharing and versioning Spinnaker configurations, or dangerously leave those secrets in a Git repository. Typically, this home-brew solution was less than robust, and was designed as a temporary solution until they could implement a more robust security solution. But then other tasks and product features take priority, and months or years later their temporary security solution was not so temporary or secure.

Implementation Basics: The Secrets to Keeping Secrets Secret

To solve this problem, we’ve added an end-to-end Secrets Management solution to Spinnaker! Secrets can be stored in encrypted S3 buckets (available in OSS and Armory Spinnaker), or in Vault (exclusive to Armory Spinnaker). The solution is extensible, so we can easily add new secret stores – reach out to us here or on Spinnaker Slack to let us know which secret stores you would like to see added.

Spinnaker users simply replace the secret value in their halconfig and/or service yamls with a syntax (described here and here) that tells Spinnaker where to fetch the secret.

Example of the Secrets syntax

The implementation uses role-based access control (RBAC) for permissions, so you simply need to ensure that Halyard and the Spinnaker services are running with roles that can access your chosen secret store.

Operation: The Secret Sauce

Halyard will read in all config values from ~/.hal/config and will decrypt a secret any time it needs the actual value for a task, such as account validation. Halyard stores the value in memory or as a temporary file on disk for any secret that needs to be in file format (e.g., kubeconfig, SAML keystore files, etc.)

When it comes time to create the profiles for each service (found under ~/.hal/default/staging/), Halyard reverts back to using the encrypted syntax and cleans up temporary files. Finally, each Spinnaker service gets deployed, starts up, and decrypts the secrets found in its config files.

Yay! Celebrating Not-So-Secretly

With Armory’s Secrets Management solution, nowhere, in any configuration file, are you required to have a secret in plain text. You can now safely share and version your Spinnaker configurations for faster, more reliable, more secure deployments.

Interested in learning more about Secrets Management with Spinnaker? Reach out to us here or on Spinnaker Slack – we’d love to chat!

Note: This process applies to configuration secrets for Spinnaker itself (GitHub tokens, AWS keys, SAML keystores, etc.), not secrets for the applications Spinnaker is deploying.

Recently Published Posts

3 Common Spinnaker Challenges (and Easy Ways to Solve Them)

Sep 27, 2022

Spinnaker is the most powerful continuous delivery tool on the market.  DevOps engineers and developers recognize this power and are looking to use Spinnaker as a foundational tool in their Continuous Integration and Continuous Delivery (CI/CD) process for hybrid and multi-cloud deployments. Such a powerful, expansive open source tool needs expertise within your organization to […]

Read more

Streamline Advanced Kubernetes Deployments from GitHub Actions with New Armory Service

Sep 23, 2022

Today, Armory is excited to announce the availability of the GitHub Action for Armory Continuous Deployment-as-a-Service. GitHub is where developers shape the future of software. After a developer writes and tests their code in GitHub, it must be deployed. Armory’s GitHub Action for Continuous Deployment-as-a-Service extends the best-in-class deployment capabilities to Kubernetes. CD-as-a-Service enables declarative […]

Read more

When everyone is facing the same headwind, go on the offensive

Sep 12, 2022

Call me Pollyanna, but what a great time to be a Platform or DevOps engineer. If you’re working in a public company, the S&P is off ~20% year over year, so the value of your RSUs has wilted. If you’re working in a private company, venture funding and M&A velocity are anemic, making expansion capital […]

Read more