Secrets, Secrets Are No Fun, When You Share with Everyone
Nov 6, 2019 by Kathryn Lewis
Motivation: Spinnaker’s Dirty Little Secret about Secrets
Passwords and tokens (aka “secrets”) are a fact of life in software development, and operating Spinnaker involves managing configs with lots and lots of secrets. Secrets can be a challenge to manage, but tend to be less work than managing a major data breach. Leaked secrets lead to compromises in user privacy, substantial fines, losses in revenue, and damage to public trust and confidence.
More facts of life in software development are version upgrades, changing infrastructure, and the need to maintain version histories of all of these configurations. Source control systems such as Git are excellent solutions for tracking configuration changes and automatically maintaining version histories. However, storing passwords in a GitHub repository is a very dangerous practice, even if it is a private repo. That is, unless you like accidentally sharing your secrets with the entire world…
Until recently, Spinnaker hasn’t had a built-in secrets management system. As a result, users had to construct their own home-brewed solutions for sharing and versioning Spinnaker configurations, or dangerously leave those secrets in a Git repository. Typically, this home-brew solution was less than robust, and was designed as a temporary solution until they could implement a more robust security solution. But then other tasks and product features take priority, and months or years later their temporary security solution was not so temporary or secure.
Implementation Basics: The Secrets to Keeping Secrets Secret
To solve this problem, we’ve added an end-to-end Secrets Management solution to Spinnaker! Secrets can be stored in encrypted S3 buckets (available in OSS and Armory Spinnaker), or in Vault (exclusive to Armory Spinnaker). The solution is extensible, so we can easily add new secret stores – reach out to us here or on Spinnaker Slack to let us know which secret stores you would like to see added.
The implementation uses role-based access control (RBAC) for permissions, so you simply need to ensure that Halyard and the Spinnaker services are running with roles that can access your chosen secret store.
Operation: The Secret Sauce
Halyard will read in all config values from
~/.hal/config and will decrypt a secret any time it needs the actual value for a task, such as account validation. Halyard stores the value in memory or as a temporary file on disk for any secret that needs to be in file format (e.g., kubeconfig, SAML keystore files, etc.)
When it comes time to create the profiles for each service (found under
~/.hal/default/staging/), Halyard reverts back to using the encrypted syntax and cleans up temporary files. Finally, each Spinnaker service gets deployed, starts up, and decrypts the secrets found in its config files.
Yay! Celebrating Not-So-Secretly
With Armory’s Secrets Management solution, nowhere, in any configuration file, are you required to have a secret in plain text. You can now safely share and version your Spinnaker configurations for faster, more reliable, more secure deployments.
Note: This process applies to configuration secrets for Spinnaker itself (GitHub tokens, AWS keys, SAML keystores, etc.), not secrets for the applications Spinnaker is deploying.
Recently Published Posts
Interview with Guillermo Huerta for National IT Professionals Day
To celebrate National IT Professionals Day, we’re sharing an interview I had with Armory’s Head of IT and Workplace, Guillermo Huerta. The interview is broken into seven parts. There’s a video and transcript for each section. Introductions Nikema Prophet: [00:00:00] I’m Nikema I am the Associate Community Marketing Manager at Armory and I’m talking to […]
Read more →
Interview with Daniel Ko — Google Summer of Code 2021
I’m personally excited about this year’s Google Summer of Code (GSoC) project try.spinnaker.io. Daniel Ko is the student who worked with Armory mentors and the Continuous Delivery Foundation to complete the project. After his midterm demo, I asked Daniel some questions about the project and why he chose it. Nikema Prophet: What inspired you to […]
Read more →
Amazon EKS Anywhere + Armory
Armory is excited to share we are an Amazon EKS Anywhere launch partner. Amazon EKS Anywhere is a new deployment option for Amazon EKS that enables you to easily create and operate Kubernetes clusters on-premises, including on your virtual machines and bare metal servers. Armory enables enterprises to unlock innovation by reliably deploying software at […]
Read more →