Safer Kubernetes Deployments with Spinnaker: With Integrated Solutions for Problems Like Config Management and Service Discovery
May 17, 2018 by Ethan Rogers
Kubernetes has fundamentally changed how we view applications that run in the cloud. With integrated solutions for problems like Config Management and Service Discovery, Kubernetes provides a relatively holistic approach to running cloud-native applications. One common misconception about Kubernetes is that it’s a deployment tool, which is it not.
Kubernetes is a portable container “cloud” that can live anywhere and creates a standard interface for infrastructure, similar to how AWS creates a standard interface for their data centers via tools like EC2.
One of the most significant problems affecting the community today is deployment. While getting applications running on Kubernetes is somewhat simple, managing the promotion, update and lifecycle of these applications in a safe and repeatable way is a problem that the Kubernetes community is just starting to discover. Safety and reliability are two critical aspects of Continuous Delivery and Spinnaker is precisely positioned to solve this problem for Kubernetes. Kubernetes does an excellent job at maintaining the availability of your application, but it isn’t so great at ensuring that your applications are functioning as they should and leave implementing those requirements as an exercise of the user. In this post, I’ll explain how the new Kubernetes provider for Spinnaker seeks to solve this problem and aims to make your Kubernetes deployments even safer.
Deploying to Kubernetes
Most applications deployed on Kubernetes tend to follow a similar pattern. First, the running application is represented as a
Deployment, defining the containers, their environment variables and the compute resources they require. This
Deployment references a
ConfigMap as a way of sourcing environment variables of other environment’s specific configuration. Each time a new
Pod is started, the current state of these resources mount to the container, and their keys inject where necessary. This
Pod function is really powerful because Kubernetes exposes resources that help us build applications that follow the principles of 12 Factor Apps. One of the limiting factors, however, is that these resources aren’t versioned alongside the
Deployments that use them. Practically, this means that configuration in your
Secret have a high likelihood to be incompatible with previous versions in the event of a rollback.
To help illustrate the point, let’s take a look at an example:
apiVersion: v1 kind: ConfigMap metadata: name: app-config data: FOO: bar --- apiVersion: apps/v1 kind: Deployment metadata: name: my-app spec: template: spec: containers: - name: app image: my-application:1.0.0 ports: - containerPort: 80 envFrom: - configMapRef: name: app-config
In this example, our
my-app) is referencing a
app-config) which contains a list of environment variables that our application depends on, specifically the
FOO environment variable. Imagine a scenario where our application no longer depended on the
FOO environment variable but began depending on a
BAR environment variable. To roll this change out using vanilla Kubernetes, you’d need to update the contents of the
ConfigMap and then apply the
Deployment manifest. If a critical bug surfaces in this deployment, the best way to resolve this is to rollback to the previous version. You could do this by using
kubectl rollout undo but that only reverts the
Deployment. The changes made to the
ConfigMap persist which breaks our application since it’s missing previously required
FOO environment variable.
One of the primary principles of Spinnaker is Immutable Infrastructure. The tools that Spinnaker exposes for your deployments are all pre-disposed to favor this style of application deployment. By following this principle, you can be sure that an application functions just as it did before in the event of a rollback.
Deployment follows the principle of Immutable Infrastructure, the rest of the Kubernetes resources do not, and that is one of the problems that Spinnaker is aiming to solve with the Kubernetes V2 provider.
To solve this problem, Spinnaker inspects every Kubernetes manifest that it deploys. If it detects a non-versioned resource is deploying alongside a versioned resource, it updates the name of that resource to include a version. For example, in our previous scenario, we updated the contents of our
ConfigMap, but the name remained the same (
app-config). Now, when we change our
ConfigMap, Spinnaker creates a new
ConfigMap to represent that change. So, if we start out with
app-config-v00, the next change creates
app-config-v001. This change ensures that changes to this resource do not affect previous versions of the application if they were to appear again.
In this way, Spinnaker improves the experience of deploying to Kubernetes by making it safer. To achieve the Immutable principle of Spinnaker, Spinnaker makes sure that each piece of a deployment is versioned such that the whole, each piece of the puzzle, is a single unit that works together. These units, comprised of
ConfigMaps for example, work together to model your application running on Kubernetes.