Skip to main content

Spinnaker, Accounts, and Kubernetes: How Accounts in Spinnaker Work and How Those Accounts Authenticate with the Kubernetes

I’ve recently had the opportunity to work with many customers who are starting their journey into Kubernetes and using Spinnaker to facilitate that process. One of the most significant points of confusion is around how accounts in Spinnaker work and how those accounts authenticate with the Kubernetes API. In this post, I hope to clear up any confusion around the subject!

I’ve written about this subject in the past so if you’d like to see an example be sure to check out this post on the Spinnaker community blog.

Accounts

The first thing we should cover is the concept of Accounts. Accounts are Spinnaker’s way of defining a set of credentials for a particular cloud provider. Since Spinnaker was initially designed around AWS, let’s use it as an example. For the majority of AWS users, it’s common practice to split your cloud footprint into separate logical AWS accounts which allows you to separate infrastructure between environments. In the most simple use case, you may have an account for Development and an account for Production. More complex organizations, however, may have tens to hundreds of accounts depending on security or organizational requirements.

With this understanding in place, we can start to look at how Accounts relate to Kubernetes. Typically, Kubernetes users split workloads across various clusters. Much like the previous example, splitting workloads across physical clusters provides a logical separation of resources and prevents outages in one cluster to affecting other environments. These physical clusters, then, effectively serve the same purpose as an AWS account.

Authentication and Kubeconfigs

An Account within Spinnaker isn’t solely about separating resources. It’s important to note that an Account represents a set of credentials for a given cloud provider and that your users interact with that cloud provider through the account. In other words, the Account is acting on behalf of your users. Whenever Spinnaker performs an action within your cloud provider, such as deploying a container or creating a load balancer, it uses credentials defined by the Account to do so.

Since an Account effectively acts as a user, we need to consider how we might enable the account to authenticate with our cloud provider. In the Kubernetes world, we use something called a kubeconfig which allows us to define a list of clusters that we can communicate with and the various ways we can authenticate as a particular user. A single user and cluster are then combined to form a context, which is used by Spinnaker to communicate with the Kubernetes API. There are a multitude of ways we can use to authenticate with our Kubernetes clusters so you should choose a method which suits the needs of your organization. For demo or POC purposes, I like to use a Kubernetes ServiceAccount token paired with a cluster-admin ClusterRole but, for more restrictive or explicit permissions, you should use Kubernetes RBAC to define a specific set of actions this user is allowed to perform.

Each Kubernetes account within Spinnaker defines the kubeconfig that it will use when authenticating with the Kubernetes API. For small shops, you may be able to use a single kubeconfig which lists all of the clusters, users, and contexts and use the --context option to specify the context when configuring accounts. For larger shops with many accounts, this approach may prove difficult to manage so you may prefer to use a single kubeconfig file per account.

RBAC

Kubernetes RBAC is a powerful tool for restricting the actions a user is allowed to perform on your cluster. You can read about it in depth here, but I’ll briefly cover some of the resources you can use to ensure proper permissions for your accounts.

Role and ClusterRole

Roles and ClusterRoles are used to define a list of APIs and actions that the holder of this role is allowed to access. A Role is a namespaced kind within Kubernetes which means that the permissions defined in the Role only apply to the namespace where it is created. A ClusterRole is a non-namespaced kind. This means that ClusterRoles can define permissions that span across all namespaces within a cluster. It has the added benefit of being able to access other non-namespaced kinds such as the Namespacekind itself.

ServiceAccount

ServiceAccounts are used within Kubernetes to provide processes a way to access the Kubernetes API. ServiceAccountsdiffer from User accounts in that they are meant to be used by a machine or application instead of a human user which means they are perfect for our use case.

Once created ServiceAccounts provision a token that can then be used to authenticate with the Kubernetes API. Any request using this token will be subject to the policy defined by the Role (or ClusterRole) that it’s bound to.

RoleBinding and ClusterRoleBinding

A RoleBinding is used to tie a Role to a ServiceAccount. Similarly, a ClusterRoleBinding is used to tie a ClusterRole to a ServiceAccount. For example, let’s consider the scenario where you want to give a ServiceAccountpermissions across multiple namespaces, but not all. In this case, you would begin by creating a ServiceAccount in one namespace and then creating a Role and RoleBinding in each namespace you’d like the ServiceAccount to have access to.

By using these Kubernetes objects, you can easily create policies for Spinnaker that allow as much or as little access to the Kubernetes API as you like. For example, if you’ve created a Role that only allows the user to create Deployments and Services, a user within Spinnaker will not be allowed to deploy anything else. If they attempt to deploy a StatefulSet, for example, it will fail.

Spinnaker RBAC

Finally, Spinnaker provides the ability to add RBAC to Accounts using Fiat. While not as rich as Kubernetes RBAC, you are still able to limit read and write access to Accounts and Applications using external systems like LDAP or Okta (SAML). To do this, you should define the groups who are allowed to use a particular Account when configuring them. Users will only be able to select and use the Accounts which match their particular set of groups. For example, the following Account configuration only permits users in the engineering team read/write access but restricts users in the qa team to read-only. This can be useful if you want to allow certain teams to see what’s deployed but disallow them from making any changes.

accounts:- name: dev-environment
  kubeconfigFile: /some/path/to/kubeconfig.yml
  context: dev-environment
  permissions:
    read:
      - engineering
      - qa
    write:
-  engineering

Conclusion

How you configure Accounts is largely dependent on the needs of your organization. By using the tools detailed in this post, you’ll be able to define a set of accounts that should suit your needs. Accounts provide a powerful abstraction that allows you to slice-and-dice your Kubernetes clusters into distinct areas for each team that uses them and adds a layer of multi-tenancy that makes it even easier to expand the use of Kubernetes within an organization. RBAC provides a way to ensure this type of multi-tenancy by defining an explicit set of actions that users are allowed to perform.

May 4, 2021
|
by Nikema Prophet

What Does It Mean To Deliver Software At The #SpeedOfLife?

In this blog post, I’m excited to announce the #SpeedOfLife campaign that I will be leading for the month of May. I’m looking forward to getting to know more of the awesome people who make up our DevOps community. What is #SpeedOfLife? #SpeedOfLife is a social campaign meant to highlight users and experts in the […]

Read more

April 23, 2021
|
by Chad Tripod

Armory Enterprise install in 10 minutes

I was introduced to Armory and the open-source project Spinnaker a little over 18 months ago.  Up to that point, I’d worked with numerous open-source and proprietary software and was a specialist in application delivery and data analytics.  I immediately found out that Spinnaker was the continuous delivery platform of choice for many large scale […]

Read more

April 21, 2021
|
by Nikema Prophet

#NikemaLearns About AWS – #4 Officially AWS Certified

I’m an AWS Certified Cloud Practitioner! I took the exam and I passed. I fell behind on blog updates starting about a month ago but I studied up until the day before the test. I was somewhat disappointed in myself for not keeping my schedule but in the end, I passed and that was the […]

Read more