Oct 14, 2022 by Stephen Atwell
On the OpenCost slack, I keep seeing questions from users struggling to get the openCost UI to run in-cluster. While we wait for a Docker image that bundles the UI to become available, I want to share the configuration I use to run the UI within a Kubernetes cluster.
My configuration is located in this git repository. This blog documents the configuration, how to use it, and why it works the way it does.
This repository deploys:
The example Configuration uses Kustomize in order to both inject a UI container, and to simplify changing configuration variables. Any configuration change to OpenCost’s allocation engine or UI that is done via environment variable can be specified in the kustomization.yml file.
This repository can be used standalone and applied via kubectl, or it can be deployed with Armory CD-as-a-Service. When using CD-as–a-Service, it uses prometheus queries during upgrade to validate that the OpenCost Prometheus queries function. The intention of this validation is to decrease the risk of upgrading to the latest version.
There is currently no docker image that contains the UI. The provided configuration deploys it by using a Docker image that contains git and npm. This image is configured to checkout the OpenCost repo and start the UI automatically. This is suboptimal since it is always grabbing the latest UI from git every-time the pod starts. Once a Docker image is available that contains the UI, I would recommend switching to it instead so that the pod is immutably defined.
To port forward both the cost model and the UI, run the following commands while connected to the Kubernetes cluster to which you have deployed OpenCost.
kubectl -n=opencost port-forward service/opencost-ui 1234
kubectl -n=opencost port-forward service/opencost 9090:9003
Currently the OpenCost UI does not support being exposed via an Ingress or a Load Balancer. This is because it is hard coded to reference ‘Localhost’ for the URL that the user’s web browser uses to connect to the OpenCost cost Model. Do the following to expose the UI:
To deploy using kubectl, clone the repo, and then in its root run:
kustomize build --enable-helm . > manifests.yml
Kubectl apply -f manifests.yml
Armory adds automated upgrade testing that ensures Prometheus is ingesting OpenCost cost data, and container memory data. It also simplifies deploying OpenCost across large numbers of clusters.
If you are not already using Armory CD-as-Service:
Now connect the prometheus deployment that this setup will create:
CD-as-a-Service can deploy either from its CLI, or using its Github Action. Instructions for both follow. For either path, start by forking this git repository. After forking, open the deploy.yml file, and replace the ‘account’ name on both environments with the name that you specified in step 2 above.
To deploy using Github Actions you must create a client credential and add it to a Github secret.
The cli can be used to deploy using an interactive login, or called from your CI system using client credentials. Here is how you deploy with an interactive login:
curl -sL go.armory.io/get-cli | bash
armory loginto login through the UI
./deploy.shto deploy. This shell script will build the kustomize files, then use the cli to deploy.
If you want this configuration to deploy to multiple Kubernetes clusters, copy the environments, and give the copies a new name and update the account. If one of your clusters is a staging cluster that should run before the others you can add a dependsOn constraint to the clusters that depend on it.
Example: add a staging cluster, and making production depend on it
#where to deploy code, and in what order. Specifies accounts and namespaces for each application.
staging: #an example new environment
Now when you deploy every cluster will get the update, and run the query validation logic.
If want to use your existing prometheus, make the following changes to your kustomization.yml:
You must also ensure you have added the OpenCost scrape configuration to your prometheus server.
We’ve covered how you can deploy OpenCost to either one or many Kubernetes clusters from a common configuration, and discussed ways to validate its health during deployment, and covered how to get the UI working. We’ve also provided a kustomize-based configuration that allows you to continue inheriting OpenCost’s default configuration while overriding just the portions that you need to change.
Reaction to Apple’s Spinnaker Summit 2022 Talk At the most recent Spinnaker Summit, Joe Cavanagh and Benjamin Powell from Apple discussed how they maximize code reuse, eliminate repository maintenance, and unify their CI/CD process across many plugins. They also discussed the mutual benefits of a well-maintained organizational plugin ecosystem for Spinnaker users, developers, and operators. […]
Read more →
Welcome to the latest release round-up for Armory’s Continuous Deployment Self-Hosted and Managed (CDSH) solution. In this release, we’ve focused on a few quality of life improvements and bug fixes, as well as introduced two user-requested early access features behind a feature flag. Here are the important quality of life and bug fixes in the […]
Read more →
Armory brings cross-environment orchestration, validation, and advanced deployment strategies to AWS Marketplace How fast is fast enough when it comes to deploying your software? Deployment frequency, along with lead time to changes and rollback failure rate, is one of the top metrics of DevOps success. Armory is working to solve the DevOps deployment slow-down through […]
Read more →