Automate Load Testing with Armory Spinnaker and Locust

Oct 16, 2017 by Ben Mappen

The new Load Testing stage is available in Armory Spinnaker. For documentation on how to install Armory Spinnaker, refer here: http://go.armory.io/install

Last week we introduced Certified Pipelines, a tool to help you define and enforce policies to ensure your teams are deploying safely and with best practices. The policies you create are comprised of pipeline stages – simply list the stages required to meet your policy and any pipelines that are missing one of those stages will be blocked from deploying.

We’re adding first class support for specific types of stages that help you automate the steps we believe create a safe deployment. We’re starting with load testing because, from experience, we know the cost of load testing is high which is why they are rarely done.

We also know that there’s huge benefit to doing more frequent load testing. Our goal is to reduce the friction associated with doing load testing such that it’s easy for everyone to add to their pipelines.

Adding a Locust.io Load Testing Stage

The load testing stage currently supports Locust.io, a simple, open-source load testing utility (we’ll be adding support for other tools shortly). To add a Locust Load Testing stage, simply select “Locust Load Test” from the stages dropdown menu in your pipeline configuration view.

Automate Load Testing using Canaries

Once you’ve configured your load testing stage, the next step is automation. To accomplish that, we’ll leverage automated canary analysis using Barometer.

  1. Define load tests. Configure the number of clients, hatch rate, duration, and host.
  2. Add a Barometer stage. The automation is enabled by Barometer, an ACA service for Spinnaker. You’ll need to add the Barometer ACA stage directly after the Load Testing stage. When configuring the Barometer stage, just make sure you select the cluster associated with the load testing environment.
  3. Define application metrics. Within Barometer, define which application metrics you want to measure, ie. response times, CPU, memory.
  4. Stand up environment. Create an environment to perform the load testing in.
  5. Deploy a baseline and canary. Within the load testing environment, standup a baseline (the existing code) and a canary (the new code).
  6. Run load tests. We’ll run the Locust load tests against both the baseline & the canary to determine if the new code is good or bad.
  7. Determine if the tests pass or fail. If the canary’s metrics are within a reasonable threshold to the baseline, continue the pipeline, otherwise fail it.

Roadmap

We have plans to add support for Gatling and Jmeter. Let us know below if there are other load testing tools you’d like to see us add support for.

Learn More

And if you are interested in trying our Load Testing stage, just drop us a note below or install Armory Spinnaker yourself here.

Recently Published Posts

How Armory Uses Makelog to Drive New Feature Adoption from Day One

Jun 23, 2022

original post For teams that practice CI/CD and ship multiple times a day, waiting weeks to communicate these updates to customers defeats the purpose. After all, if a feature ships and nobody knows about it, did it really ship? You miss out on delivering value to customers sooner, learning which features they’re most excited about, […]

Read more

Armory Continuous Deployment-as-a-Service is Now Available

Jun 6, 2022

Over the last two years, our customers and prospects have said one thing: everyone wants Continuous Deployment, but adopting CD to production is too hard for many companies. Today we’re excited to announce that Armory is making production-grade continuous deployment available to all users of Kubernetes, regardless of size. Armory Continuous Deployment-as-a-Service has been in […]

Read more

Introducing OpenCost—An Open Source Project for Monitoring Kubernetes Spend

Jun 2, 2022

We are excited to share the launch of OpenCost, an open source project that provides real-time cost monitoring for teams running Kubernetes workloads.   As container and Kubernetes adoption continues to grow, navigating the complexities around measuring and allocating cost is becoming a business-critical challenge. A recent CNCF survey showed overspend is increasingly a problem for […]

Read more