Skip to main content

Automate Load Testing with Armory Spinnaker and Locust

Oct 16, 2017 by Ben Mappen

The new Load Testing stage is available in Armory Spinnaker. For documentation on how to install Armory Spinnaker, refer here: http://go.armory.io/install

Last week we introduced Certified Pipelines, a tool to help you define and enforce policies to ensure your teams are deploying safely and with best practices. The policies you create are comprised of pipeline stages – simply list the stages required to meet your policy and any pipelines that are missing one of those stages will be blocked from deploying.

We’re adding first class support for specific types of stages that help you automate the steps we believe create a safe deployment. We’re starting with load testing because, from experience, we know the cost of load testing is high which is why they are rarely done.

We also know that there’s huge benefit to doing more frequent load testing. Our goal is to reduce the friction associated with doing load testing such that it’s easy for everyone to add to their pipelines.

Adding a Locust.io Load Testing Stage

The load testing stage currently supports Locust.io, a simple, open-source load testing utility (we’ll be adding support for other tools shortly). To add a Locust Load Testing stage, simply select “Locust Load Test” from the stages dropdown menu in your pipeline configuration view.

Automate Load Testing using Canaries

Once you’ve configured your load testing stage, the next step is automation. To accomplish that, we’ll leverage automated canary analysis using Barometer.

  1. Define load tests. Configure the number of clients, hatch rate, duration, and host.
  2. Add a Barometer stage. The automation is enabled by Barometer, an ACA service for Spinnaker. You’ll need to add the Barometer ACA stage directly after the Load Testing stage. When configuring the Barometer stage, just make sure you select the cluster associated with the load testing environment.
  3. Define application metrics. Within Barometer, define which application metrics you want to measure, ie. response times, CPU, memory.
  4. Stand up environment. Create an environment to perform the load testing in.
  5. Deploy a baseline and canary. Within the load testing environment, standup a baseline (the existing code) and a canary (the new code).
  6. Run load tests. We’ll run the Locust load tests against both the baseline & the canary to determine if the new code is good or bad.
  7. Determine if the tests pass or fail. If the canary’s metrics are within a reasonable threshold to the baseline, continue the pipeline, otherwise fail it.

Roadmap

We have plans to add support for Gatling and Jmeter. Let us know below if there are other load testing tools you’d like to see us add support for.

Learn More

And if you are interested in trying our Load Testing stage, just drop us a note below or install Armory Spinnaker yourself here.

Recently Published Posts

September 21, 2021
|
by Nikema Prophet

Interview with Guillermo Huerta for National IT Professionals Day

To celebrate National IT Professionals Day, we’re sharing an interview I had with Armory’s Head of IT and Workplace, Guillermo Huerta. The interview is broken into seven parts. There’s a video and transcript for each section. Introductions Nikema Prophet: [00:00:00] I’m Nikema I am the Associate Community Marketing Manager at Armory and I’m talking to […]

Read more

September 7, 2021
|
by Nikema Prophet

Interview with Daniel Ko — Google Summer of Code 2021

I’m personally excited about this year’s Google Summer of Code (GSoC) project try.spinnaker.io. Daniel Ko is the student who worked with Armory mentors and the Continuous Delivery Foundation to complete the project. After his midterm demo, I asked Daniel some questions about the project and why he chose it. Nikema Prophet: What inspired you to […]

Read more

September 7, 2021
|
by Armory

Amazon EKS Anywhere + Armory

Armory is excited to share we are an Amazon EKS Anywhere launch partner. Amazon EKS Anywhere is a new deployment option for Amazon EKS that enables you to easily create and operate Kubernetes clusters on-premises, including on your virtual machines and bare metal servers.  Armory enables enterprises to unlock innovation by reliably deploying software at […]

Read more