Automate Load Testing with Armory Spinnaker and Locust

Oct 16, 2017 by Ben Mappen

The new Load Testing stage is available in Armory Spinnaker. For documentation on how to install Armory Spinnaker, refer here: http://go.armory.io/install

Last week we introduced Certified Pipelines, a tool to help you define and enforce policies to ensure your teams are deploying safely and with best practices. The policies you create are comprised of pipeline stages – simply list the stages required to meet your policy and any pipelines that are missing one of those stages will be blocked from deploying.

We’re adding first class support for specific types of stages that help you automate the steps we believe create a safe deployment. We’re starting with load testing because, from experience, we know the cost of load testing is high which is why they are rarely done.

We also know that there’s huge benefit to doing more frequent load testing. Our goal is to reduce the friction associated with doing load testing such that it’s easy for everyone to add to their pipelines.

Adding a Locust.io Load Testing Stage

The load testing stage currently supports Locust.io, a simple, open-source load testing utility (we’ll be adding support for other tools shortly). To add a Locust Load Testing stage, simply select “Locust Load Test” from the stages dropdown menu in your pipeline configuration view.

Automate Load Testing using Canaries

Once you’ve configured your load testing stage, the next step is automation. To accomplish that, we’ll leverage automated canary analysis using Barometer.

  1. Define load tests. Configure the number of clients, hatch rate, duration, and host.
  2. Add a Barometer stage. The automation is enabled by Barometer, an ACA service for Spinnaker. You’ll need to add the Barometer ACA stage directly after the Load Testing stage. When configuring the Barometer stage, just make sure you select the cluster associated with the load testing environment.
  3. Define application metrics. Within Barometer, define which application metrics you want to measure, ie. response times, CPU, memory.
  4. Stand up environment. Create an environment to perform the load testing in.
  5. Deploy a baseline and canary. Within the load testing environment, standup a baseline (the existing code) and a canary (the new code).
  6. Run load tests. We’ll run the Locust load tests against both the baseline & the canary to determine if the new code is good or bad.
  7. Determine if the tests pass or fail. If the canary’s metrics are within a reasonable threshold to the baseline, continue the pipeline, otherwise fail it.

Roadmap

We have plans to add support for Gatling and Jmeter. Let us know below if there are other load testing tools you’d like to see us add support for.

Learn More

And if you are interested in trying our Load Testing stage, just drop us a note below or install Armory Spinnaker yourself here.

Recently Published Posts

A Faster Way to Evaluate Self-Hosted Continuous Deployment from Armory

Sep 30, 2022

Introducing Quick Spin One of the most common challenges that organizations face when implementing a continuous deployment strategy is the time and focus that it takes to set up the tools and processes. But a secure, flexible, resilient and scalable solution is available right now. Want to see if it’s the right tool for your […]

Read more

3 Common Spinnaker Challenges (and Easy Ways to Solve Them)

Sep 27, 2022

Spinnaker is the most powerful continuous delivery tool on the market.  DevOps engineers and developers recognize this power and are looking to use Spinnaker as a foundational tool in their Continuous Integration and Continuous Delivery (CI/CD) process for hybrid and multi-cloud deployments. Such a powerful, expansive open source tool needs expertise within your organization to […]

Read more

Streamline Advanced Kubernetes Deployments from GitHub Actions with New Armory Service

Sep 23, 2022

Today, Armory is excited to announce the availability of the GitHub Action for Armory Continuous Deployment-as-a-Service. GitHub is where developers shape the future of software. After a developer writes and tests their code in GitHub, it must be deployed. Armory’s GitHub Action for Continuous Deployment-as-a-Service extends the best-in-class deployment capabilities to Kubernetes. CD-as-a-Service enables declarative […]

Read more