Skip to main content

Automate Load Testing with Armory Spinnaker and Locust

Oct 16, 2017 by Ben Mappen

The new Load Testing stage is available in Armory Spinnaker. For documentation on how to install Armory Spinnaker, refer here: http://go.armory.io/install

Last week we introduced Certified Pipelines, a tool to help you define and enforce policies to ensure your teams are deploying safely and with best practices. The policies you create are comprised of pipeline stages – simply list the stages required to meet your policy and any pipelines that are missing one of those stages will be blocked from deploying.

We’re adding first class support for specific types of stages that help you automate the steps we believe create a safe deployment. We’re starting with load testing because, from experience, we know the cost of load testing is high which is why they are rarely done.

We also know that there’s huge benefit to doing more frequent load testing. Our goal is to reduce the friction associated with doing load testing such that it’s easy for everyone to add to their pipelines.

Adding a Locust.io Load Testing Stage

The load testing stage currently supports Locust.io, a simple, open-source load testing utility (we’ll be adding support for other tools shortly). To add a Locust Load Testing stage, simply select “Locust Load Test” from the stages dropdown menu in your pipeline configuration view.

Automate Load Testing using Canaries

Once you’ve configured your load testing stage, the next step is automation. To accomplish that, we’ll leverage automated canary analysis using Barometer.

  1. Define load tests. Configure the number of clients, hatch rate, duration, and host.
  2. Add a Barometer stage. The automation is enabled by Barometer, an ACA service for Spinnaker. You’ll need to add the Barometer ACA stage directly after the Load Testing stage. When configuring the Barometer stage, just make sure you select the cluster associated with the load testing environment.
  3. Define application metrics. Within Barometer, define which application metrics you want to measure, ie. response times, CPU, memory.
  4. Stand up environment. Create an environment to perform the load testing in.
  5. Deploy a baseline and canary. Within the load testing environment, standup a baseline (the existing code) and a canary (the new code).
  6. Run load tests. We’ll run the Locust load tests against both the baseline & the canary to determine if the new code is good or bad.
  7. Determine if the tests pass or fail. If the canary’s metrics are within a reasonable threshold to the baseline, continue the pipeline, otherwise fail it.

Roadmap

We have plans to add support for Gatling and Jmeter. Let us know below if there are other load testing tools you’d like to see us add support for.

Learn More

And if you are interested in trying our Load Testing stage, just drop us a note below or install Armory Spinnaker yourself here.

Recently Published Posts

Welcoming 2022: Reflecting and looking forward

Dec 22, 2021
|
by Jim Douglas

Nearly all cultures globally have some form of celebration marking the Winter Solstice. Common threads found in most observances of the annual event are celebration of family and friends (living and past), reflection of the past year, and some form of giving thanks for continued health and sustenance. Exiting 2021, said celebrations would seem especially […]

Read more

Resiliency and Load distribution

Dec 16, 2021
|
by Daniel Gonzalez

Introduction When scaling a network service, there are always two concerns: resiliency and load distribution, to understand these concepts let us first understand the broader term “Redundancy”. Redundancy is the duplication of a component to increase reliability of the system, usually in the form of a backup, fail-safe, or to improve actual system performance. Resiliency […]

Read more

CVE-2021-44228 – log4j (Log4Shell) – an analysis

Dec 10, 2021
|
by Jason McIntosh

Today marked a 0-day disclosure of a rather nasty vulnerability in one of the most commonly used frameworks for logging – log4j.  This one is nasty on multiple levels.  Note that Armory Enterprise is NOT affected by this vulnerability.  The impact on this vulnerability is likely huge and is already being exploited.  Additionally it can […]

Read more