Horizontal Pod Autoscaling during Kubernetes deployments

Autoscaling, which helps to optimize utilization and minimize costs, is one of the most important aspects of cloud computing. Autoscaling dynamically allocates resources to your cloud application based on load. It handles bursts of load and traffic during peak hours, ensuring that end user experience is not degraded. Autoscaling also guarantees that you don’t have servers running idle and costing you money in the non-peak hours.

Autoscaling in Kubernetes

In the Kubernetes world, there are 3 different autoscalers that help manage costs and resource optimization:

• Cluster Autoscaler scales the number of worker nodes on the Kubernetes cluster. Most cloud Kubernetes providers like EKS, GKE, and AKS support some form of Cluster Autoscaler, but they each have their own “gotchas“.

• Vertical Pod Autoscaler (VPA) manages traffic surges by altering the resource requests and limits on pods, giving pods more resources when needed. This frees up the user from having to set up optimal CPU and memory requests. However, VPA requires pods to restart to apply the new recommended CPU and memory recommendations. Restarting pods may lead to interruption of service to end users of the service.

• Horizontal Pod Autoscaler (HPA) manages the number of pods based on the load. When the load increases, HPA creates identical replicas of the application and distributes load among different pods. The main advantage that HPA gives over VPA is increased reliability. HPA scales a larger number of smaller pods that are packed more efficiently across nodes. This ensures minimal disruption of service should a single node fail. HPA is the only autoscaling resource supported in the standard Kubernetes API.

HPA presents unique challenges when used in conjunction with advanced deployment strategies like Blue/Green. For instance, one of our customers wanted to use a Blue/Green strategy to move all users on the old version to the new version without interruption. However, what our customer discovered was that HPA scaled down the next version as it was not receiving any traffic. This meant that end users saw slow/degraded service for a while when traffic cut over to the next version. When Armory set out to add support for HPA within Continuous Deployments-as-a-Service, we wanted to provide reliability when using deployment strategies without having to increase costs for our customers. Here’s what we came up with.

How Armory’s Continuous Deployment-as-a-Service optimizes HPA during deployment

Step 1: Before starting a deployment to an environment, CD-as-a-Service notes the scale that your application is currently running at and maintains this scale until the deployment to the environment is finished.

The Details: Because CD-as-a-Service knows the scale right before the deployment starts, it can alter deployment strategies in a way that does not cause degraded service. For example, when using a Blue/Green strategy, the next version of the application is scaled up to meet the current traffic load before traffic is moved over to the new version. Once the traffic is cut over to the new version, your users would not experience interruptions or degraded service. Additionally, this approach ensures reliability and speed when rolling back – your rollbacks don’t have to scale up to support the required traffic.

Step 2: CD-as-a-Service then disables the HPA for your application and executes your deployment strategy.

The Details: By disabling HPA during the deployment strategy, CD-as-a-Service ensures that HPA does not interfere with that strategy. For example, when using automated analysis results, if the new version has a memory leak, the HPA could scale up and thwart the threshold-based queries. Or during pod-ratio canary deployments, HPA could scale down pods and force more users into the canary version of application.

Step 3: After successfully deploying the new version of the application, CD-as-a-Service then enables HPA for the new version of the application.

The Details: CD-as-a-Service deploys every change you make, even changes to the HPA. That means as you refine your HPA configuration to the application’s needs, you can run automated performance tests in staging environments before ever deploying new HPA configurations to production.

In Conclusion:

With CD-as-a-Service, you can leverage all the benefits of HPA alongside advanced deployment strategies to deploy reliably without causing service interruptions. You can start using HPA with your applications by adding it to the list of manifests without adding any extra configuration. Test out your own HPA deployment scenarios by trying CD-as-a-Service for free by signing up today!