Optimizing Application Throughput with Condition-based Autoscaling and Employing Alternatives to Scaling In/Out.

Containerization-based deployment is widely practiced by organiza-tions because of its lightweight and portable nature. The containers are managed by orchestrators, such as Kubernetes, to ensure reli-ability and scalability among other properties. Autoscaling is an essential component for cloud applications and thus Kubernetes’s Horizontal Pod Autoscaler (HPA) makes scaling decisions utiliz-ing default metrics, such as CPU and memory utilization or other application-oriented metrics like event loop lag. We consider the above-mentioned metrics as the primary metrics for the autoscaling decision, but explore other metrics, such as HTTP requests and garbage collection related metrics. We consider these additional metrics as secondary metrics that may help better explain the val-ues seen for the primary metrics. We investigated and analyzed the correlation of secondary metrics to the primary metrics and observed patterns indicating significant coupling. Based on these observations, we propose a condition-based autoscaling process using secondary metrics like total garbage collection pause time to improve scaling based on the primary metrics. We demonstrate using a technique of killing pods that exceed thresholds on the sec-ondary metrics instead of scaling in or out as one example of how secondary metrics can be used to improve autoscaling based on the primary metrics. We evaluate our methodology and compare the performance results against the default autoscaling mechanism for various workloads and share results illustrating that it can achieve better results.