Supervisory Event Loop-based Autoscaling of Node.js Deployments

Autoscaling mechanisms are used widely to scale computing instances, under varying load conditions. Containerized cloud applications managed by orchestrators-such as Kubernetes with the Horizontal Pod Autoscaler (HPA)-can be scaled based on CPU utilization. However, the prevalent approach may not be ideal for every language runtime, such as the event-driven Node.js. Language runtime-specific metrics can be utilized to accurately describe the state of the application and use it to monitor and affect the application scalability in control loop-based systems at the desired conditions dynamically. Hence, we investigate event loop lag as an alternative and Node.js-specific metric to drive autoscaling with HPA controllers. Additionally, we synthesize a three-tier adaptive mechanism on top of the cluster autoscaler, which acts as a supervisory controller, to change the setpoint value dynamically based on the divergence from the service level objectives. We further extend the aforementioned architecture to re-evaluate the system model and the controller gain parameters at runtime. To assess the methodology, we evaluate and compare the performance of our adaptive autoscaling approach against the CPU-utilization-based autoscaling mechanism, under various load patterns and workloads.