B-Optimal: Resource Allocation Optimization for High Workload Applications

We propose B-Optimal, a Bayesian Optimization based resource allocator for large-scale commercial or personal applications. Our optimizer automatically determines the optimal resource allocation of CPU Quota, Memory, Disk Bandwidth, and Network Bandwidth for different microservices by measuring the impact of configuration changes on application performance. We treat microservices as black-boxes and our end-to-end optimizer requires very minimal human intervention. While prior resource allocation systems function well for laptop applications, such as those running the ELK or MEAN Stack, we design a system that supports high traffic web-applications, adding features that enable horizontal scaling within the optimized application. We also provide an analysis on the feasibility, advantages, and shortcomings of Bayesian Optimization relative to other resource allocation techniques. B-Optimal outperforms prior work within the short term producing response latencies that are 25% and 66% lower than a gradient descent based optimizer across two different applications, respectively, after ten minutes of running both optimizers. Finally, horizontal scaling on B-Optimal produces a 52% improvement from a balanced replica configuration, where every service has an equal number of replicas to fill the machine, and limits latency increase to a factor of 2.5x when workload is increased by 4x.