Load is not what you should balance: Introducing Prequal
CoRR(2023)
摘要
We present Prequal (Probing to Reduce Queuing and Latency), a load balancer
for distributed multi-tenant systems. Prequal aims to minimize real-time
request latency in the presence of heterogeneous server capacities and
non-uniform, time-varying antagonist load. It actively probes server load to
leverage the power-of-d-choices paradigm, extending it with asynchronous and
reusable probes. Cutting against received wisdom, Prequal does not balance CPU
load, but instead selects servers according to estimated latency and active
requests-in-flight (RIF). We explore its major design features on a testbed
system and evaluate it on YouTube, where it has been deployed for more than two
years. Prequal has dramatically decreased tail latency, error rates, and
resource use, enabling YouTube and other production systems at Google to run at
much higher utilization.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要