MIFO: A Query-Semantic Aware Resource Allocation Policy
Proceedings of the 2019 International Conference on Management of Data(2019)
摘要
Data Analytics Frameworks encourage sharing of clusters for execution of mixed workloads by promising fairness and isolation along with high performance and resource utilization. However, concurrent query executions on such shared clusters result in increased queue and resource waiting times for queries affecting their overall performance. MIFO is a dataflow aware scheduling policy that mitigates the impacts due to queue and resource contentions by reducing the waiting times for queries near completion. We present heuristics that exploit query semantics to proactively trigger MIFO-based allocations in a workload. Our experiments on Apache Spark using TPCDS benchmark show that compared to a FAIR policy, MIFO provides an improved mean response time, reduced makespan of the workload and average speedup between 1.2x-2.7x in highly concurrent setting with only a momentary deviation in fairness.
更多查看译文
关键词
cluster scheduling, dynamic resource allocation, resource interferences, scheduler queue wait time
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络