Fault-Tolerant Streaming Computation With Blockmon
2015 IEEE Global Communications Conference (GLOBECOM)(2015)
摘要
As the amount of data being exchanged over the network increases, algorithms originally implemented for running on a single machine have been re-designed to work in a distributed manner, with a processing platform that splits tasks among machines and cores. Brand new frameworks have emerged for the analysis of unbound streams of data, aiming at processing data and retrieving information nearly real-time by using clusters of machines. Node failure and recovery are crucial issues related to distributed systems, especially when using commodity hardware and when continuously processing data coming real-time into the system. In this paper we present the performance of the distributed stream-processing platform BlockMon, with the novel fault-tolerant mechanism that we implement on top, and compare it against Spark Streaming, the state-of-the art in terms of fault-tolerant stream-processing platform. Our experimental results suggest that BlockMon performs around two times faster than Spark Streaming, with a twenty times reduced memory footprint, showing the feasibility of using BlockMon on popular energy-efficient architectures such as the ARM ones.
更多查看译文
关键词
energy efficient architectures,reduced memory footprint,distributed stream processing platform,data recovery,node failure,distributed processing platform,BlockMon,fault tolerant streaming computation
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络