JackRabbit : Improved agility in elastic distributed storage

Elastic storage systems can be expanded or contracted to meet current demand, allowing servers to be turned off or used for other tasks. However, the usefulness of an elastic distributed storage system is limited by its agility: how quickly it can increase or decrease its number of servers. This paper describes an elastic storage system, called JackRabbit, that can quickly change its number of active servers. JackRabbit uses a combination of agility-aware offloading and reorganization techniques to minimize the work needed before deactivation or activation of servers. Analysis of real-world traces and experiments with the JackRabbit prototype confirm JackRabbit’s agility and show that it significantly reduces wasted server-time relative to state-of-the-art designs. Acknowledgements: We thank the members and companies of the PDL Consortium (including Actifio, APC, EMC, Emulex, Facebook, Fusion-io, Google, Hewlett-Packard Labs, Hitachi, Huawei Technologies Co., Intel, Microsoft Research, NEC Laboratories, NetApp, Oracle, Panasas, Riverbed, Samsung, Seagate, STEC, Symantec, VMWare, and Western Digital) for their interest, insights, feedback, and support. This research was sponsored in part by Intel and by CyLab at Carnegie Mellon University under grant #DAAD19–02–1–0389 from the Army Research Office. Experiments were enabled by generous hardware donations from Intel, NetApp, and APC. We thank Cloudera and Facebook for sharing the traces and Yanpei Chen for releasing SWIM.