Unequal Failure Protection Coding Technology for Cloud Storage Systems

In recent years, erasure codes have become the de facto standard for data protection of large scale distributed cloud storage systems at the cost of an affordable storage overhead. While traditional erasure coding schemes, such as Reed-Solomon codes, suffer from high reconstruction cost and I/Os. The recent past has seen a plethora of efforts to optimize the tradeoff between the reconstruction cost, I/Os and storage overhead. Quietly different from all prior studies, in this paper, our erasure coding technology makes the first attempt to take advantage of the unequal failure rates across the disks/nodes to optimize the reconstruction performance and system reliability. Specifically, our proposed technology, the Unequal Failure Protection based Local Reconstruction Code (UFP-LRC) divides the data blocks into several unequal-sized groups with local parities, assigning the data blocks stored on more failure-prone disks/nodes into the smaller-sized group, so as to provide unequal failure protection for each group. In this way, by exploiting the nonuniform local parity degrees, the proposed UFP-LRC enables the data blocks that are stored on more failure-prone disks/nodes to tolerate a greater number of failures while suffer from less repair cost than others, leading to a substantial improvement of overall repair performance and reliability for cloud storage system. We perform numerical analysis and build a prototype storage system to verify our approach. The analytical results show that the UFPLRC technique gradually outperforms LRC along the increase of failure rate ratio. Also, extensive experiments show that, when compared to LRC, UFP-LRC is able to achieve a 10% to 13% improvement in throughput, and a 8% to 12% reduction in decoding latency, while retaining a comparable overall reliability.