Flexfs: Transparent Resilience For Grid Storage Resourcea

Existing GRID infrastructures rely on explicit user instructions in order to replicate files for the purposes of resiliency. This human-intensive process is inefficient, error prone and, more importantly, makes file replication in GRIDs a cumbersome task. To address this problem, we introduce FlexFS - a fully automated file-system framework that seamlessly plugs into existing GRID structures providing automated file replication and transparent-to-user resilience. FlexFS breaks apart files into blocks and injects resilient information into these blocks through the use of Forward Erasure Correction codes. FlexFS employs a number of methods that facilitate the automated storage and efficient retrieval of the blocks in order to provide 110 throughput similar to that of local hard disks, all in the face of ever-changing utilization and availability of the GRID resources. Compared to currently available GRID replication schemes, FlexFS attains 15% to 230% higher throughput, both for reading and writing files.