Virtualraid: a mass storage architecture for out-of-core applications

To support the performance requirements of modern computer systems, parallel disk systems have evolved. Because disks suffer from independent failures, parallel storage systems suffer from degraded reliability as systems increase in size. This drawback is addressed by incorporating redundancy in the form of additional disks. This redundancy results in higher cost and degraded performance. To improve performance, expensive non-volatile cache memories are commonly used. Memories are required to be non-volatile to prevent unrecoverable data loss in the event of system failure, including power failure, operator error, hardware breakdown, or software crash. This dissertation proposes a new approach to parallel storage for Out-of-Core applications. Our storage solution combines (1) a redundant RAID Level for permanent storage, and (2) a striped disk array for temporary storage into a dual configuration RAID, resulting in performance superior to redundant RAIDs. Our approach also provides higher storage efficiency than redundant disk arrays, because temporary files require no redundancy. The drawback of not providing redundancy for temporary files is that rollback is required for system crashes or disk failures to re-create temporary files. Because temporary data need not be recovered, temporary metadata is not stored to disk, resulting in fewer disk accesses. To analyze the performance improvements of our proposed architectures, we created a suite of application benchmarks. These applications were traced to characterize these benchmarks. These traces were then analyzed using a simulator of our proposed architectures. Our architecture observed up to a 10X performance improvement in raw disk access times over the conventional RAID 5 case, resulting in an overall performance improvement of 1.2X for most of the benchmarks. A raw disk performance improvement of 3X was observed over a cached-RAID 5 architecture. virtualRAID benefited from not having to store temporary metadata to disk. Full simulation results are presented for virtualRAID, non-cached RAID 5, cached RAID 5, and a RAID 0 configuration. The partitioning structure can significantly degrade performance for applications that contain high concentration of seek accesses. We will present a novel reconfigurable RAID architecture that stores temporary non-redundant data along with permanent redundant data. By optimizing the partition layout, this approach improves performance by up to 2X over our original proposed architecture, and up to 3.37X over the conventional RAID 5 case. Since this architecture minimizes the number of required seeks and minimizes temporary metadata accesses, it also outperforms a non-redundant RAID 0 disk array by a factor of up to 1.7X.