MiF: Mitigating the Intra-file Fragmentation in Parallel File System

Parallel file systems have been broadly deployed in large scale data centers, supporting a wide range of applications across a variety of industries. Unfortunately, most parallel file systems suffer from the intra-file fragmentation which is the disk performance killer. This paper presents the design and implementation of MiF, which introduces two techniques: on-demand preallocation and embedded directory, to Mitigate theintra-file Fragmentation of data placement, improving the disk performance in parallel 茂卢聛le system. The key insight of on-demand preallocation is that the preallocation of a 茂卢聛le should be aware of concurrent process streams and recognizes the write characteristic. The background rationale of embedded directory is that, since modern parallel 茂卢聛le systems aggregate many normal operation pairs, exploring the disk bandwidth for metadata access requires all metadata of sub-茂卢聛les in the same directory be placed adjacently. Measurements of our MiF implementation in a block-based parallel 茂卢聛le system demonstrate that it can signi茂卢聛cantly improve I/O performance of parallel programs.