MERCURY: A Transparent Guided I/O Framework for High Performance I/O Stacks

The performance gap between processors and I/O represents a serious scalability limitation for applications running on computing clusters. Parallel file systems often provide mechanisms that allow programmers to disclose their I/O pattern knowledge to the lower layers of the I/O stack through a hints API. This information can be used by the file system to boost the application performance. Unfortunately, programmers rarely make use of these features, missing the opportunity to exploit the full potential of the storage system. In this paper we propose MERCURY, a transparent guided I/O framework able to optimize file I/O patterns in scientific applications, allowing users to control the I/O behavior of applications without modifications. This is done by exploiting the hints API provided by the back-end file system to guide data prefetching. MERCURY effeciently converts numerous small read requests into a few larger requests. Furthermore, it increases the I/O bandwidth, reduces the number of I/O requests, and ultimately the application running time. Moreover, we also propose a Linux kernel modification that allows network file systems, specifically Lustre, to work with our guided I/O framework through the posix_fadvise interface.