Analyzing the I/O Patterns of Deep Learning Applications

A traditional HPC storage system is designed to manage an I/O workload dominated by write operation bursts, mainly for applications carrying out simulations and checkpointing partial results. Currently, this context is more diverse because of artificial intelligence applications’ workload, such as machine learning and deep learning. As ML/DL applications are becoming more compute-intensive, they require the power of HPC systems. However, the HPC I/O system could be a bottleneck to scaling these kind of applications, mainly in the training stage. In this paper, we present a methodology for analyzing the I/O patterns of deep learning applications that allows us to understand the DL applications’ I/O in HPC systems. We have applied our approach to serial and distributed DL codes by using the TensorFlow2 and PyTorch framework for the MNIST and CIFAR-10 datasets.