MARs: Memory Access Rearrangements in Open MPI

The datatype engine in Message Passing Interface (MPI) libraries supports the communication layer by handling the transfer of non-contiguous datatypes. Basic datatypes (integer, float, etc.) serve as building blocks for more complex, and potentially non-contiguous, derived datatypes. In this context, the datatype engine facilitates the description of complex datatypes and provides an efficient and portable interface between complex types and the network communication layer. In this paper, we focus on the Open MPI datatype engine with its compact type representations and pipeline techniques used to hide communication latency. We identified cases where the current datatype representation is sub-optimal and provide an alternative based on range descriptions and Memory Access Rearrangements (MARs) for more efficient pack/unpack operations. As a result, we obtain a performance improvement between 1.2x and 3.2x compared to the current datatype description in Open MPI.