An Analysis of NIC Resource Usage for Offloading MPI

Modern cluster interconnection networks rely on pro- cessing on the network interface to deliver higher band- width and lower latency than what could be achieved oth- erwise. These processors are relatively slow, but they pro- vide adequate capabilities to accelerate some portion of the protocol stack in a cluster computing environment. This of- fload capability is conceptually appealing, but the standard evaluation of NIC-based protocol implementations relies on simplistic microbenchmarks that create idealized usage sce- narios. In this paper, we evaluate characteristics of MPI us- age scenarios using application benchmarks to help define the parameter space that protocol offload implementations should target. Specifically, we analyze characteristics that we expect to have an impact on NIC resource allocation and management strategies, including the length of the MPI posted receive and unexpected message queues, the number of entries in these queues that are examined for a typical operation, and the number of unexpected and expected mes- sages.