Communication Characteristics and Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-core SMP Nodes

Hybrid MPI/OpenMP and pure MPI on clusters of multi- core SMP nodes involve several mismatch problems be- tween the parallel programming models and the hardware architectures. Measurements of communication character- istics between cores on the same socket, on the same SMP node, and between SMP nodes on several platforms (includ- ing Cray XT4 and XT5) show that machine topology has a significant impact on performance for all parallelization strategies and that topology awareness should be built into all applications in the future. We describe potentials and challenges of the dominant programming models on hierar- chically structured hardware. Case studies with the multi- zone NAS parallel benchmarks on several platforms demon- strate the opportunities of hybrid programming.