OpenMP and MPI application energy measurement variation

Power, energy, and compute time are all important metrics that can act as either objectives or constraints in program or system optimization. Recent microprocessors include sensors (counters) for monitoring these metrics as well as on-chip system controllers that may use this information. Code optimization is relatively straightforward if the measurements are stable and repeatable over time on nominally identical hardware, if there is a lot of variance it becomes very difficult. This paper describes experiments that expose the variability of performance and energy usage on recent Intel processors for some parallel benchmarks using shared memory (OpenMP) and message passing (MPI) programming models. During the start up phase going from a quiescent to a "hot" steady state temperature differences of greater than 26°C were seen resulting in run-to-run energy differences as large as 10%. Even in steady state, run-to-run variability in execution time and energy usage were problematic. The patterns of variability found in execution time and energy consumption pose a challenge to simple strategies for running performance experiments as part of a tuning framework.