Variability: A Tuning Headache

Performance tuning is an ongoing activity at most HPC sites. Small performance improvements can save thousands of dollars. Run-to-run performance variations significantly impact performance tuning. Not being able to tell which code version is faster (or more energy efficient) in a single run greatly increases the computational expense and uncertainty for the programmer. We will show examples where autotuning frameworks could easily choose a sub-optimal kernel. We will also examine the difficulty optimizing a real-world HPC application.