COMPSYS 2022 Keynote Talk: Composability at the Boundary Between HPC and Cloud.

Disaggregated memory is a topic of active development, as a way to allow flexible and fine-grained allocation of memory capacity, mitigating the mismatch between fixed per-node resource provisioning and varying application requirements. This talk addresses the development and atscale evaluation of HPC job scheduling policies for disaggregated memory, advocating for a simulation approach together with a model of the slowdown due to resource contention. A key parameter that is generally expected to be provided by the users at job submission time is the job�s peak memory capacity demand. The memory demands of HPC jobs vary dramatically, due to differing application behaviour and strong scaling. It is therefore unrealistic to expect this number to be precise, especially for small jobs. We discuss the incentives to provide an accurate estimate and show that there is a strong risk of tragedy of the commons. We conclude with ways to mitigate this problem.