Portability and Scalability Evaluation of Large-Scale Statistical Modeling and Prediction Software through HPC-Ready Containers
CoRR(2023)
摘要
HPC-based applications often have complex workflows with many software
dependencies that hinder their portability on contemporary HPC architectures.
In addition, these applications often require extraordinary efforts to deploy
and execute at performance potential on new HPC systems, while the users expert
in these applications generally have less expertise in HPC and related
technologies. This paper provides a dynamic solution that facilitates
containerization for transferring HPC software onto diverse parallel systems.
The study relies on the HPC Workflow as a Service (HPCWaaS) paradigm proposed
by the EuroHPC eFlows4HPC project. It offers to deploy workflows through
containers tailored for any of a number of specific HPC systems. Traditional
container image creation tools rely on OS system packages compiled for generic
architecture families (x86\_64, amd64, ppc64, ...) and specific MPI or GPU
runtime library versions. The containerization solution proposed in this paper
leverages HPC Builders such as Spack or Easybuild and multi-platform builders
such as buildx to create a service for automating the creation of container
images for the software specific to each hardware architecture, aiming to
sustain the overall performance of the software. We assess the efficiency of
our proposed solution for porting the geostatistics ExaGeoStat software on
various parallel systems while preserving the computational performance. The
results show that the performance of the generated images is comparable with
the native execution of the software on the same architectures. On the
distributed-memory system, the containerized version can scale up to 256 nodes
without impacting performance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要