Interactive, Cloud-Native Workflows on HPC Using KNoC.
ISC Workshops(2022)
摘要
Cloud and HPC platforms differentiate by many aspects, but both can run applications in identical contexts using containers. In this paper we present KNoC, an open-source virtual node (kubelet) for Kubernetes that transparently manages the container lifecycle on a remote HPC cluster using Slurm and Singularity. Our goal is on one hand to allow HPC users to leverage existing cloud-native tools, such as the popular Argo Workflows language to express complex data-processing pipelines, while on the other hand enabling Cloud setups to exploit computing resources available in HPC centers. KNoC bridges Cloud and HPC, transforming Argo to a cross-environment, portable solution, which allows the combination of Cloud-based tools and HPC steps into the same workflow, controlled and monitored through an interactive frontend. Deploying KNoC requires only a secure shell connection to the cluster's login node. We describe the design and implementation of KNoC, and evaluate the integration using several proof-of-concept workflows.
更多查看译文
关键词
hpc,cloud-native
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要