A Task-Parallel Runtime for Heterogeneous Multi-node Vector Systems

In recent years, high-performance computing systems are equipped with not only host processors but also accelerators, and becoming more heterogeneous as well as becoming larger in scale. The task parallel execution model is promising to efficiently utilize such a large-scale system by minimizing synchronizations in comparison with traditional models. In this paper, we propose a task-parallel runtime system that individually considers the processors for task management and task execution; those two roles could be assigned to different processors. This paper focuses on NEC SX-Aurora TSUBASA as an example of heterogeneous multi-node systems, which are equipped with two kinds of general-purpose processors, to exploit the system heterogeneity for efficient task-parallel execution. Specifically, the proposed runtime system is used to select an appropriate processor for task management, depending on several execution conditions. The performance of the proposed runtime is discussed by running a Cholesky factorization implementation. The evaluation results show that the proposed runtime system can improve performance by more than 25% in comparison with a conventional a conventional implementation.