Shmem+ and scf: system-level programming models for scalable reconfigurable computing

Heterogeneous computing systems comprised of FPGAs coupled with standard microprocessors are becoming an increasingly popular solution for building future high-performance computing (HPC) and high-performance embedded computing (HPEC) systems due to their higher performance and energy efficiency versus their CPU-only counterparts. Unfortunately, the time and difficulty associated with developing scalable, parallel applications for reconfigurable computing (RC) platforms is often prohibitive, making it difficult to exploit the potential gains in performance and energy savings. Design and implementation of applications for these systems involve many system-wide considerations such as algorithm decomposition and architecture mappings to exploit multiple levels of parallelism, inter-device communication and control, and system-level debug and verification. Thus, system-level languages with constructs for expressing multiple levels and forms of parallelism are vital for productive implementation of designs. In order to improve a developer’s productivity as well as the portability of scalable RC applications, we propose and analyze two different approaches for establishing communication in parallel RC applications. The first approach extends the traditional partitioned, global-address-space (PGAS) model to a multilevel abstraction by integrating a hierarchy of multiple memory components present in reconfigurable HPC systems into a single virtual memory layer. Based on this model, we adapt the SHMEM communication library to become what we call SHMEM+, the first known SHMEM library enabling coordination between FPGAs and CPUs in a reconfigurable HPC system. The second approach provides a system coordination framework (SCF) based on a message-passing programming model. SCF enables transparent communication and synchronization between tasks running on heterogeneous processing devices in a system by hiding low-level communication details while presenting a uniform communication interface across heterogeneous devices. Besides improving developer productivity, both of these approaches enhance the portability of applications (by hiding the platform-specific details from applications). Further improvements in productivity are obtained by combining techniques for modeling performance with the aforementioned approaches for application design. By allowing developers to estimate the performance of their application, modeling enables the developers to make critical design decisions before undertaking an expensive implementation. Case studies illustrate the merits and effectiveness of the proposed approaches through increased performance and portability of applications along with improvement in developer productivity. The resulting communication libraries and coordination framework are projected to provide large productivity improvements, thus expanding the use of RC technology in the fields of HPC and HPEC.