Characterizing non-volatile memory transactional systems

Emerging Non-Volatile Memory (NVM) technologies, like Intel’s DC Optane persistent memory, offer byte-addressability and orders of magnitude faster access to storage than traditional storage technologies. Their key appeal is that they allow applications to access storage directly using processor load and store instructions rather than relying on a software intermediary like the file system or a database [1]. However, ensuring that data stored in NVM is always in a safe and recoverable state is both hard and incurs performance overheads [1]–[3]. To ensure data recoverability, application developers have to carefully orchestrate data movement from the volatile to the persistent components in the memory hierarchy, subject to applicationspecific constraints. This task is especially complex due to two factors: (1) NVM applications have very diverse crash-consistency requirements [4]; and (2) the persistence domain is different across platforms. For example, Intel and Micron guarantee that data becomes persistent only when it reaches the memory controller of the NVM device, i.e., the persistence domain of the system includes the memory controller and the NVM devices [5]. We refer to such systems as having transient caches. However, HPE’s NVM [6] guarantees that the entire cache hierarchy is persistent, i.e., the persistence domain includes the entire memory hierarchy. We refer to such systems as having persistent caches. In this context, researchers have proposed various transactional systems that provide the well known “ACID” guarantees for NVM applications. These transactional systems significantly simplify NVM application development and leave the complexities of achieving data recoverability on various platforms to the low-level systems software developers. While these systems all provide ACID guarantees, they go about providing these guarantees in different ways: UNDO vs. REDO logging, software vs. hardware transactions. Low-level developers designing ACID transaction systems face a bewildering array of choices, with varied performance characteristics that change with the applications and the platform used. For these developers, we aim to answer the question: how to quickly explore the design space and arrive at a correct and high-performance implementation of an NVM transactional system? Reasoning about implementation details rather than the overall guarantees provided to the user (ACID) helps transaction system developers traverse the design-space more efficiently. To provide ACID guarantees, the underlying transaction system has to correctly ensure three properties: (1) crash consistency individual transactions are failure-atomic, i.e., after a crash, either all or none of the transaction has persisted, (2) synchronization transactions are correctly isolated from other transactions executed on different threads, and (3) composability the crash consistency and synchronization techniques used compose to provide the overall ACID guarantees, by ensuring that dependent transactions are correctly ordered. This new characterization of transaction systems provides a basis to compare different implementations and to identify the right set of crash-consistency and synchronization mechanisms for particular applications and hardware platforms. We perform a detailed characterization study of systems with different implementations (hardware transactional memory (HTM) [7], software transactional memory (STM) [3], and undo/redo logging with locks [3], [8]) under various persistence domains (transient vs. persistent caches). We perform our study on real hardware using the recently released Intel’s DC Optane Persistent Memory [9] and using simulation. Our empirical study results in several interesting insights for NVM transaction system developers: