MATRYOSHKA: Non-Exclusive Memory Tiering via Transactional Page Migration

With the advent of byte-addressable memory devices, such as CXL memory, persistent memory, and storage-class memory, tiered memory systems have become a reality. Page migration is the de facto method within operating systems for managing tiered memory. It aims to bring hot data whenever possible into fast memory to optimize the performance of data accesses while using slow memory to accommodate data spilled from fast memory. While the existing research has demonstrated the effectiveness of various optimizations on page migration, it falls short of addressing a fundamental question: Is exclusive memory tiering, in which a page is either present in fast memory or slow memory, but not both simultaneously, the optimal strategy for tiered memory management? We demonstrate that page migration-based exclusive memory tiering suffers significant performance degradation when fast memory is under pressure. In this paper, we propose non-exclusive memory tiering, a page management strategy that retains a copy of pages recently promoted from slow memory to fast memory to mitigate memory thrashing. To enable non-exclusive memory tiering, we develop MATRYOSHKA, a new mechanism that features transactional page migration and page shadowing. MATRYOSHKA removes page migration off the program's critical path and makes migration asynchronous. Evaluations with microbenchmarks and realworld applications show that MATRYOSHKA achieves 6x performance improvement over the state-of-the-art transparent page placement (TPP) approach under memory pressure. We also compare MATRYOSHKA with a recently proposed sampling-based migration approach and demonstrate MATRYOSHKA's strengths and potential weaknesses in various scenarios. Through the evaluations, we discover a serious issue facing all tested approaches, unfortunately including MATRYOSHKA, and call for further research on tiered memory-aware memory allocation.