An Extended SSD-Based Cache for Efficient Object Store Access in SAP IQ

The cloud-native version of SAP IQ aims to reduce the storage and compute costs by storing data directly on object stores while benefiting from the greater elasticity and scale-out properties that are offered. In the cloud-native version of SAP IQ, the buffer manager has gone through a significant re-design for two major reasons. First, RAM is more expensive on the cloud; therefore, a buffer manager that relies exclusively on RAM would not have been a viable solution. Second, on object stores, read and write operations are generally associated with higher latencies. To counteract the impact of higher latencies without utilizing more RAM, we have extended SAP IQ's buffer manager with a second layer of cache, namely, the Extended Cache Manager (ECM). The ECM relies on fast solid state drives (SSDs). In this paper, we describe our experience with the design and implementation of the ECM, in particular, the design choices we had to make to overcome the fact that, when compared to object stores, SSDs have a much more limited I/O bandwidth.