iWriter: An Offloading Method for Indirectly Writing Remote Data

Remote Direct Memory Access (RDMA) is expected to offer lower latency and higher throughput than standard kernel-based TCP/IP networking due to features such as kernel bypass and protocol offloading. For this reason, RDMA is often used as an excellent optimization approach to implement fundamental operations such as indirect writes, which have an effect on the performance of distributed storage systems. However, using the RPC method to accomplish indirect writes sacrifices CPU cycles. On the other hand, using a one-sided method constrained by the RDMA Verbs APIs requires at least two Round-Trip Times (RTTs).In this paper, we propose iWriter, a novel offloading method that enables clients to indirectly write data into a remote host’s memory without involving the remote CPU or extra RTTs. In order to reduce CPU overheads, iWriter instructs the RDMA-capable NIC (RNIC) to carry out indirect writes using native RDMA primitives. This way, the request, execution and reply are all automatically completed by the RNIC in only one RTT. Using commercial RoCEv2 NICs, we implement iWriter without making any changes to the RDMA drivers and compare it with existing methods. The evaluation results show that iWriter reduces latency by 19.1% to 47.1% compared to the RPC method and by 11.4% to 18.2% compared to the one-sided method.