An Efficient Lock Protocol in Remote Direct Memory Access Network

Distributed lock is a crucial component in distributed storage systems, where the performance of the lock protocol significantly influences overall system performance. Remote Direct Memory Access (RDMA), an emerging data center networking technology, supports one-sided communication primitives and offers low CPU overhead, low latency, and high throughput, presenting new opportunities for designing high-performance distributed lock protocols. However, designing RDMA-based distributed lock protocols faces significant challenges, particularly concerning performance, scalability, and fairness. This paper addresses these challenges by proposing FeLock, a high-performance distributed lock protocol for RDMA networks. FeLock leverages diverse RDMA communication primitives, enabling clients not only to communicate with the server for lock acquisition and release but also to interact directly with other clients to hand off lock ownership. This design aims to achieve high performance while ensuring fairness and scalability. Specifically, to guarantee high performance, FeLock introduces the per-node lock management mechanism to reduce network round trips on the critical paths of the lock protocol. To achieve scalability, FeLock incorporates a round-robin handover mechanism where nodes are logically organized into a ring, and clients hand over lock ownership sequentially based on their position within this ring. To ensure fairness and prevent client starvation, FeLock employs a node credit mechanism that limits the number of consecutive lock acquisitions by any single node, thereby preventing clients on other nodes from being indefinitely blocked. Experimental results demonstrate that FeLock achieves performance comparable to or exceeding that of existing one-sided RDMA lock protocols, such as DSLR, while exhibiting superior fairness and scalability. With 3 to 120 clients, FeLock achieves throughput 1.01 to 7.51 times of DSLR, with its fairness improved by up to 2.24 times.