高级检索

    一种基于远程直接内存访问网络的高效分布式锁协议

    An Efficient Lock Protocol in Remote Direct Memory Access Network

    • 摘要: 分布式锁是分布式存储系统的重要组件,锁协议的性能对系统整体的性能有关键性影响.远程直接内存访问(RDMA)是一种新兴的数据中心网络技术,它支持单边网络通信原语,可以降低系统CPU开销,同时具备低延迟、高吞吐的性能特性,为设计高速分布式锁协议提供了新机遇.然而,设计基于RDMA的分布式锁协议面临诸多挑战.本文着重在保证高性能的前提下解决扩展性和公平性挑战,提出一种RDMA网络下的高性能分布式锁协议FeLock,它利用多种类型的RDMA网络通信原语,使客户端不仅能与服务端通信加解锁,还能与其他客户端直接通信以移交锁所有权,同时实现了高性能、公平性和性能的扩展性.具体地,为保证高性能,FeLock引入了节点粒度锁管理机制,缩减锁协议在关键路径上的网络往返次数.为实现扩展性,FeLock引入了轮转移交机制,将所有节点排成一个环,客户端按照其在环中的顺序依次移交锁的所有权.为实现公平性、避免客户端饥饿,FeLock引入了节点信用机制,限制节点连续加锁的次数,避免其他节点上的客户端无法加锁.实验显示,FeLock相比于现有单边RDMA锁协议(如DSLR)表现出相似或更高的性能,并且具有更好的公平性和扩展性.在3至120个客户端的环境下,FeLock的吞吐量达到DSLR的1.01至7.51倍,公平性提升至多2.24倍.

       

      Abstract: Distributed lock is a crucial component in distributed storage systems, where the performance of the lock protocol significantly influences overall system performance. Remote Direct Memory Access (RDMA), an emerging data center networking technology, supports one-sided communication primitives and offers low CPU overhead, low latency, and high throughput, presenting new opportunities for designing high-performance distributed lock protocols. However, designing RDMA-based distributed lock protocols faces significant challenges, particularly concerning performance, scalability, and fairness. This paper addresses these challenges by proposing FeLock, a high-performance distributed lock protocol for RDMA networks. FeLock leverages diverse RDMA communication primitives, enabling clients not only to communicate with the server for lock acquisition and release but also to interact directly with other clients to hand off lock ownership. This design aims to achieve high performance while ensuring fairness and scalability. Specifically, to guarantee high performance, FeLock introduces the per-node lock management mechanism to reduce network round trips on the critical paths of the lock protocol. To achieve scalability, FeLock incorporates a round-robin handover mechanism where nodes are logically organized into a ring, and clients hand over lock ownership sequentially based on their position within this ring. To ensure fairness and prevent client starvation, FeLock employs a node credit mechanism that limits the number of consecutive lock acquisitions by any single node, thereby preventing clients on other nodes from being indefinitely blocked. Experimental results demonstrate that FeLock achieves performance comparable to or exceeding that of existing one-sided RDMA lock protocols, such as DSLR, while exhibiting superior fairness and scalability. With 3 to 120 clients, FeLock achieves throughput 1.01 to 7.51 times of DSLR, with its fairness improved by up to 2.24 times.

       

    /

    返回文章
    返回