高级检索

    对象存储系统中一种高效的分层对象布局算法

    An Efficient Hierarchical Object Placement Algorithm for Object Storage Systems

    • 摘要: 在对象存储系统中,如何有效地在对象存储设备上分布对象是其面临的重大挑战.需要一个能够常数时间内定位对象,同时能公平地分布对象以及自适应存储规模变化的对象布局算法.目前大部分布局算法只能适应单层模式,少数的多层模式对设备配置有严格的要求,而且无法在常数时间内定位对象,自适应性较差.提出了一种新的分层对象布局算法,首先使用最大最小聚类算法将设备集合进行分类,支持灵活的设备配置.然后使用提出的EFAH Hashing算法在集群间和集群内分布对象.理论和实验证明,新的分层对象布局算法可以在常数时间内定位对象,从而减轻元数据服务器的计算量.同时可以在设备之间较公平地分布对象,达到I/O负载均衡的目的.而且在设备集合变化时,迁移较少的对象数以满足对象再次分布的公平性.

       

      Abstract: With the prevalence of object-based storage systems, one of the big challenges in such systems is how to design an effective object placement algorithm which can locate object in constant time, distribute data evenly among object-based devices and adapt well to the changes of devices. A majority of proposed approaches are appropriate for single level mode, while the multi-level approaches cannot locate object in constant time and have bad adaptability. This paper presents a novel hierarchy object placement algorithm to distribute several petabytes of objects among tens or hundreds of thousands of devices. Specially, it uses Max-Min algorithm to classify the devices into some classes for different devices configuration. Then, we propose EFAH hashing algorithm to assign data between classes and within a class. The theoretical analysis and experimental study show that this new hierarchy object placement can locate data in constant time to reduce the computation overhead of metadata server and avoid the performance bottleneck. Moreover, it can distribute objects evenly among devices to balance I/O load. In the event of devices changes, our approach can redistribute fewer objects to preserve even distribution so that the performance of systems would not be affected in the process of rebalancing I/O load.

       

    /

    返回文章
    返回