高级检索
    章紫琳, 刘铎, 谭玉娟, 吴宇, 罗龙攀, 王纬略, 乔磊. 一种适用于分布式存储集群的纠删码数据更新方法[J]. 计算机研究与发展, 2022, 59(11): 2451-2466. DOI: 10.7544/issn1000-1239.20210211
    引用本文: 章紫琳, 刘铎, 谭玉娟, 吴宇, 罗龙攀, 王纬略, 乔磊. 一种适用于分布式存储集群的纠删码数据更新方法[J]. 计算机研究与发展, 2022, 59(11): 2451-2466. DOI: 10.7544/issn1000-1239.20210211
    Zhang Zilin, Liu Duo, Tan Yujuan, Wu Yu, Luo Longpan, Wang Weilüe, Qiao Lei. An Erasure-Coded Data Update Method for Distributed Storage Clusters[J]. Journal of Computer Research and Development, 2022, 59(11): 2451-2466. DOI: 10.7544/issn1000-1239.20210211
    Citation: Zhang Zilin, Liu Duo, Tan Yujuan, Wu Yu, Luo Longpan, Wang Weilüe, Qiao Lei. An Erasure-Coded Data Update Method for Distributed Storage Clusters[J]. Journal of Computer Research and Development, 2022, 59(11): 2451-2466. DOI: 10.7544/issn1000-1239.20210211

    一种适用于分布式存储集群的纠删码数据更新方法

    An Erasure-Coded Data Update Method for Distributed Storage Clusters

    • 摘要: 目前分布式存储集群广泛采用纠删码来保证数据可靠性,但是数据更新密集时存储集群的磁盘I/O开销会成为性能瓶颈.在常用的纠删码数据更新方法中,磁盘I/O开销主要包括:1)更新数据块时对数据节点的读后写操作;2)更新校验块时读写日志的磁盘寻道开销.针对这些问题,提出PARD(parity logging with reserved space and data delta)数据更新方法,其主要思想是首先利用纠删码线性运算的特性来减少读后写操作;然后根据磁盘特性来降低磁盘寻道开销.PARD包含3个设计要点:1)采用即时的数据块更新和基于日志的校验块更新;2)利用纠删码线性运算的特性,构建基于数据增量的日志,极大限度地消除对数据节点的读后写操作;3)根据磁盘特性,在数据文件末尾为日志预留空间,减少读写日志的磁盘寻道开销.实验结果表明,当块大小为4 MB时,PARD的更新吞吐率相较于PLR(parity logging with reserved space),PARIX(speculative partial write),FO(full overwrite),分别至少提升了30.4%,47.0%,82.0%.

       

      Abstract: Erasure coding is widely deployed in distributed storage clusters to provide data reliability, but the disk I/O overhead becomes a performance bottleneck when data updates are intensive. On the one hand, traditional data update strategies need to read the original data chunk, and then write new data when updating the data chunk. In the case of intensive updates, frequent write-after-read seriously affects the write performance of the storage clusters. On the other hand, the operations of updating the parity chunk include reading the increments randomly distributed in the log file and merging them with the data file, which also introduces additional disk seek overhead. In this paper, a data updating method, named PARD (parity logging with reserved space and data delta), is proposed to solve these problems. The main idea of PARD is to use the linear calculations of erasure coding to reduce write-after-read, and take advantage of the disk characteristics to reduce the disk seek overhead. PARD comprises three key design features: 1) Adopting in-place data updates and log-based parity updates. 2) Taking advantage of the linear calculations of erasure coding to construct the log based on data increments. For a series of write requests to the same data chunk, only the first update needs to read the original data chunk, and the subsequent update executes the pure write, which remarkably reduces the write-after-read. 3) According to the characteristics of disk, reserving space for the log at the end of data file to reduce the disk seek overhead of reading and writing log. Experiments show that when the chunk size is 4 MB, PARD gains at least, 30.4%, 47.0% and 82.0% improvements in update throughput compared with PLR, PARIX, and FO, respectively.

       

    /

    返回文章
    返回