分布式存储中精确修复最小带宽再生码的性能研究

卫东升; 李  钧; 王  新

doi:10.7544/issn1000-1239.2014.20121095

分布式存储中精确修复最小带宽再生码的性能研究

(智能信息处理上海市重点实验室(复旦大学计算机科学技术学院) 上海 201203) (12210240069@fudan.edu.cn)

基金项目: 国家自然科学基金项目(61171074)；国家“八六三”高技术研究发展计划基金项目(2009AA01A348)；教育部新世纪优秀人才支持计划基金项目(NCET-11-0113)

详细信息

中图分类号: TP302.8
计量
- 文章访问数: 1654
- HTML全文浏览量: 1
- PDF下载量: 625
出版历程
- 发布日期: 2014-08-14

Performance Study of Exact Minimum Bandwidth Regenerating Codes in Distributed Storage

(Shanghai Key Laboratory of Intelligent Information Processing (School of Computer Science, Fudan University), Shanghai 201203)

摘要

摘要: 分布式存储系统为保证数据可靠性，需要对数据进行冗余存储来应对由于节点失效所带来的数据不可靠性.基于矩阵积构造的精确修复最小带宽再生码除了能够显著降低系统的存储冗余，而且编码的构造参数之间没有约束限制，还能够显著降低修复带宽的开销，具有广阔的应用前景.然而，基于此编码方案所设计的分布式存储系统的性能开销并没有得到充分的研究和分析.针对该编码在分布式存储系统中数据上传、修复、下载3个阶段，分别比较CPU使用率、文件大小、缓冲区大小以及有限域大小对上述3个阶段中运算速度的影响，发现通过对相关参数进行合理配置，可以使得基于相应编码方案的分布式存储系统能够获得良好的运行性能.
- 分布式存储 /
- 再生码 /
- 网络编码 /
- 矩阵积 /
- 性能研究
Abstract: Distributed storage systems need to introduce redundancy to ensure data reliability against node failures. To repair failed nodes, a significant amount of bandwidth is consumed. Regenerating codes are able to achieve the optimal tradeoff between the storage overhead and the repair bandwidth overhead. Based on the current situation that bandwidth resources are more precious than computing resources in distributed storage systems, exact minimum bandwidth regenerating (E-MBR) codes, which can be implemented by a product-matrix construction, enjoy the advantages of regenerating codes as well as systematic codes, and have no restrictions for all construction parameters, making themselves a promising candidate towards the application in distributed storage systems. However, the performance overhead of distributed storage systems based on this coding scheme has not been investigated and analyzed. This paper gives a formal description of coding operations, which can be categorized into three distinct phrases: uploading, downloading and repairing. We hereby analyze the impact of the CPU utilization, the file size, the buffer size and the Galois field size to the computing rates in the three distinct phrases above. We find that distributed storage systems based on E-MBR codes are able to achieve a high computing throughput if we configure the construction parameters of E-MBR codes appropriately.
- distributed storage /
- regenerating codes /
- network coding /
- product-matrix /
- performance study

HTML全文

参考文献(0)

施引文献(9)

期刊类型引用(4)

1.	王鑫，李瑞，兰蓝，白波，白伊玎. 北京市检查检验结果互认数据对接实践与思考. 中国卫生信息管理杂志. 2024(06): 838-843 . 百度学术
2.	高茂，张丽萍，侯敏，闫盛，赵宇博. 基于BERT的百科知识库实体对齐. 内蒙古师范大学学报(自然科学汉文版). 2023(06): 630-637 . 百度学术
3.	李翠华，高昭昇，刘玉转. 区域检验检查结果互认平台建设与应用探讨. 中国卫生信息管理杂志. 2022(06): 835-841 . 百度学术
4.	姚华彦，张鑫金，何萍. 基于大数据的患者画像标签体系构建方法及应用研究. 中国卫生信息管理杂志. 2019(06): 667-671 . 百度学术