ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2015, Vol. 52 ›› Issue (4): 929-942.doi: 10.7544/issn1000-1239.2015.20131911

Previous Articles     Next Articles

A High Performance Management Schema of Metadata Clustering for Large-Scale Data Storage Systems

Xiao Zhongzheng1,Chen Ningjiang1,Wei Jun2, Zhang Wenbo2   

  1. 1(School of Computer and Electronic Information, Guangxi University, Nanning 530004); 2(Technology Center of Software Engineering, Institute of Software, Chinese Academy of Sciences, Beijing 100190)
  • Online:2015-04-01

Abstract: An efficient, decentralized metadata management schema plays a vital role in large-scale distributed storage systems. The Hash-based partition schema and tree-based partition schema pay huge cost for expansion, and are sensitive to changes in cluster. In response to these problems, CH-MMS(consistent Hash based metadata management schema), is proposed. Virtual MDS (metadata server) is introduced in CH-MMS, and good effect for the cluster’s load balance is proved. Combining the standby mechanism with lazy-update policy, CH-MMS achieves fast failover and zero migration when the cluster changes. Due to its distributed metadata structure, CH-MMS has fast metadata lookup speed. In order to solve the problem that the Hash structure will cause damage to file system hierarchical semantics, a simple and flexible mechanism based on regular expression matching has been introduced. The following work is presented in the paper: 1)Expound the architecture of CH-MMS; 2)Introduce the core data structure of layout-table, virtual MDS and lazy-update policy, and their relevant algorithms; 3)Qualitatively analyze scalability and fault tolerance. The prototype system and simulation show that, CH-MMS is metadata-balancing and has fast failover, flexible expansion and zero migration when cluster changes. CH-MMS can meet the needs of flexible, efficient metadata management of large-scale storage systems with increasing data.

Key words: metadata management, consistent Hash, large-scale data storage, metadata server(MDS), distributed file system

CLC Number: