• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Chen Maotang, Zheng Sheng’an, You Litong, Wang Jingyu, Yan Tian, Tu Yaofeng, Han Yinjun, Huang Linpeng. A Distributed Persistent Memory File System Based on RDMA Multicast[J]. Journal of Computer Research and Development, 2021, 58(2): 384-396. DOI: 10.7544/issn1000-1239.2021.20200369
Citation: Chen Maotang, Zheng Sheng’an, You Litong, Wang Jingyu, Yan Tian, Tu Yaofeng, Han Yinjun, Huang Linpeng. A Distributed Persistent Memory File System Based on RDMA Multicast[J]. Journal of Computer Research and Development, 2021, 58(2): 384-396. DOI: 10.7544/issn1000-1239.2021.20200369

A Distributed Persistent Memory File System Based on RDMA Multicast

Funds: This work was supported by the National Key Research and Development Program of China (2018YFB1003302) and the SJTU-Huawei Innovation Research Lab Project (FA2018091021-202004).
More Information
  • Published Date: January 31, 2021
  • The development of persistent memory and remote direct memory access(RDMA) provides new opportunities for designing efficient distributed systems. However, the existing RDMA-based distributed systems are far from fully exploiting RDMA multicast capabilities, which makes them difficult to solve the problem of multi-copy file data transmission in one-to-many transmission, degrading system performance. In this paper, a distributed persistent memory and RDMA multicast transmission based file system(MTFS) is proposed. It efficiently transmits data to different data nodes by the low-latency multicast transmission mechanism, which makes full use of the RDMA multicast capability, hence avoiding high latency due to multi-copy file data transmission operations. To improve the flexibility of transmission operations, a multi-mode multicast remote procedure call(RPC) mechanism is proposed, which enables the adaptive recognition of RPC requests, and moves transmission operations out of the critical path to further improve transmission efficiency. MTFS also provides a lightweight consistency guarantee mechanism. By designing a crash recovery mechanism, a data verification module and a retransmission scheme, MTFS is able to quickly recover from a crash, and achieves file system reliability and data consistency by error detection and data correction. Experimental results show that MTFS has greatly increased the throughput by 10.2-219 times compared with GlusterFS. MTFS outperforms NOVA by 10.7% on the Redis workload, and achieves good scalability in multi-thread workloads.
  • Related Articles

    [1]Sun Qingxiao, Yang Hailong. Generalized Stencil Auto-Tuning Framework on GPU Platform[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440612
    [2]Li Maowen, Qu Guoyuan, Wei Dazhou, Jia Haipeng. Performance Optimization of Neural Network Convolution Based on GPU Platform[J]. Journal of Computer Research and Development, 2022, 59(6): 1181-1191. DOI: 10.7544/issn1000-1239.20200985
    [3]Zhang Shuai, Li Tao, Jiao Xiaofan, Wang Yifeng, Yang Yulu. Parallel TNN Spectral Clustering Algorithm in CPU-GPU Heterogeneous Computing Environment[J]. Journal of Computer Research and Development, 2015, 52(11): 2555-2567. DOI: 10.7544/issn1000-1239.2015.20148151
    [4]Luo Xinyuan, Chen Gang, Wu Sai. A GPU-Accelerated Highly Compact and Encoding Based Database System[J]. Journal of Computer Research and Development, 2015, 52(2): 362-376. DOI: 10.7544/issn1000-1239.2015.20140254
    [5]Tang Liang, Luo Zuying, Zhao Guoxing, and Yang Xu. SOR-Based P/G Solving Algorithm of Linear Parallelism for GPU Computing[J]. Journal of Computer Research and Development, 2013, 50(7): 1491-1500.
    [6]Cai Yong, Li Guangyao, and Wang Hu. Parallel Computing of Central Difference Explicit Finite Element Based on GPU General Computing Platform[J]. Journal of Computer Research and Development, 2013, 50(2): 412-419.
    [7]Wang Zhuowei, Xu Xianbin, Zhao Wuqing, He Shuibing, Zhang Yuping. Parallel Acceleration and Performance Optimization for GRAPES Model Based on GPU[J]. Journal of Computer Research and Development, 2013, 50(2): 401-411.
    [8]Wu Xiaoxiao, Liang Xiaohui, Xu Qidi, and Zhao Qinping. An Algorithm of Physically-based Scalar-fields Guided Deformation on GPU[J]. Journal of Computer Research and Development, 2010, 47(11): 1857-1864.
    [9]Wang Jing, Wang Lili, and Li Shuai. Pre-Computed Radiance Transport All-Frequency Shadows Algorithm on GPU[J]. Journal of Computer Research and Development, 2006, 43(9): 1505-1510.
    [10]Hu Wei and Qin Kaihuai. A New Rendering Technology of GPU-Accelerated Radiosity[J]. Journal of Computer Research and Development, 2005, 42(6): 945-950.

Catalog

    Article views PDF downloads Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return