A Reconfigurable Cache Consistency Mechanism for Distributed Memory Pool

Duan Zhuohui; Liu Haikun; Zhao Jinwei; Liu Yihang; Liao Xiaofei; Jin Hai

doi:10.7544/issn1000-1239.202330448

Journal of Computer Research and Development > 2023 > 60(9): 1960-1972. > DOI: 10.7544/issn1000-1239.202330448

Duan Zhuohui, Liu Haikun, Zhao Jinwei, Liu Yihang, Liao Xiaofei, Jin Hai. A Reconfigurable Cache Consistency Mechanism for Distributed Memory Pool[J]. Journal of Computer Research and Development, 2023, 60(9): 1960-1972. DOI: 10.7544/issn1000-1239.202330448

Citation:

PDF (2009 KB)

A Reconfigurable Cache Consistency Mechanism for Distributed Memory Pool

National Engineering Research Center for Big Data Technology and System (Huazhong University of Science and Technology), Wuhan 430074
Services Computing Technology and System Lab (Huazhong University of Science and Technology), Ministry of Education, Wuhan 430074
Cluster and Grid Computing Lab (Huazhong University of Science and Technology), Wuhan 430074
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074

Funds: This work was supported by the National Key Research and Development Program of China（2022YFB4500303）and the National Natural Science Foundation of China (62072198, 61825202, 61929103).

More Information

Author Bio:
Duan Zhuohui: born in 1994. PhD, post-doc researcher. His main research interests include non-volatile memory systems, storage systems and techniques, and distributed systems

Liu Haikun: born in 1981. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include in-memory computing, cloud computing, and distributed systems

Zhao Jinwei: born in 1996. Master. His main research interests include remote direct memory access and distributed systems

Liu Yihang: born in 1994. Master candidate. His main research interests include non-volatile storage and distributed systems

Liao Xiaofei: born in 1978. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include computer architecture, system software, and big data processing

Jin Hai: born in 1966. PhD, professor, PhD supervisor. Fellow of CCF, fellow of IEEE. His main research interests include computer architecture, parallel and distributed computing, big data processing, data storage, and system security
Received Date: June 04, 2023
Revised Date: July 18, 2023
Available Online: July 23, 2023

Graphical Abstract

Abstract

Abstract

In distributed memory systems, caching is an effective way for reducing the latency of remote memory accesses. However, a single cache consistency mechanism often fails to efficiently adapt to the access behaviors of various workloads. We propose a hybrid and reconfigurable cache consistency mechanism for distributed heterogeneous memory pool systems, which has advantages of both directory-based and broadcast-based cache consistency mechanisms. We use the four-quadrant matrix analysis method to analyze the access pattern of each object, and then adopt the most efficient cache consistency mechanism. Moreover, the adopted cache consistency mechanism can be dynamically shifted to another mechanism based on the change of memory access pattern. Experimental results show that the reconfigurable hybrid cache consistency mechanism can improve the read and write performance of distributed heterogeneous memory pool systems by 32.31% and 31.20% on average, respectively, compared with a single cache consistency mechanism. Moreover, the hybrid cache consistency mechanism shows good scalability when the number of clients increases.

FullText(HTML)

References (33)

References

[1]	Bailleu M, Giantsidi D, Gavrielatos V, et al. Avocado: A secure in-memory distributed storage system[C]//Proc of the 2021 USENIX Annual Technical Conf (ATC 2021). Berkeley, CA: USENIX Association, 2021: 65−79
[2]	肖仁智,冯丹,胡燏翀,等. 面向非易失内存的数据一致性研究综述[J]. 计算机研究与发展,2020,57(1):85−101 Xiao Renzhi, Feng Dan, Hu Yuchong, et al. A survey of data consistency research for non-volatile memory[J]. Journal of Computer Research and Development, 2020, 57(1): 85−101 (in Chinese)
[3]	舒继武,陆游游,张佳程,等. 基于非易失性存储器的存储系统技术研究进展[J]. 科技导报,2016,34(14):86−94 Shu Jiwu, Lu Youyou, Zhang Jiacheng, et al. Research progress on non-volatile memory based storage system[J]. Science & Technology Review, 2016, 34(14): 86−94 (in Chinese)
[4]	Gu Juncheng, Lee Y, Zhang Yiwen, et al. Efficient memory disaggregation with infiniswap[C]//Proc of the 14th USENIX Symp on Networked Systems Design and Implementation (NSDI 2017). Berkeley, CA: USENIX Association, 2017: 649−667
[5]	Zheng Shengan, Hoseinzadeh M, Swanson S. Ziggurat: A tiered file system for non-volatile main memories and disks[C] //Proc of the 17th USENIX Conf on File and Storage Technologies (FAST 2019). Berkeley, CA: USENIX Association, 2019: 207−219
[6]	Duan Zhuohui, Liu Haikun, Lu Haodi, et al. Gengar: An RDMA-based distributed hybrid memory pool[C] //Proc of the 2021 IEEE 41st Int Conf on Distributed Computing Systems (ICDCS 2021). Piscataway, NJ: IEEE, 2021: 92−103
[7]	Liao Jianwei, Trahay F, Cai Zhigang, et al. Fine granularity and adaptive cache update mechanism for client caching[J]. IEEE Systems Journal, 2019, 13(2): 1587−1598 doi: 10.1109/JSYST.2018.2866905
[8]	Cai Qingchao, Guo Wentian, Zhang Hao, et al. Efficient distributed memory management with RDMA and caching[J]. Proceedings of the VLDB Endowment, 2018, 11(11): 1604−1617 doi: 10.14778/3236187.3236209
[9]	Fawaz K, Artail H. Dcim: Distributed cache invalidation method for maintaining cache consistency in wireless mobile networks[J]. IEEE Transactions on Mobile Computing, 2013, 12(4): 680−693 doi: 10.1109/TMC.2012.37
[10]	Gao Yixiao, Li Qiang, Tang Lingbo, et al. When cloud storage meets RDMA[C] //Proc of the 18th USENIX Symp on Networked Systems Design and Implementation (NSDI 2021). Berkeley, CA: USENIX Association, 2021: 519−533
[11]	Gray C G, Cheriton D R. Leases: An efficient fault-tolerant mechanism for distributed file cache consistency[C]//Proc of the ACM Symp on Operating System Principles (SOSP 1989). New York: ACM, 1989: 202−210
[12]	Li Xiaoqian, Zhou Huachun, Qiu Feng, et al. A lease algorithm to maintain strong mapping cache consistency[J]. Wireless Personal Communications, 2014, 76(4): 713−730 doi: 10.1007/s11277-013-1530-0
[13]	Yu Xiangyao, Xia Yu, Pavlo A, et al. Sundial: Harmonizing concurrency control and caching in a distributed OLTP database management system[C] // Proc of the 44th Int Conf on Very Large Data Bases (VLDB 2018). Rio de Janeiro, Brazil: VLDB Endowment Inc., 2018: 1289−1302
[14]	Caheny P, Alvarez L, Valero M, et al. Runtime-assisted cache coherence deactivation in task parallel programs[C] // Proc of the Int Conf for High Performance Computing, Networking, Storage and Analysis (SC 2018). Piscataway, NJ: IEEE, 2018: 454−465
[15]	He Ximing, Ma Sheng, Liu Wenjie, et al. VISU: A simple and efficient cache coherence protocol based on self-updating[C] // Proc of the Int Conf on Algorithms and Architectures for Parallel Processing (ICA3PP 2018). Cham, Switzerland: Springer, 2018: 341−357
[16]	Al-Waisi Z, O. Agyeman M. An overview of on-chip cache coherence protocols[C] // Proc of the Intelligent Systems Conf (IntelliSys 2017). Piscataway, NJ: IEEE, 2017: 304−309
[17]	Shukur H, Zeebaree S, Zebari R, et al. Cache coherence protocols in distributed systems[J]. Journal of Applied Science and Technology Trends, 2020, 1(3): 92−97 doi: 10.38094/jastt1329
[18]	Amory I A, Ahmed A H, Hasan Z. MESI protocol for multicore processors based on FPGA[J]. Periodicals of Engineering and Natural Sciences, 2021, 9(1): 80−89 doi: 10.21533/pen.v9i1.1772
[19]	Kaur D P, Sulochana V. Design and implementation of cache coherence protocol for high-speed multiprocessor system[C] //Proc of the 2018 2nd IEEE Int Conf on Power Electronics, Intelligent Control and Energy Systems (ICPEICES 2018). Piscataway, NJ: IEEE, 2018: 1097−1102
[20]	Nair A S, Pai A V, Raveendran B K, et al. MOESIL: A cache coherency protocol for locked mixed criticality l1 data cache[C] //Proc of the 2021 IEEE/ACM 25th Int Symp on Distributed Simulation and Real Time Applications (DS-RT 2021). Piscataway, NJ: IEEE, 2021: 1−8
[21]	Ibrahim R K, Jumma L F, Amory I A, et al. Design of MOESI protocol for multicore processors based on FPGA[J]. International Journal of Nonlinear Analysis and Applications, 2021, 12(Special Issue): 1229−1242
[22]	Duvvuri V, Shenoy P, Tewari R. Adaptive leases: A strong consistency mechanism for the world wide web[J]. IEEE Transactions on Knowledge and Data Engineering, 2003, 15(5): 1266−1276 doi: 10.1109/TKDE.2003.1232277
[23]	Gafni E, Lamport L. Disk Paxos[J]. Distributed Computing, 2003, 16(1): 1−20 doi: 10.1007/s00446-002-0070-8
[24]	Lamport L. Fast Paxos[J]. Distributed Computing, 2006, 19(2): 79−103 doi: 10.1007/s00446-006-0005-x
[25]	Kraska T, Pang G, Franklin M J, et al. MDCC: Multi-data center consistency[C] //Proc of the 8th ACM European Conf on Computer Systems (EuroSys 2013). New York: ACM, 2013: 113−126
[26]	Liu Shenling, Zhang Chunyuan, Chen Yujiao. DCC: Distributed cache consistency[C] //Proc of the Int Conf of Pioneering Computer Scientists, Engineers and Educators (ICPCSEE 2017). Berlin: Springer, 2017: 377−387
[27]	Jialin Li, Nelson J, Michael E, et al. Pegasus: Tolerating skewed workloads in distributed storage with in-network coherence directories[C] //Proc of the 14th USENIX Conf on Operating Systems Design and Implementation (OSDI 2020). Berkeley, CA: USENIX Association, 2020: 387−406
[28]	Attiya H, Gramoli V, Milani A. A provably starvation-free distributed directory protocol[C] //Proc of the Int Symp on Stabilizing, Safety, and Security of Distributed Systems (SSS 2010). Berlin: Springer, 2010: 405−419
[29]	陈志强,周宏伟,冯权友,等. 面向多核处理器的可配置缓存一致性协议设计与实现[J]. 计算机研究与发展,2021,58(6):1166−1175 Chen Zhiqiang, Zhou Hongwei, Feng Quanyou, et al. Design and implementation of configurable cache coherence protocol for multi-core processor[J]. Journal of Computer Research and Development, 2021, 58(6): 1166−1175 (in Chinese)
[30]	郑阳,陈海波,臧斌宇. 基于广播的分布式共享内存[J]. 上海交通大学学报,2018,52(10):1333−1338 doi: 10.16183/j.cnki.jsjtu.2018.10.023 Zheng Yang, Chen Haibo, Zang Binyu. Snooping-based distributed shared memory[J]. Journal of Shanghai Jiaotong University, 2018, 52(10): 1333−1338 (in Chinese) doi: 10.16183/j.cnki.jsjtu.2018.10.023
[31]	Liu Guoxin, Shen Haiying, Chandler H, et al. Measuring and evaluating live content consistency in a large-scale CDN[J]. IEEE Transactions on Parallel and Distributed Systems, 2016, 27(7): 2074−2090 doi: 10.1109/TPDS.2015.2479222
[32]	Bhat S, Vijayakumar V. Adaptive directory based cache coherency model to optimize the network bandwidth in distributed applications[J]. IOSR Journal of Engineering, 2018, 8(9): 41−46
[33]	Wang Qing, Lu Youyou, Xu E, et al. Concordia: Distributed shared memory with in-network cache coherence[C] //Proc of the 19th USENIX Conf on File and Storage Technologies (FAST 2021). Berkeley, CA: USENIX Association, 2021: 277−292

[1]	Wang Huiju, Huang Weixuan, Yue Xiao. Novel Practical Query Pricing Algorithm Based on Labor Game Model[J]. Journal of Computer Research and Development, 2024, 61(12): 3154-3167. DOI: 10.7544/issn1000-1239.202330791
[2]	Wang Zhen, Fan Hongjie, Liu Junfei. An Alleviate Exposure Bias Method in Joint Extraction of Entities and Relations[J]. Journal of Computer Research and Development, 2022, 59(9): 1980-1992. DOI: 10.7544/issn1000-1239.20210078
[3]	Wang Dong, Li Zhenyu, Xie Gaogang. Unbiased Sampling Technologies on Online Social Network[J]. Journal of Computer Research and Development, 2016, 53(5): 949-967. DOI: 10.7544/issn1000-1239.2016.20148387
[4]	Yang Tan, Feng Xiang, Yu Huiqun. Feature Selection Algorithm Based on the Multi-Colony Fairness Model[J]. Journal of Computer Research and Development, 2015, 52(8): 1742-1756. DOI: 10.7544/issn1000-1239.2015.20150245
[5]	Gong Jibing, Wang Rui, Wang Xiaofeng, Cui Li. Health Status Detection via Temporal-Spatial Factor Graph Model in Medical Social Networks[J]. Journal of Computer Research and Development, 2013, 50(6): 1285-1296.
[6]	Wang Peng, Meng Dan, Zhan Jianfeng, Tu Bibo. Review of Programming Models for Data-Intensive Computing[J]. Journal of Computer Research and Development, 2010, 47(11): 1993-2002.
[7]	Liu Zhenglin, Han Yu, Zou Xuecheng, and ChenYicheng. Power Analysis Attacks Against AES Based on Maximal Bias Signal[J]. Journal of Computer Research and Development, 2009, 46(3): 370-376.
[8]	Lin Fen, Shi Chuan, Luo Jiewen, Shi Zhongzhi. Dual Reinforcement Learning Based on Bias Learning[J]. Journal of Computer Research and Development, 2008, 45(9): 1455-1462.
[9]	Lin Jianning, Wu Huizhong. Research on a Trust Model Based on the Subjective Logic Theory[J]. Journal of Computer Research and Development, 2007, 44(8): 1365-1370.
[10]	Ma Liang, Chen Qunxiu, and Cai Lianhong. An Improved Model for Adaptive Text Information Filtering[J]. Journal of Computer Research and Development, 2005, 42(1): 79-84.

Cited By

Cited by

Periodical cited type(7)

1.	冯杨洋，汪庆，舒继武. 大模型时代下的存储系统挑战与技术发展. 大数据. 2025(01): 79-91 .
2.	栾昊立，王晓东，杨锐，郝建宇，赵铭浩，尹祖新，王丽琼. AI智算发展对高速光模块的应用需求研究. 邮电设计技术. 2024(06): 7-11 .
3.	刘少堃，何仲廉，李彬，李超峰. 基于大模型的电子病历自动生成系统的设计与应用探讨. 中国数字医学. 2024(08): 8-13 .
4.	孙一尧，刘馨，刘晓丹，李琳. 人工智能应用设计创新与非物质文化遗产结合——以新疆毛皮画推广APP为例. 鞋类工艺与设计. 2024(15): 151-153 .
5.	童俊杰，申佳，赫罡，张奎. 运营商智算中心建设思路及方案. 邮电设计技术. 2024(09): 68-73 .
6.	丛凯，陈宏，苏征，任心钰，黄若铖，李国. 人工智能大模型在电子政务中的应用研究. 中国信息界. 2024(06): 92-94 .
7.	赵明江，刘艳梅，杨婧一，张星奎，贾占宇. 基于非Transformer架构大模型的技术研究及应用探索. 电力大数据. 2024(06): 11-21 .