A Reconfigurable Cache Consistency Mechanism for Distributed Memory Pool

Duan Zhuohui; Liu Haikun; Zhao Jinwei; Liu Yihang; Liao Xiaofei; Jin Hai

doi:10.7544/issn1000-1239.202330448

Journal of Computer Research and Development > 2023 > 60(9): 1960-1972. > DOI: 10.7544/issn1000-1239.202330448

Duan Zhuohui, Liu Haikun, Zhao Jinwei, Liu Yihang, Liao Xiaofei, Jin Hai. A Reconfigurable Cache Consistency Mechanism for Distributed Memory Pool[J]. Journal of Computer Research and Development, 2023, 60(9): 1960-1972. DOI: 10.7544/issn1000-1239.202330448

Citation:

PDF (2009 KB)

A Reconfigurable Cache Consistency Mechanism for Distributed Memory Pool

National Engineering Research Center for Big Data Technology and System (Huazhong University of Science and Technology), Wuhan 430074
Services Computing Technology and System Lab (Huazhong University of Science and Technology), Ministry of Education, Wuhan 430074
Cluster and Grid Computing Lab (Huazhong University of Science and Technology), Wuhan 430074
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074

Funds: This work was supported by the National Key Research and Development Program of China（2022YFB4500303）and the National Natural Science Foundation of China (62072198, 61825202, 61929103).

More Information

Author Bio:
Duan Zhuohui: born in 1994. PhD, post-doc researcher. His main research interests include non-volatile memory systems, storage systems and techniques, and distributed systems

Liu Haikun: born in 1981. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include in-memory computing, cloud computing, and distributed systems

Zhao Jinwei: born in 1996. Master. His main research interests include remote direct memory access and distributed systems

Liu Yihang: born in 1994. Master candidate. His main research interests include non-volatile storage and distributed systems

Liao Xiaofei: born in 1978. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include computer architecture, system software, and big data processing

Jin Hai: born in 1966. PhD, professor, PhD supervisor. Fellow of CCF, fellow of IEEE. His main research interests include computer architecture, parallel and distributed computing, big data processing, data storage, and system security
Received Date: June 04, 2023
Revised Date: July 18, 2023
Available Online: July 23, 2023

Graphical Abstract

Abstract

Abstract

In distributed memory systems, caching is an effective way for reducing the latency of remote memory accesses. However, a single cache consistency mechanism often fails to efficiently adapt to the access behaviors of various workloads. We propose a hybrid and reconfigurable cache consistency mechanism for distributed heterogeneous memory pool systems, which has advantages of both directory-based and broadcast-based cache consistency mechanisms. We use the four-quadrant matrix analysis method to analyze the access pattern of each object, and then adopt the most efficient cache consistency mechanism. Moreover, the adopted cache consistency mechanism can be dynamically shifted to another mechanism based on the change of memory access pattern. Experimental results show that the reconfigurable hybrid cache consistency mechanism can improve the read and write performance of distributed heterogeneous memory pool systems by 32.31% and 31.20% on average, respectively, compared with a single cache consistency mechanism. Moreover, the hybrid cache consistency mechanism shows good scalability when the number of clients increases.

FullText(HTML)

References (33)

References

[1]	Bailleu M, Giantsidi D, Gavrielatos V, et al. Avocado: A secure in-memory distributed storage system[C]//Proc of the 2021 USENIX Annual Technical Conf (ATC 2021). Berkeley, CA: USENIX Association, 2021: 65−79
[2]	肖仁智,冯丹,胡燏翀,等. 面向非易失内存的数据一致性研究综述[J]. 计算机研究与发展,2020,57(1):85−101 Xiao Renzhi, Feng Dan, Hu Yuchong, et al. A survey of data consistency research for non-volatile memory[J]. Journal of Computer Research and Development, 2020, 57(1): 85−101 (in Chinese)
[3]	舒继武,陆游游,张佳程,等. 基于非易失性存储器的存储系统技术研究进展[J]. 科技导报,2016,34(14):86−94 Shu Jiwu, Lu Youyou, Zhang Jiacheng, et al. Research progress on non-volatile memory based storage system[J]. Science & Technology Review, 2016, 34(14): 86−94 (in Chinese)
[4]	Gu Juncheng, Lee Y, Zhang Yiwen, et al. Efficient memory disaggregation with infiniswap[C]//Proc of the 14th USENIX Symp on Networked Systems Design and Implementation (NSDI 2017). Berkeley, CA: USENIX Association, 2017: 649−667
[5]	Zheng Shengan, Hoseinzadeh M, Swanson S. Ziggurat: A tiered file system for non-volatile main memories and disks[C] //Proc of the 17th USENIX Conf on File and Storage Technologies (FAST 2019). Berkeley, CA: USENIX Association, 2019: 207−219
[6]	Duan Zhuohui, Liu Haikun, Lu Haodi, et al. Gengar: An RDMA-based distributed hybrid memory pool[C] //Proc of the 2021 IEEE 41st Int Conf on Distributed Computing Systems (ICDCS 2021). Piscataway, NJ: IEEE, 2021: 92−103
[7]	Liao Jianwei, Trahay F, Cai Zhigang, et al. Fine granularity and adaptive cache update mechanism for client caching[J]. IEEE Systems Journal, 2019, 13(2): 1587−1598 doi: 10.1109/JSYST.2018.2866905
[8]	Cai Qingchao, Guo Wentian, Zhang Hao, et al. Efficient distributed memory management with RDMA and caching[J]. Proceedings of the VLDB Endowment, 2018, 11(11): 1604−1617 doi: 10.14778/3236187.3236209
[9]	Fawaz K, Artail H. Dcim: Distributed cache invalidation method for maintaining cache consistency in wireless mobile networks[J]. IEEE Transactions on Mobile Computing, 2013, 12(4): 680−693 doi: 10.1109/TMC.2012.37
[10]	Gao Yixiao, Li Qiang, Tang Lingbo, et al. When cloud storage meets RDMA[C] //Proc of the 18th USENIX Symp on Networked Systems Design and Implementation (NSDI 2021). Berkeley, CA: USENIX Association, 2021: 519−533
[11]	Gray C G, Cheriton D R. Leases: An efficient fault-tolerant mechanism for distributed file cache consistency[C]//Proc of the ACM Symp on Operating System Principles (SOSP 1989). New York: ACM, 1989: 202−210
[12]	Li Xiaoqian, Zhou Huachun, Qiu Feng, et al. A lease algorithm to maintain strong mapping cache consistency[J]. Wireless Personal Communications, 2014, 76(4): 713−730 doi: 10.1007/s11277-013-1530-0
[13]	Yu Xiangyao, Xia Yu, Pavlo A, et al. Sundial: Harmonizing concurrency control and caching in a distributed OLTP database management system[C] // Proc of the 44th Int Conf on Very Large Data Bases (VLDB 2018). Rio de Janeiro, Brazil: VLDB Endowment Inc., 2018: 1289−1302
[14]	Caheny P, Alvarez L, Valero M, et al. Runtime-assisted cache coherence deactivation in task parallel programs[C] // Proc of the Int Conf for High Performance Computing, Networking, Storage and Analysis (SC 2018). Piscataway, NJ: IEEE, 2018: 454−465
[15]	He Ximing, Ma Sheng, Liu Wenjie, et al. VISU: A simple and efficient cache coherence protocol based on self-updating[C] // Proc of the Int Conf on Algorithms and Architectures for Parallel Processing (ICA3PP 2018). Cham, Switzerland: Springer, 2018: 341−357
[16]	Al-Waisi Z, O. Agyeman M. An overview of on-chip cache coherence protocols[C] // Proc of the Intelligent Systems Conf (IntelliSys 2017). Piscataway, NJ: IEEE, 2017: 304−309
[17]	Shukur H, Zeebaree S, Zebari R, et al. Cache coherence protocols in distributed systems[J]. Journal of Applied Science and Technology Trends, 2020, 1(3): 92−97 doi: 10.38094/jastt1329
[18]	Amory I A, Ahmed A H, Hasan Z. MESI protocol for multicore processors based on FPGA[J]. Periodicals of Engineering and Natural Sciences, 2021, 9(1): 80−89 doi: 10.21533/pen.v9i1.1772
[19]	Kaur D P, Sulochana V. Design and implementation of cache coherence protocol for high-speed multiprocessor system[C] //Proc of the 2018 2nd IEEE Int Conf on Power Electronics, Intelligent Control and Energy Systems (ICPEICES 2018). Piscataway, NJ: IEEE, 2018: 1097−1102
[20]	Nair A S, Pai A V, Raveendran B K, et al. MOESIL: A cache coherency protocol for locked mixed criticality l1 data cache[C] //Proc of the 2021 IEEE/ACM 25th Int Symp on Distributed Simulation and Real Time Applications (DS-RT 2021). Piscataway, NJ: IEEE, 2021: 1−8
[21]	Ibrahim R K, Jumma L F, Amory I A, et al. Design of MOESI protocol for multicore processors based on FPGA[J]. International Journal of Nonlinear Analysis and Applications, 2021, 12(Special Issue): 1229−1242
[22]	Duvvuri V, Shenoy P, Tewari R. Adaptive leases: A strong consistency mechanism for the world wide web[J]. IEEE Transactions on Knowledge and Data Engineering, 2003, 15(5): 1266−1276 doi: 10.1109/TKDE.2003.1232277
[23]	Gafni E, Lamport L. Disk Paxos[J]. Distributed Computing, 2003, 16(1): 1−20 doi: 10.1007/s00446-002-0070-8
[24]	Lamport L. Fast Paxos[J]. Distributed Computing, 2006, 19(2): 79−103 doi: 10.1007/s00446-006-0005-x
[25]	Kraska T, Pang G, Franklin M J, et al. MDCC: Multi-data center consistency[C] //Proc of the 8th ACM European Conf on Computer Systems (EuroSys 2013). New York: ACM, 2013: 113−126
[26]	Liu Shenling, Zhang Chunyuan, Chen Yujiao. DCC: Distributed cache consistency[C] //Proc of the Int Conf of Pioneering Computer Scientists, Engineers and Educators (ICPCSEE 2017). Berlin: Springer, 2017: 377−387
[27]	Jialin Li, Nelson J, Michael E, et al. Pegasus: Tolerating skewed workloads in distributed storage with in-network coherence directories[C] //Proc of the 14th USENIX Conf on Operating Systems Design and Implementation (OSDI 2020). Berkeley, CA: USENIX Association, 2020: 387−406
[28]	Attiya H, Gramoli V, Milani A. A provably starvation-free distributed directory protocol[C] //Proc of the Int Symp on Stabilizing, Safety, and Security of Distributed Systems (SSS 2010). Berlin: Springer, 2010: 405−419
[29]	陈志强,周宏伟,冯权友,等. 面向多核处理器的可配置缓存一致性协议设计与实现[J]. 计算机研究与发展,2021,58(6):1166−1175 Chen Zhiqiang, Zhou Hongwei, Feng Quanyou, et al. Design and implementation of configurable cache coherence protocol for multi-core processor[J]. Journal of Computer Research and Development, 2021, 58(6): 1166−1175 (in Chinese)
[30]	郑阳,陈海波,臧斌宇. 基于广播的分布式共享内存[J]. 上海交通大学学报,2018,52(10):1333−1338 doi: 10.16183/j.cnki.jsjtu.2018.10.023 Zheng Yang, Chen Haibo, Zang Binyu. Snooping-based distributed shared memory[J]. Journal of Shanghai Jiaotong University, 2018, 52(10): 1333−1338 (in Chinese) doi: 10.16183/j.cnki.jsjtu.2018.10.023
[31]	Liu Guoxin, Shen Haiying, Chandler H, et al. Measuring and evaluating live content consistency in a large-scale CDN[J]. IEEE Transactions on Parallel and Distributed Systems, 2016, 27(7): 2074−2090 doi: 10.1109/TPDS.2015.2479222
[32]	Bhat S, Vijayakumar V. Adaptive directory based cache coherency model to optimize the network bandwidth in distributed applications[J]. IOSR Journal of Engineering, 2018, 8(9): 41−46
[33]	Wang Qing, Lu Youyou, Xu E, et al. Concordia: Distributed shared memory with in-network cache coherence[C] //Proc of the 19th USENIX Conf on File and Storage Technologies (FAST 2021). Berkeley, CA: USENIX Association, 2021: 277−292

[1]	Zheng Jieyu, Song Zhenyu, Zhu Haoliang, Zhao Yunlei, Lin Jingqiang, Fan jin. Efficient Software Implementations of NTRU Lattice-Based Key Encapsulation Mechanisms[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440478
[2]	Wei Hongru, Huang Jingyi. SOTS: A Hash Function-Based Shorter Post-Quantum Digital Signature Scheme[J]. Journal of Computer Research and Development, 2021, 58(10): 2300-2309. DOI: 10.7544/issn1000-1239.2021.20210619
[3]	Wang Yongli, Xu Qiuliang. Principle and Research Progress of Quantum Computation and Quantum Cryptography[J]. Journal of Computer Research and Development, 2020, 57(10): 2015-2026. DOI: 10.7544/issn1000-1239.2020.20200615
[4]	Li Zichen, Xie Ting, Zhang Juanmei, Xu Ronghua. Post Quantum Authenticated Key Exchange Protocol Based on Ring Learning with Errors Problem[J]. Journal of Computer Research and Development, 2019, 56(12): 2694-2701. DOI: 10.7544/issn1000-1239.2019.20180874
[5]	Zhang Pingyuan, Jiang Han, Cai Jie, Wang Chenguang, Zheng Zhihua, Xu Qiuliang. Recent Advances in Lattice-Based Cryptography[J]. Journal of Computer Research and Development, 2017, 54(10): 2121-2129. DOI: 10.7544/issn1000-1239.2017.20170628
[6]	Zhang Junwei, Ma Zhuo, Ma Jianfeng, and Ji Wenjiang. UC Security Model of Position-Based Key Exchange[J]. Journal of Computer Research and Development, 2014, 51(2): 353-359.
[7]	Lai Xin, Huang Xiaofang, He Dake. An ID-Based Efficient Signcryption Key Encapsulation Scheme[J]. Journal of Computer Research and Development, 2009, 46(5): 857-863.
[8]	Liu Yongliang, Gao Wen, Yao Hongxun, Huang Tiejun. Security on Aydos et al's Elliptic Curve Cryptography Based Wireless Authentication Protocol[J]. Journal of Computer Research and Development, 2006, 43(12): 2076-2081.
[9]	Tian Ye, Zhang Yujun, Li Zhongcheng. A Survey of Identity-Based Cryptography Using Pairing[J]. Journal of Computer Research and Development, 2006, 43(10): 1810-1819.
[10]	Zheng Ji. An MDP Public-Key Digital Signature Scheme[J]. Journal of Computer Research and Development, 2005, 42(2).