ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2020, Vol. 57 ›› Issue (6): 1125-1139.doi: 10.7544/issn1000-1239.2020.20200010

Special Issue: 2020计算机体系结构前沿技术专题

Previous Articles     Next Articles

An Energy Consumption Optimization and Evaluation for Hybrid Cache Based on Reinforcement Learning

Fan Hao1,2, Xu Guangping1,2, Xue Yanbing1,2, Gao Zan1,2, Zhang Hua3   

  1. 1(School of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384);2(Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology(Tianjin University of Technology), Tianjin 300384);3(Tianjin Sino-German University of Applied Sciences, Tianjin 300350)
  • Online:2020-06-01
  • Supported by: 
    This work was supported by the National Natural Science Foundation of China (61971309) and the Tianjin Natural Science Foundation (17JCYBJC15600, 18JCYBJC84800).

Abstract: Emerging non-volatile memory STT-RAM has the characteristics of low leakage power, high density, fast read speed, and high write energy. Meanwhile, SRAM has the characteristics of high leakage power, low density, fast read and write speed, low write energy, etc. The hybrid cache of SRAM and STT-RAM fully utilizes the respective advantages of both memory medias, providing lower leakage power and higher cell density than SRAM, higher write speed and lower write energy than STT-RAM. The architecture of hybrid cache mainly achieves both of benefits by putting write-intensive data into SRAM and read-intensive data into STT-RAM. Therefore, how to identify and allocate read-write-intensive data is the key challenge for the hybrid cache design. This paper proposes a cache management method based on the reinforcement learning that uses the write intensity and reuse information of cache access requests to design a cache allocation policy and optimize energy consumption. The key idea is to use the reinforcement learning algorithm to get the weight for the set allocating to SRAM or STT-RAM by learning from the energy consumption of cache line sets. The algorithm allocates a cache line in a set to the region with greater weight. Evaluations show that our proposed policy reduces the average energy consumption by 16.9%(9.7%) in a single-core (quad-core) system compared with the previous policies.

Key words: reinforcement learning, hybrid cache architecture, cache, spin transfer torque random access memory, allocation policy

CLC Number: