ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2019, Vol. 56 ›› Issue (6): 1182-1191.doi: 10.7544/issn1000-1239.2019.20190113

Special Issue: 2019面向人工智能的计算机体系结构专题

Previous Articles     Next Articles

Training and Software Simulation for ReRAM-Based LSTM Neural Network Acceleration

Liu He1, Ji Yu1, Han Jianhui2, Zhang Youhui1, Zheng Weimin1   

  1. 1(Deparment of Computer Science and Technology, Tsinghua University, Beijing 100084);2(Institute of Microelectronics, Tsinghua University, Beijing 100084)
  • Online:2019-06-01
  • Supported by: 
    This work was supported by the Science and Technology Innovation Special Zone Project.

Abstract: Long short-term memory (LSTM) is mostly used in fields of speech recognition, machine translation, etc., owing to its expertise in processing and predicting events with long intervals and long delays in time series. However, most of existing neural network acceleration chips cannot perform LSTM computation efficiently, as limited by the low memory bandwidth. ReRAM-based crossbars, on the other hand, can process matrix-vector multiplication efficiently due to its characteristic of processing in memory (PIM). However, a software tool of broad architectural exploration and end-to-end evaluation for ReRAM-based LSTM acceleration is still missing. This paper proposes a simulator for ReRAM-based LSTM neural network acceleration and a corresponding training algorithm. Main features (including imperfections) of ReRAM devices and circuits are reflected by the highly configurable tools, and the core computation of simulation can be accelerated by general-purpose graphics processing unit (GPGPU). Moreover, the core component of simulator has been verified by the corresponding circuit simulation of a real chip design. Within this framework, architectural exploration and comprehensive end-to-end evaluation can be achieved.

Key words: ReRAM, long short-term memory (LSTM), training algorithm, simulation framework, neural network

CLC Number: