ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2019, Vol. 56 ›› Issue (12): 2641-2648.doi: 10.7544/issn1000-1239.2019.20180474

Previous Articles     Next Articles

Long Term Recurrent Neural Network with State-Frequency Memory

Zhuang Liansheng1, Lü Yang1, Yang Jian2,3, Li Houqiang1   

  1. 1(School of Information Science and Technology, University of Science and Technology of China,Hefei 230027);2(Institute of System Engineering, Academy of Military Science, Beijing 100191);3(Peng Cheng Laboratory, Shenzhen, Guangdong 518000)
  • Online:2019-12-01

Abstract: Modeling time series has become one of the research hotspots in the field of machine learning because of its important application value. Recurrent neural network (RNN) is a crucial tool for modeling time series in recent years. However, existing RNNs are commonly hard to learn long-term dependency in the temporal domain and unable to model the frequency patterns in time series. The two problems seriously limit the performance of existing RNNs for the time series that contain long-term dependencies and rich frequency components. To solve these problems, we propose the long term recurrent neural network with state-frequency memory (LTRNN-SFM), which allows the network to model the uncovered features in both frequency and temporal domains by replacing state vector of the hidden layer in conventional RNNs to state-frequency matrix. Meanwhile, the proposed network can effectively avoid the interference of the gradient vanishing and exploding problems by separating neurons in the same layer, using activation functions such as rectified linear unit (ReLU) and clipping weight. In this way, a LTRNN-SFM with long-term memory and multiple layers can be trained easily. Experimental results have demonstrated that the proposed network achieves the best performance in processing time series with long-term dependencies and rich frequency components.

Key words: recurrent neural network (RNN), long-term dependency, frequency domain analysis, time series classification, time series prediction

CLC Number: