ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2019, Vol. 56 ›› Issue (12): 2641-2648.doi: 10.7544/issn1000-1239.2019.20180474

• 人工智能 • 上一篇    下一篇

时频联合长时循环神经网络

庄连生1,吕扬1,杨健2,3,李厚强1   

  1. 1(中国科学技术大学信息科学技术学院 合肥 230027);2(军事科学院系统工程研究院 北京 100191);3(鹏城实验室 广东深圳 518000) (lszhuang@ustc.edu.cn)
  • 出版日期: 2019-12-01
  • 基金资助: 
    国家自然科学基金项目(61472379)

Long Term Recurrent Neural Network with State-Frequency Memory

Zhuang Liansheng1, Lü Yang1, Yang Jian2,3, Li Houqiang1   

  1. 1(School of Information Science and Technology, University of Science and Technology of China,Hefei 230027);2(Institute of System Engineering, Academy of Military Science, Beijing 100191);3(Peng Cheng Laboratory, Shenzhen, Guangdong 518000)
  • Online: 2019-12-01

摘要: 时间序列建模问题因有着重要的应用价值已经成为机器学习领域的研究热点之一.循环神经网络(recurrent neural network, RNN)是近年来时间序列建模的一个重要工具.但是,现有循环神经网络无法处理长时依赖关系的时序数据,也没有在频域对时间序列数据的特征模式进行建模.对于那些包含长时依赖且频率成分丰富的时序数据,这2个问题大大限制了现有循环神经网络的性能.针对这些问题,提出了时频联合长时循环神经网络(long term recurrent neural network with state-frequency memory, LTRNN-SFM),通过将传统循环神经网络隐藏层的状态向量替换为状态-频率矩阵,实现对时间序列的时域特征和频域特征的联合建模.同时,通过解耦隐藏层神经元、引入ReLU(rectified linear unit)激活函数和权重裁剪,该模型可以有效避免梯度消失和梯度爆炸问题的干扰,使得深层网络训练更加容易、网络记忆周期更长.实验表明:时频联合长时循环神经网络在处理长时依赖且频率成分丰富的时序数据上,取得了最好的性能.

关键词: 循环神经网络, 长时依赖, 频域分析, 时间序列分类, 时间序列预测

Abstract: Modeling time series has become one of the research hotspots in the field of machine learning because of its important application value. Recurrent neural network (RNN) is a crucial tool for modeling time series in recent years. However, existing RNNs are commonly hard to learn long-term dependency in the temporal domain and unable to model the frequency patterns in time series. The two problems seriously limit the performance of existing RNNs for the time series that contain long-term dependencies and rich frequency components. To solve these problems, we propose the long term recurrent neural network with state-frequency memory (LTRNN-SFM), which allows the network to model the uncovered features in both frequency and temporal domains by replacing state vector of the hidden layer in conventional RNNs to state-frequency matrix. Meanwhile, the proposed network can effectively avoid the interference of the gradient vanishing and exploding problems by separating neurons in the same layer, using activation functions such as rectified linear unit (ReLU) and clipping weight. In this way, a LTRNN-SFM with long-term memory and multiple layers can be trained easily. Experimental results have demonstrated that the proposed network achieves the best performance in processing time series with long-term dependencies and rich frequency components.

Key words: recurrent neural network (RNN), long-term dependency, frequency domain analysis, time series classification, time series prediction

中图分类号: