高级检索
    庄连生, 吕扬, 杨健, 李厚强. 时频联合长时循环神经网络[J]. 计算机研究与发展, 2019, 56(12): 2641-2648. DOI: 10.7544/issn1000-1239.2019.20180474
    引用本文: 庄连生, 吕扬, 杨健, 李厚强. 时频联合长时循环神经网络[J]. 计算机研究与发展, 2019, 56(12): 2641-2648. DOI: 10.7544/issn1000-1239.2019.20180474
    Zhuang Liansheng, Lü Yang, Yang Jian, Li Houqiang. Long Term Recurrent Neural Network with State-Frequency Memory[J]. Journal of Computer Research and Development, 2019, 56(12): 2641-2648. DOI: 10.7544/issn1000-1239.2019.20180474
    Citation: Zhuang Liansheng, Lü Yang, Yang Jian, Li Houqiang. Long Term Recurrent Neural Network with State-Frequency Memory[J]. Journal of Computer Research and Development, 2019, 56(12): 2641-2648. DOI: 10.7544/issn1000-1239.2019.20180474

    时频联合长时循环神经网络

    Long Term Recurrent Neural Network with State-Frequency Memory

    • 摘要: 时间序列建模问题因有着重要的应用价值已经成为机器学习领域的研究热点之一.循环神经网络(recurrent neural network, RNN)是近年来时间序列建模的一个重要工具.但是,现有循环神经网络无法处理长时依赖关系的时序数据,也没有在频域对时间序列数据的特征模式进行建模.对于那些包含长时依赖且频率成分丰富的时序数据,这2个问题大大限制了现有循环神经网络的性能.针对这些问题,提出了时频联合长时循环神经网络(long term recurrent neural network with state-frequency memory, LTRNN-SFM),通过将传统循环神经网络隐藏层的状态向量替换为状态-频率矩阵,实现对时间序列的时域特征和频域特征的联合建模.同时,通过解耦隐藏层神经元、引入ReLU(rectified linear unit)激活函数和权重裁剪,该模型可以有效避免梯度消失和梯度爆炸问题的干扰,使得深层网络训练更加容易、网络记忆周期更长.实验表明:时频联合长时循环神经网络在处理长时依赖且频率成分丰富的时序数据上,取得了最好的性能.

       

      Abstract: Modeling time series has become one of the research hotspots in the field of machine learning because of its important application value. Recurrent neural network (RNN) is a crucial tool for modeling time series in recent years. However, existing RNNs are commonly hard to learn long-term dependency in the temporal domain and unable to model the frequency patterns in time series. The two problems seriously limit the performance of existing RNNs for the time series that contain long-term dependencies and rich frequency components. To solve these problems, we propose the long term recurrent neural network with state-frequency memory (LTRNN-SFM), which allows the network to model the uncovered features in both frequency and temporal domains by replacing state vector of the hidden layer in conventional RNNs to state-frequency matrix. Meanwhile, the proposed network can effectively avoid the interference of the gradient vanishing and exploding problems by separating neurons in the same layer, using activation functions such as rectified linear unit (ReLU) and clipping weight. In this way, a LTRNN-SFM with long-term memory and multiple layers can be trained easily. Experimental results have demonstrated that the proposed network achieves the best performance in processing time series with long-term dependencies and rich frequency components.

       

    /

    返回文章
    返回