时频联合长时循环神经网络

庄连生; 吕扬; 杨健; 李厚强

doi:10.7544/issn1000-1239.2019.20180474

时频联合长时循环神经网络

¹(中国科学技术大学信息科学技术学院合肥 230027)
²(军事科学院系统工程研究院北京 100191)
³(鹏城实验室广东深圳 518000) (lszhuang@ustc.edu.cn)

基金项目: 国家自然科学基金项目(61472379)

详细信息

中图分类号: TP391; TP18
计量
- 文章访问数: 1038
- HTML全文浏览量: 5
- PDF下载量: 457
出版历程
- 发布日期: 2019-11-30

Long Term Recurrent Neural Network with State-Frequency Memory

¹(School of Information Science and Technology, University of Science and Technology of China,Hefei 230027)
²(Institute of System Engineering, Academy of Military Science, Beijing 100191)
³(Peng Cheng Laboratory, Shenzhen, Guangdong 518000)

摘要

摘要: 时间序列建模问题因有着重要的应用价值已经成为机器学习领域的研究热点之一.循环神经网络(recurrent neural network, RNN)是近年来时间序列建模的一个重要工具.但是，现有循环神经网络无法处理长时依赖关系的时序数据，也没有在频域对时间序列数据的特征模式进行建模.对于那些包含长时依赖且频率成分丰富的时序数据，这2个问题大大限制了现有循环神经网络的性能.针对这些问题，提出了时频联合长时循环神经网络(long term recurrent neural network with state-frequency memory, LTRNN-SFM)，通过将传统循环神经网络隐藏层的状态向量替换为状态-频率矩阵，实现对时间序列的时域特征和频域特征的联合建模.同时，通过解耦隐藏层神经元、引入ReLU(rectified linear unit)激活函数和权重裁剪，该模型可以有效避免梯度消失和梯度爆炸问题的干扰，使得深层网络训练更加容易、网络记忆周期更长.实验表明：时频联合长时循环神经网络在处理长时依赖且频率成分丰富的时序数据上，取得了最好的性能.
- 循环神经网络 /
- 长时依赖 /
- 频域分析 /
- 时间序列分类 /
- 时间序列预测
Abstract: Modeling time series has become one of the research hotspots in the field of machine learning because of its important application value. Recurrent neural network (RNN) is a crucial tool for modeling time series in recent years. However, existing RNNs are commonly hard to learn long-term dependency in the temporal domain and unable to model the frequency patterns in time series. The two problems seriously limit the performance of existing RNNs for the time series that contain long-term dependencies and rich frequency components. To solve these problems, we propose the long term recurrent neural network with state-frequency memory (LTRNN-SFM), which allows the network to model the uncovered features in both frequency and temporal domains by replacing state vector of the hidden layer in conventional RNNs to state-frequency matrix. Meanwhile, the proposed network can effectively avoid the interference of the gradient vanishing and exploding problems by separating neurons in the same layer, using activation functions such as rectified linear unit (ReLU) and clipping weight. In this way, a LTRNN-SFM with long-term memory and multiple layers can be trained easily. Experimental results have demonstrated that the proposed network achieves the best performance in processing time series with long-term dependencies and rich frequency components.
- recurrent neural network (RNN) /
- long-term dependency /
- frequency domain analysis /
- time series classification /
- time series prediction

HTML全文

参考文献(0)

施引文献(22)

期刊类型引用(8)

1.	袁振，侯玉亮，杜宇慧. 面向脑核磁共振识别运动任务的门控循环单元方法. 中国图象图形学报. 2023(02): 589-600 . 百度学术
2.	姚志武，侯丽娜，文茂华. 基于LSTM算法的大坝坝体渗透压力预测. 水利建设与管理. 2023(08): 54-59 . 百度学术
3.	潘丹，林灵婷，翁凌雯，李棋，常尧. 基于时空双分支网络的行为检测与识别技术研究. 电子设计工程. 2023(18): 191-195 . 百度学术
4.	赵佳佳，刘磊. 鱼类集群运动的注意力模型研究. 软件导刊. 2022(06): 36-40 . 百度学术
5.	郭勇，赵康，潘力. 结合改进Bi-LSTM和CNN的文本情感分析. 信息技术. 2021(02): 50-55 . 百度学术
6.	高瑞，郝乐，刘宝，文静怡，陈宇航. 基于改进ResNet网络的井下钻杆计数方法. 工矿自动化. 2020(10): 32-37 . 百度学术
7.	许春冬，徐琅，周滨，凌贤鹏. 单通道语音增强技术的研究现状与发展趋势. 江西理工大学学报. 2020(05): 55-64 . 百度学术
8.	何永勃，李明伟. 基于循环神经网络的飞机货舱火灾快速识别算法. 消防科学与技术. 2020(11): 1490-1494 . 百度学术