Long Term Recurrent Neural Network with State-Frequency Memory
-
摘要: 时间序列建模问题因有着重要的应用价值已经成为机器学习领域的研究热点之一.循环神经网络(recurrent neural network, RNN)是近年来时间序列建模的一个重要工具.但是，现有循环神经网络无法处理长时依赖关系的时序数据，也没有在频域对时间序列数据的特征模式进行建模.对于那些包含长时依赖且频率成分丰富的时序数据，这2个问题大大限制了现有循环神经网络的性能.针对这些问题，提出了时频联合长时循环神经网络(long term recurrent neural network with state-frequency memory, LTRNN-SFM)，通过将传统循环神经网络隐藏层的状态向量替换为状态-频率矩阵，实现对时间序列的时域特征和频域特征的联合建模.同时，通过解耦隐藏层神经元、引入ReLU(rectified linear unit)激活函数和权重裁剪，该模型可以有效避免梯度消失和梯度爆炸问题的干扰，使得深层网络训练更加容易、网络记忆周期更长.实验表明：时频联合长时循环神经网络在处理长时依赖且频率成分丰富的时序数据上，取得了最好的性能.Abstract: Modeling time series has become one of the research hotspots in the field of machine learning because of its important application value. Recurrent neural network (RNN) is a crucial tool for modeling time series in recent years. However, existing RNNs are commonly hard to learn long-term dependency in the temporal domain and unable to model the frequency patterns in time series. The two problems seriously limit the performance of existing RNNs for the time series that contain long-term dependencies and rich frequency components. To solve these problems, we propose the long term recurrent neural network with state-frequency memory (LTRNN-SFM), which allows the network to model the uncovered features in both frequency and temporal domains by replacing state vector of the hidden layer in conventional RNNs to state-frequency matrix. Meanwhile, the proposed network can effectively avoid the interference of the gradient vanishing and exploding problems by separating neurons in the same layer, using activation functions such as rectified linear unit (ReLU) and clipping weight. In this way, a LTRNN-SFM with long-term memory and multiple layers can be trained easily. Experimental results have demonstrated that the proposed network achieves the best performance in processing time series with long-term dependencies and rich frequency components.
-
-
期刊类型引用(11)
1. 肖宇庭，吕晓琪，谷宇，刘传强. 基于拆分残差网络的糖尿病视网膜病变分类. 广西师范大学学报(自然科学版). 2024(01): 91-101 . 
百度学术
2. 吕德珍，赵玉，苗素琴. 基于分布式多节点医疗管理系统进程设计. 计算机与数字工程. 2024(02): 382-387 . 百度学术
3. 盛文娟，赖振谱，杨宁，Peng Gangding. 基于改进AdaBoost算法的可调谐F-P滤波器温漂补偿方法. 光学学报. 2023(03): 48-56 . 百度学术
4. 傅懋钟，胡海洋，李忠金. 面向GPU集群的动态资源调度方法. 计算机研究与发展. 2023(06): 1308-1321 . 本站查看
5. 杨小琴，朱玉全. 基于距离限定优化的多姿态人脸图像智能识别. 计算机仿真. 2022(01): 200-203+282 . 百度学术
6. 王昕. 梯度下降及优化算法研究综述. 电脑知识与技术. 2022(08): 71-73 . 百度学术
7. 赵永亮，于倩，邓博，韩丽君，高红梅. 基于博弈论及机器学习的最优化算法设计与仿真. 电子设计工程. 2022(13): 23-27 . 百度学术
8. 李晓锋，燕少飞，吴宸. 移动终端操作系统应用程序恶意检测系统技术研究. 电子技术与软件工程. 2022(17): 75-79 . 百度学术
9. 蒋平. 基于卷积神经网络的图像精度深度优化. 淮阴工学院学报. 2021(03): 30-34 . 百度学术
10. 杨国葳，李宏坤，张明亮，黄刚劲. 基于一维深度卷积自动编码器的刀具状态监测方法. 振动与冲击. 2021(21): 223-233+274 . 百度学术
11. 郑雯，沈琪浩，任佳. 基于Improved DR-Net算法的糖尿病视网膜病变识别与分级. 光学学报. 2021(22): 72-83 . 百度学术
其他类型引用(24)
计量
- 文章访问数: 1036
- HTML全文浏览量: 5
- PDF下载量: 456
- 被引次数: 35
下载:
