Non-Stationary Multivariate Time Series Prediction with MIX Gated Unit
-
摘要: 非平稳多变量时间序列(non-stationary multivariate time series, NSMTS)预测目前仍是一个具有挑战性的任务.基于循环神经网络的深度学习模型，尤其是基于长短期记忆(long short-term memory, LSTM)和门循环单元(gated recurrent unit, GRU)的神经网络已获得了令人印象深刻的预测性能.尽管LSTM结构上较为复杂，却并不总是在性能上占优.最近提出的最小门单元(minimal gated unit, MGU)神经网络具有更简单的结构，并在图像处理和一些序列处理问题中能够提升训练效率.更为关键的是，实验中我们发现该门单元可以高效运用于NSMTS的预测，并达到了与基于LSTM和GRU的神经网络相当的预测性能.然而，基于这3类门单元的神经网络中，没有任何一类总能保证性能上的优势.为此提出了一种线性混合门单元(MIX gated unit, MIXGU)，试图利用该单元动态调整GRU和MGU的混合权重，以便在训练期间为网络中的每个MIXGU获得更优的混合结构.实验结果表明，与基于单一门单元的神经网络相比，混合2类门单元的MIXGU神经网络具有更优的预测性能.Abstract: Non-stationary multivariate time series (NSMTS) forecasting is still a challenging issue nowadays. The existing deep learning models based on recurrent neural networks (RNNs), especially long short-term memory (LSTM) and gated recurrent unit (GRU) neural networks, have received impressive performance in prediction. Although the architecture of the LSTM is relatively complex, it cannot always dominate in performance. Latest researches show that with a simpler gated unit structure, the minimal gated unit (MGU) can not only simplify the network architecture, but also improve the training efficiency in computer vision and some sequence problems. Most importantly, our experiments show that this kind of unit can be effectively applied to the NSMTS predictions and achieve comparable results with LSTM and MGU neural networks. However, none of the three gated unit based neural networks can always dominate in performance over all the NSMTS. Therefore, in this paper we propose a novel linear MIX gated unit (MIXGU). This gated unit can adjust the importance weights of GRU and MGU dynamically to achieve a better hybrid structure for each MIXGU in the network during training. The experimental results show that this MIXGU neural network has higher prediction performance than other state-of-the-art one gated unit neural network models.
-
计量
- 文章访问数: 1498
- HTML全文浏览量: 3
- PDF下载量: 740
下载:
