Fusion Learning Based Unsupervised Anomaly Detection for Multi-Dimensional Time Series

Zhou Xiaohui; Wang Yijie; Xu Hongzuo; Liu Mingyu

doi:10.7544/issn1000-1239.202220490

Journal of Computer Research and Development > 2023 > 60(3): 496-508. > DOI: 10.7544/issn1000-1239.202220490 CSTR: 32373.14.issn1000-1239.202220490

Zhou Xiaohui, Wang Yijie, Xu Hongzuo, Liu Mingyu. Fusion Learning Based Unsupervised Anomaly Detection for Multi-Dimensional Time Series[J]. Journal of Computer Research and Development, 2023, 60(3): 496-508. DOI: 10.7544/issn1000-1239.202220490

Citation:

PDF (1545 KB)

Fusion Learning Based Unsupervised Anomaly Detection for Multi-Dimensional Time Series

National Laboratory for Parallel and Distributed Processing (National University of Defense Technology), Changsha 410073
College of Computer, National University of Defense Technology, Changsha 410073

Funds: This work was supported by the National Key Research and Development Program of China (2022ZD0115302), the National Natural Science Foundation of China (61379052), the Science Foundation of Ministry of Education of China(2018A02002), and the Natural Science Foundation for Distinguished Young Scholars of Hunan Province(14JJ1026).

More Information

Received Date: June 09, 2022
Revised Date: September 26, 2022
Available Online: February 26, 2023

Graphical Abstract

Abstract

Abstract

With the arrival of the multi-cloud era, cloud-based intelligent operations can detect and handle cloud platform failures in advance to ensure their high availability. Because of the complexity of cloud systems, operational data shows various temporal dependency and inter-metric dependency in data locality and data globality, which brings great challenges to multi-dimensional time series anomaly detection. However, most of the existing anomaly detection methods for multi-dimensional time series learn feature representation from normal time series data and detect anomalies based on reconstruction error or prediction error. These methods cannot capture the local and global information dependency of multi-dimensional time series at the same time, resulting in poor anomaly detection effect. To solve the above problems, a fusion learning based unsupervised anomaly detection method for multi-dimensional time series is proposed. The local features and global features of multi-dimensional time series are modeled simultaneously to obtain more abundant time series reconstruction information, and anomalies are detected based on reconstruction errors. Specifically, by introducing the self-attention mechanism to the temporal convolutional network, the proposed model pays more attention to the global features of data while modeling the partial correlation of data. In addition, the information sharing mechanism is added between the temporal convolutional module and the self-attention module to realize the information fusion, so as to better reconstruct the normal mode of multi-dimensional time series. Our experimental results on multi-dimensional time series real-world datasets show that the proposed method improves F1 score by up to 0.0882 compared with the previous multi-dimensional time series anomaly detection.
- multi-dimensional time series,
- unsupervised anomaly detection,
- information dependency,
- fusion learning,
- self-attention

FullText(HTML)

References (41)

References

[1]	Wang Huaiming, Shi Peichang, Zhang Yiming. Jointcloud: A cross-cloud cooperation architecture for integrated Internet service customization[C] //Proc of the 37th Int Conf on Distributed Computing Systems. Piscataway, NJ: IEEE, 2017: 1846−1855
[2]	Chalapathy R, Chawla S. Deep learning for anomaly detection: A survey [J]. arXiv preprint, arXiv: 1901.03407, 2019
[3]	Audibert J, Michiardi P, Guyard F, et al. USAD: Unsupervised anomaly detection on multivariate time series[C] //Proc of the 26th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2020: 3395−3404
[4]	Tran L, Mun M Y, Shahabi C. Real-time distance-based outlier detection in data streams[J]. Proceedings of the VLDB Endowment, 2020, 14(2): 141−153 doi: 10.14778/3425879.3425885
[5]	He Xin, Zhao Kaiyong, Chu Xiaowen. AutoML: A survey of the state-of-the-art[J]. arXiv preprint, arXiv: 1908.00709, 2019
[6]	Kingsbury K, Alvaro P. Elle: Inferring isolation anomalies from experimental observations[J]. Proceedings of the VLDB Endowment, 2020, 14(3): 268−280 doi: 10.14778/3430915.3430918
[7]	Witkowski K. Internet of things, big data, industry 4.0-innovative solutions in logistics and supply chains management[J]. Procedia Engineering, 2017, 182(3): 763−769
[8]	Ren Hansheng, Xu Bixiong, Wang Yujing, et al. Time-series anomaly detection service at microsoft[C] //Proc of the 25th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2019: 3009−3017
[9]	Hundman K, Constantinou V, Laporte C, et al. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding[C] //Proc of the 24th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2018: 387−395
[10]	Malhotra P, Ramakrishnan A, Anand G, et al. LSTM-based encoder-decoder for multi-sensor anomaly detection[J]. arXiv preprint, arXiv: 1607.00148, 2016
[11]	Park D, Hoshi Y, Kemp C C. A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018, 3(3): 1544−1551 doi: 10.1109/LRA.2018.2801475
[12]	Park D, Kim H, Hoshi Y, et al. A multimodal execution monitor with anomaly classification for robot-assisted feeding[C] //Proc of the 30th IEEE/RSJ Int Conf on Intelligent Robots and Systems. Piscataway, NJ: IEEE, 2017: 5406−5413
[13]	Chandola V, Banerjee A, Kumar V. Anomaly detection: A survey[J]. ACM Computing Surveys, 2009, 41(3): 1−58
[14]	Pang Guansong, Shen Chunhua, Cao Longbing, et al. Deep learning for anomaly detection: A review[J]. ACM Computing Surveys, 2021, 54(2): 1−38
[15]	Zhang Chuxu, Song Dongjin, Chen Yuncong, et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[C] //Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 1409−1416
[16]	Li Dan, Chen Dacheng, Jin Baihong, et al. MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks[C] //Proc of the 28th Int Conf on Artificial Neural Networks. Berlin: Springer, 2019: 703−716
[17]	Su Ya, Zhao Youjian, Niu Chenhao, et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C] //Proc of the 25th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2019: 2828−2837
[18]	Zong Bo, Song Qi, Min M R, et al. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection[C/OL] //Proc of the 6th Int Conf on Learning Representations. 2018[2022-06-10].https://sites.cs.ucsb.edu /~bzong/doc/iclr18-dagmm.pdf
[19]	Huang Siteng, Wang Donglin, Wu Xuehan, et al. DSANet: Dual self-attention network for multivariate time series forecasting[C] //Proc of the 28th ACM Int Conf on Information and Knowledge Management. New York: ACM, 2019: 2129−2132
[20]	Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735−1780 doi: 10.1162/neco.1997.9.8.1735
[21]	Kingma D P, Welling M. Auto-encoding variational Bayes[J]. arXiv preprint, arXiv: 1312.6114, 2013
[22]	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C] //Proc of the 27th Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2014: 2672−2680
[23]	Laptev N, Amizadeh S, Flint I. Generic and scalable framework for automated time-series anomaly detection[C] //Proc of the 21st ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining. New York: ACM, 2015: 1939−1947
[24]	Lai K H, Zha Daochen, Xu Junjie, et al. Revisiting time series outlier detection: Definitions and benchmarks[C] //Proc of the 35th Conf on Neural Information Processing Systems Datasets and Benchmarks Track. Cambridge, MA: MIT Press, 2021: 1−13
[25]	Yaacob A H, Tan I K T, Chien S F, et al. Arima based network anomaly detection[C] //Proc of the 2nd Int Conf on Communication Software and Networks. Piscataway, NJ: IEEE, 2010: 205−209
[26]	Rousseeuw P J, Leroy A M. Robust Regression and Outlier Detection[M]. New York: John Wiley & Sons, 2005
[27]	Bontemps L, Cao V L, McDermott J, et al. Collective anomaly detection based on long short-term memory recurrent neural networks[C] //Proc of the 3rd Int Conf on Future Data and Security Engineering. Berlin: Springer, 2016: 141−152
[28]	Elsayed S, Thyssens D, Rashed A, et al. Do we really need deep learning models for time series forecasting?[J]. arXiv preprint, arXiv: 2101.02118, 2021
[29]	Bai S, Kolter J Z, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[J]. arXiv preprint, arXiv: 1803.01271, 2018
[30]	Garg A, Zhang Wenyu, Samaran J, et al. An evaluation of anomaly detection and diagnosis in multivariate time series[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(6): 2508−2517 doi: 10.1109/TNNLS.2021.3105827
[31]	Deldari S, Smith D V, Xue Hao, et al. Time series change point detection with self-supervised contrastive predictive coding[C] //Proc of the 30th Web Conf. New York: ACM, 2021: 3124−3135
[32]	Gugulothu N, Malhotra P, Vig L, et al. Sparse neural networks for anomaly detection in high-dimensional time series[C/OL] //Proc of AI4IOT Workshop in Conjunction with ICML, IJCAI and ECAI. San Francisco: Morgan Kaufmann, 2018[2022-06-10].https://www.zurich.ibm.com/AI4IoT /2018/AI4IoT-18_Gugulothu.pdf
[33]	Deng A, Hooi B. Graph neural network-based anomaly detection in multivariate time series[C] //Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 4027−4035
[34]	Misra I, Shrivastava A, Gupta A, et al. Cross-stitch networks for multi-task learning[C] //Proc of the 29th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 3994−4003
[35]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C] //Proc of the 31st Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2017: 5998−6008
[36]	Li Zhihan, Zhao Youjian, Han Jiaqi, et al. Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding[C] //Proc of the 27th ACM SIGKDD Conf on Knowledge Discovery & Data Mining. New York: ACM, 2021: 3220−3230
[37]	Moody G B, Mark R G. The impact of the MIT-BIH arrhythmia database[J]. IEEE Engineering in Medicine and Biology Magazine, 2001, 20(3): 45−50 doi: 10.1109/51.932724
[38]	Kim S, Choi K, Choi H S, et al. Towards a rigorous evaluation of time-series anomaly detection[J]. arXiv preprint, arXiv: 2109.05257, 2021
[39]	Xu Haowen, Chen Wenxiao, Zhao Nengwen, et al. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications[C] //Proc of the 27th World Wide Web Conf. New York: ACM, 2018: 187−196
[40]	Liu F T, Ting K M, Zhou Zhihua. Isolation forest[C] //Proc of the 8th IEEE Int Conf on Data Mining. Piscataway, NJ: IEEE, 2008: 413−422
[41]	Li Zheng, Zhao Yue, Botta N, et al. COPOD: Copula-based outlier detection[C] //Proc of the 20th IEEE Int Conf on Data Mining. Piscataway, NJ: IEEE, 2020: 1118−1123

[1]	Wu Jingya, Lu Wenyan, Yan Guihai, Li Xiaowei. HyperTree: High Concurrent B+tree Index Accelerator[J]. Journal of Computer Research and Development, 2023, 60(7): 1661-1677. DOI: 10.7544/issn1000-1239.202111055
[2]	Yang Yongpeng, Jiang Dejun. A Method for Solving the wandering B+ tree Problem[J]. Journal of Computer Research and Development, 2023, 60(3): 539-554. DOI: 10.7544/issn1000-1239.202220555
[3]	Yan Wei, Zhang Xingjun, Ji Zeyu, Dong Xiaoshe, Ji Chenzhao. One-Direction Shift B⁺-Tree Based on Persistent Memory[J]. Journal of Computer Research and Development, 2021, 58(2): 371-383. DOI: 10.7544/issn1000-1239.2021.20200403
[4]	Te Rigen, Li Wei, and Li Xiongfei. Storage Model and Implementation of the Dynamic Ordered Tree[J]. Journal of Computer Research and Development, 2013, 50(5): 969-985.
[5]	Shen Yan, Song Shunlin, Zhu Yuquan. Mining Algorithm of Association Rules Based on Disk Table Resident FP-TREE[J]. Journal of Computer Research and Development, 2012, 49(6): 1313-1322.
[6]	Wang Hongqiang, Li Jianzhong, and Wang Hongzhi. Processing XPath over F&B-Index[J]. Journal of Computer Research and Development, 2010, 47(5): 866-877.
[7]	Zhou Da, Liang Zhichao, Meng Xiaofeng. HF-Tree: An Update-Efficient Index for Flash Memory[J]. Journal of Computer Research and Development, 2010, 47(5): 832-840.
[8]	Sun Xiaojuan, Sun Ninghui, Chen Mingyu. Optimization of B-NIDS for Multicore[J]. Journal of Computer Research and Development, 2007, 44(10): 1733-1740.
[9]	Ju Dapeng, Li Ming, Hu Jinfeng, Wang Dongsheng, Zheng Weimin, and Ma Yongquan. An Algorithm of B\++ Tree Management in P2P Environment[J]. Journal of Computer Research and Development, 2005, 42(8): 1438-1444.
[10]	Dong Daoguo, Liang Liuhong, and Xue Xiangyang. VAR-Tree—A New High-Dimensional Data Index Structure[J]. Journal of Computer Research and Development, 2005, 42(1): 10-17.

Cited By

Cited by

Periodical cited type(1)

1.	LUO Haoran，HU Shuisong，WANG Wenyong，TANG Yuke，ZHOU Junwei. Research on Multi-Core Processor Analysis for WCET Estimation. ZTE Communications. 2024(01): 87-94 .