Dynamic Decision-Driven Threat Detection Method for Data Elements in Industrial Control Networks
-
摘要:
近年来，工控网络发展势头迅猛. 其数字化、智能化、自动化的优势为工业带来巨大效益的同时，也面临着愈发复杂多变的攻击威胁. 在数据要素安全的背景下，及时发现和应对工控网络威胁成为一项迫切需要得到解决的任务. 通过对工控网络中的数据流进行连续监测和分析，工控网络威胁检测问题可以转化为时间序列异常检测问题. 然而现有时间序列异常检测方法受限于工控网络数据集的质量，且往往仅对单一类型异常敏感而忽略其他异常. 针对上述问题，提出了一种基于深度强化学习和数据增强的工控网络威胁检测方法（deep reinforcement learning and data augmentation based threat detection method in industrial control networks，DELTA）. 该方法提出了一种新的时序数据集数据增强选择方法，可以针对不同的基准模型选择合适的数据增强操作集以提升工控网络时间序列数据集的质量；同时使用深度强化学习算法（A2C/PPO）在不同时间点从基线模型中动态选取候选模型，以利用多种类型的异常检测模型解决单一类型异常敏感问题. 与现有时间序列异常检测模型对比的实验结果表明，在付出可接受的额外时间消耗成本下，DELTA在准确率和F1值上比所有基线模型有明显的提升，验证了方法的有效性与实用性.
Abstract:In recent years, the industrial control network has been developing rapidly. The advantages of digitization, intelligence, and automation have brought significant benefits to the industry while also introducing increasingly complex and variable attack threats. In the context of data element security, timely detection and response to industrial control network threats have become an urgent task to be solved. By continuously monitoring and analyzing the data flow in industrial control networks, the problem of industrial control network threat detection can be transformed into a time series anomaly detection problem. However, the existing time-series anomaly detection methods are limited by the quality of industrial control network datasets and are often sensitive to only a single type of anomaly while ignoring other anomalies. Therefore, in this paper, we propose a deep reinforcement learning and data augmentation based threat detection method in industrial control networks (DELTA). DELTA introduces a novel data augmentation selection technique for time series datasets, which allows for the selection of appropriate data augmentation operations sets tailored to different baseline models to enhance the quality of the industrial control network time series datasets. Simultaneously, deep reinforcement learning algorithms (A2C/PPO) dynamically select candidate models from the baseline models at different time points, leveraging multiple types of anomaly detection models to address the issue of sensitivity to single-type anomalies. The experimental results compared with the existing time series anomaly detection models show that DELTA has a significant improvement in accuracy and F1 value over all baseline models at an acceptable cost of additional time consumption, which verifies the effectiveness and practicality of the method.
-
-
![]()
图 7 阈值d对DELTA模型性能的影响
Figure 7. Impact of threshold d on the performance of DELTA model
![]()
图 8 不同模型的训练时间和推理时间对比
Figure 8. Comparison of training time and inference time of different models
表 1 主要参数列表
Table 1 List of the Major Parameters
参数 取值 PPO学习率 3E–4 A2C学习率 7E–4 折扣因子γ 0.99 裁剪因子ε 0.2 批次大小 64 最大仿真步数 2048 参数更新轮数 10 梯度裁剪最大值 0.5 滑动窗口值 12 
下载: 导出CSV
表 2 总体性能结果
Table 2 Overall Performance Results
% 模型类型 模型 准确率 召回率 F1值 无监督学习 iForest 67.30 62.79 64.97 DIF 69.23 59.55 67.66 LODA 66.62 63.47 65.00 VAE 66.55 63.40 64.94 beta-VAE 67.53 64.33 65.89 ECOD 71.42 68.04 69.69 COPOD 71.62 68.23 69.88 强化学习 RLMSAD 71.75±0.39 68.31±0.06 69.99±0.19 DELTA-A（本文） 88.84±4.93 63.07±3.51 73.57±1.72 DELTA-P（本文） 88.40±2.64 46.68±6.34 60.88±5.72 注：黑体数值表示最优结果，“±”后的值表示多次实验的标准差.
下载: 导出CSV
表 3 数据增强模块对基准模型性能的影响
Table 3 Impact of Data Augmentation Module on the Performance of Baseline Model
% 模型 准确率 召回率 F1值 iForest w/o DA 67.30 62.79 64.97 iForest 69.69 66.39 68.00 DIF w/o DA 69.23 59.55 67.66 DIF 69.70 66.40 68.01 ECOD w/o DA 71.42 68.04 69.69 ECOD 72.22 68.81 70.47 COPOD w/o DA 71.62 68.23 69.88 COPOD 72.03 68.62 70.28 注：黑体数值表示最优结果.
下载: 导出CSV
表 4 数据增强模块对DELTA模型性能的影响
Table 4 Impact of Data Augmentation Module on the Performance of DELTA Model
% 模型 准确率 召回率 F1值 DELTA-A 88.84±4.93 63.07±3.51 73.57±1.72 DELTA-A w/o DA 87.44±7.34 55.04±18.75 64.51±20.27 DELTA-P 88.40±2.64 46.68±6.34 60.88±5.72 DELTA-P w/o DA 85.30±3.50 44.15±14.28 56.93±13.62 注：黑体数值表示最优结果，“±”后的值表示多次实验的标准差.
下载: 导出CSV
-
[1] Wu H S. A survey of research on anomaly detection for time series[C]//Proc of 2016 13th Int Computer Conf on Wavelet Active Media Technology and Information Processing (ICCWAMTIP). Piscataway, NJ: IEEE, 2016: 426−431
[2] Chalapathy R, Chawla S. Deep learning for anomaly detection: A survey[J]. arXiv preprint, arXiv: 1901.03407, 2019
[3] 徐丽娟,王佰玲,杨美红,等. 工业控制网络多模式攻击检测及异常状态评估方法[J]. 计算机研究与发展,2021,58(11):2333−2349 doi: 10.7544/issn1000-1239.2021.20210598 Xu Lijuan, Wang Bailing, Yang Meihong, et al. Multi-mode attack detection and evaluation of abnormal states for industrial control network[J]. Journal of Computer Research and Development, 2021, 58(11): 2333−2349 (in Chinese) doi: 10.7544/issn1000-1239.2021.20210598
[4] 席亮,王勇,张凤斌. 基于自适应人工鱼群FCM的异常检测算法[J]. 计算机研究与发展,2019,56(5):1048−1059 doi: 10.7544/issn1000-1239.2019.20180099 Xi Liang, Wang Yong, Zhang Fengbin. Anomaly detection algorithm based on FCM with adaptive artificial Fish-Swarm[J]. Journal of Computer Research and Development, 2019, 56(5): 1048−1059 (in Chinese) doi: 10.7544/issn1000-1239.2019.20180099
[5] 陈波冯,李靖东,卢兴见,等. 基于深度学习的图异常检测技术综述[J]. 计算机研究与发展,2021,58(7):1436−1455 doi: 10.7544/issn1000-1239.2021.20200685 Chen Bofeng, Li Jingdong, Lu Xingjian, et al. Survey of deep learning based graph anomaly detection methods[J]. Journal of Computer Research and Development, 2021, 58(7): 1436−1455 (in Chinese) doi: 10.7544/issn1000-1239.2021.20200685
[6] Audibert J, Michiardi P, Guyard F, et al. USAD: Unsupervised anomaly detection on multivariate time series[C]//Proc of the 26th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2020: 3395−3404
[7] Zhang Chuxu, Song Dongjin, Chen Yuncong, et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 1409−1416
[8] Park D, Hoshi Y, Kemp C C. A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018, 3(3): 1544−1551 doi: 10.1109/LRA.2018.2801475
[9] Su Ya, Zhao Youjian, Niu Chenhao, et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]//Proc of the 25th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2019: 2828−2837
[10] Xin Ruyue, Liu Hongyun, Chen Peng, et al. Robust and accurate performance anomaly detection and prediction for cloud applications: A novel ensemble learning-based framework[J]. Journal of Cloud Computing, 2023, 12(1): 1−16 doi: 10.1186/s13677-022-00383-6
[11] Zhao Yubo, Guo Ni, Chen Wei, et al. Multi-step ahead forecasting for electric power load using an ensemble model[J]. Expert Systems with Applications, 2023, 211: 118649 doi: 10.1016/j.eswa.2022.118649
[12] Aggarwal C C, Sathe S. Theoretical foundations and algorithms for outlier ensembles[J]. ACM Sigkdd Explorations Newsletter, 2015, 17(1): 24−47 doi: 10.1145/2830544.2830549
[13] Tama B A, Nkenyereye L, Islam S M R, et al. An enhanced anomaly detection in web traffic using a stack of classifier ensemble[J]. IEEE Access, 2020, 8: 24120−24134 doi: 10.1109/ACCESS.2020.2969428
[14] Adeyemo V E, Abdullah A, JhanJhi N Z, et al. Ensemble and deep-learning methods for two-class and multi-attack anomaly intrusion detection: An empirical study[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(9): 520−528
[15] Ünlü R, Xanthopoulos P. A weighted framework for unsupervised ensemble learning based on internal quality measures[J]. Annals of Operations Research, 2019, 276: 229−247 doi: 10.1007/s10479-017-2716-8
[16] Wen Qingsong, Sun Liang, Yang Fan, et al. Time series data augmentation for deep learning: A survey[C]//Proc of the 30th Int Joint Conf on Artificial Intelligence. Freiburg, German: IJCAI, 2021: 4653−4660
[17] Gao Jingkun, Song Xiaomin, Wen Qingsong, et al. Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks[C]//Proc of ACM SIGKDD Workshop on Mining and Learning from Time Series (KDD-MiLeTS 2020). New York: ACM, 2020: 1−9
[18] Steven Eyobu O, Han D S. Feature representation and data augmentation for human activity classification based on wearable IMU sensor data using a deep LSTM neural network[J]. Sensors, 2018, 18(9): 2892−2917 doi: 10.3390/s18092892
[19] Park D S, Chan W, Zhang Yu, et al. SpecAugment: A simple data augmentation method for automatic speech recognition[C]. Proc of. Interspeech New York: ACM, 2019: 2613−2617
[20] Park D S, Chan W, Zhang Yu, et al. RobustSTL: A robust seasonal-trend decomposition algorithm for long time series[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019, 33(1): 5409−5416
[21] Li Yan, Lu Xinjiang, Wang Yaqing, et al. Generative time series forecasting with diffusion, denoise, and disentanglement[J]. Advances in Neural Information Processing Systems, 2022, 35: 23009−23022
[22] Kang Yanfei, Hyndman R J, Li Feng. GRATIS: GeneRAting time series with diverse and controllable characteristics[J]. Statistical Analysis and Data Mining: The ASA Data Science Journal, 2020, 13(4): 354−376 doi: 10.1002/sam.11461
[23] Devries T, Taylor G W. Dataset augmentation in feature space[J]. arXiv preprint, arXiv: 1702.05538, 2017
[24] Cheung T H, Yeung D Y. Modals: Modality-agnostic automated data augmentation in the latent space[C]//Proc of the 9th Int Conf on Learning Representations. Washington, DC: ICLR, 2021: 1−18
[25] Yoon J, Jarrett D, Van der Schaar M. Time-series generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2019, 32: 5508−5518
[26] Fons E, Dawson P, Zeng Xiaojun, et al. Adaptive weighting scheme for automatic time-series data augmentation[J]. arXiv preprint, arXiv: 2102.08310, 2021
[27] Zhang J E, Wu Di, Boulet B. Time series anomaly detection via reinforcement learning-based model selection[C]//Proc of 2022 IEEE Canadian Conf on Electrical and Computer Engineering (CCECE). Piscataway, NJ: IEEE, 2022: 193−199
[28] Sutton R S, Barto A G. Reinforcement Learning: An Introduction[M]. Cambridge, MA: MIT press, 2018
[29] Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint, arXiv: 1707.06347, 2017
[30] Goh J, Adepu S, Junejo K N, et al. A dataset to support research in the design of secure water treatment systems[C]//Proc of the 11th Int Conf on Critical Information Infrastructures Security (CRITIS 2016). Berlin: Springer, 2016: 88−99
[31] Liu F T, Ting Kaiming, Zhou Zhihua. Isolation forest[C]//Proc of 2008 8th IEEE Int Conf on Data Mining. Piscataway, NJ: IEEE, 2008: 413−422
[32] Xu Hongzuo, Pang Guansong, Wang Yijie, et al. Deep isolation forest for anomaly detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2023: 12591−12604
[33] Pevný T. Loda: Lightweight on-line detector of anomalies[J]. Machine Learning, 2016, 102: 275−304 doi: 10.1007/s10994-015-5521-0
[34] Kingma D P, Welling M. Auto-encoding variational Bayes[J]. arXiv preprint, arXiv: 1312.6114, 2013
[35] Higgins I, Matthey L, Pal A, et al. Beta-vae: Learning basic visual concepts with a constrained variational framework[C]//Proc of the 5th Int Conf on Learning Representations. Washington, DC: ICLR, 2017: 1−22
[36] Li Zheng, Zhao Yue, Hu Xiyang, et al. Ecod: Unsupervised outlier detection using empirical cumulative distribution functions[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 35(12): 12181−12193
[37] Li Zheng, Zhao Yue, Botta N, et al. COPOD: Copula-based outlier detection[C]//Proc of 2020 IEEE Int Conf on Data Mining (ICDM). Piscataway, NJ: IEEE, 2020: 1118−1123
[38] Scikit-learn. Scikit-Learn[EB/OL]. 2024 [2024-04-02]. https://github. com/scikit-learn/scikit-learn
[39] Zhao Yue. PyOD[EB/OL]. 2024 [2024-04-02]. https://github.com/ yzhao062/pyod
[40] Zhang J E. RLMSAD[EB/OL]. 2022 [2024-04-02]. https://github. com/elisejiuqizhang/RLMSAD
[41] Raffin A, Hill A, Gleave A, et al. Stable-baselines3: Reliable reinforcement learning implementations[J]. Journal of Machine Learning Research, 2021, 22(268): 1−8
-
期刊类型引用(2)
1. 聂萌瑶，刘鑫. 考虑最大通信量的物联网群体访问路由算法. 计算机仿真. 2024(02): 415-419 . 
百度学术
2. 陈淑平，李祎，何王全，漆锋滨. 胖树拓扑中高效实用的定制多播路由算法. 计算机研究与发展. 2022(12): 2689-2707 . 本站查看
其他类型引用(0)
计量
- 文章访问数: 333
- HTML全文浏览量: 43
- PDF下载量: 151
- 被引次数: 2
