基于牵引控制的深度强化学习路由策略生成

孙鹏浩; 兰巨龙; 申涓; 胡宇翔

doi:10.7544/issn1000-1239.2021.20200018

基于牵引控制的深度强化学习路由策略生成

(解放军战略支援部队信息工程大学郑州 450002) (sphshine@126.com)

基金项目: 国家重点研发计划项目(2020YFB1804803)；国家自然科学基金项目(62002382,61702547,61872382)；广东省重点领域研发计划项目(2018B010113001)

详细信息

中图分类号: TP393
计量
- 文章访问数: 742
- HTML全文浏览量: 0
- PDF下载量: 296
出版历程
- 发布日期: 2021-06-30

Pinning Control-Based Routing Policy Generation Using Deep Reinforcement Learning

(PLA Strategic Support Force Information Engineering University, Zhengzhou 450002)

Funds: This work was supported by the National Key Research and Development Program of China (2020YFB1804803), the National Natural Science Foundation of China (62002382, 61702547, 61872382), and the Key Research and Development Project of Guangdong Province (2018B010113001).

摘要

摘要: 当前网络规模的高速增长带来网络流量复杂度的日益提高，增加了对流量特征精确建模的难度.近年来业界提出使用深度强化学习技术实现网络路由的智能化生成，一定程度上克服了人工进行流量分析和建模的缺点.然而，目前提出的解决方案普遍存在可扩展性差等问题.对此，提出了一种基于牵引控制理论的深度强化学习路由策略生成技术Hierar-DRL，通过引入牵引控制理论并结合深度强化学习的自动策略搜索能力，提高了智能路由算法可扩展性.仿真实验结果表明：所提方案相比当前最优方案的端到端时延最多降低了28.5%，证明了所提智能路由方案的有效性.
- 路由优化 /
- 软件定义网络 /
- 人工智能 /
- 深度强化学习 /
- 牵引控制
Abstract: Computer networks have been playing an important role in modern society. The rapid growth of the network scale makes the network traffic more and more complicated, which is hard to accurately model. This condition makes the optimal routing policy in communication networks an NP-hard problem. To solve this problem, traditional methods for routing and traffic engineering mainly use hand-crafted algorithms, which cannot ensure both the accuracy and efficiency. In recent years, deep reinforcement learning (DRL)-based network routing strategies have been proposed, which overcome the shortcomings of manually analysis and modelling by human experts to some extent. However, current DRL-based routing strategies all have problems in scalability, which means they cannot be used in large scale networks. Under this circumstance, this paper proposes Hierar-DRL, a DRL-based network routing technology that employs pinning control theory. Pinning control helps Hierar-DRL to select a subset of network nodes as the target control nodes of DRL. With the advantages of pinning control and the automatic policy exploring ability of DRL, Hierar-DRL shows better scalability in large networks. Simulation results show that the proposed scheme can reduce the average end-to-end transmission delay in the test network topologies by up to 28.5% compared with the state-of-the-art, which validates the proposed scheme.
- routing optimization /
- software-defined networking /
- artificial intelligence /
- deep reinforce-ment learning /
- pinning control

HTML全文

参考文献(0)

施引文献(13)

期刊类型引用(9)

1.	孔晓光，李曼. 改进数字滤波的网络终端信息防泄漏方法仿真. 计算机仿真. 2024(01): 447-451 . 百度学术
2.	徐承志，徐磊，许春根. 基于可更新加密的保护搜索模式的动态可搜索加密方案. 计算机科学. 2024(03): 340-350 . 百度学术
3.	苏蕊，王亚婷，闫润珍，王悦. 基于近似匹配模型的电网多模态数据检索研究. 电子设计工程. 2024(07): 153-157 . 百度学术
4.	刘亦哲，包森成，方国强. 基于大数据的增量式隐私信息迭代加密. 电子设计工程. 2024(13): 154-157 . 百度学术
5.	任瑞仙，吕巾娇. 区块链技术下移动用户位置隐私可搜索加密. 计算机仿真. 2024(08): 418-422 . 百度学术
6.	黄一才，李森森，郁滨. 云环境下对称可搜索加密研究综述. 电子与信息学报. 2023(03): 1134-1146 . 百度学术
7.	马亚蕾，张怡. 基于区块链的通信网络节点位置隐私加密控制模型设计. 计算机测量与控制. 2023(04): 246-251 . 百度学术
8.	晏银芳，田维维. 基于随机森林的财务报表隐私数据自动加密方法研究. 兰州文理学院学报(自然科学版). 2023(04): 46-51 . 百度学术
9.	张蓝蓝，曹卫东，王怀超. 一种支持联合搜索的多用户动态对称可搜索加密方案. 计算机研究与发展. 2022(10): 2309-2322 . 本站查看