ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2018, Vol. 55 ›› Issue (8): 1706-1716.doi: 10.7544/issn1000-1239.2018.20180310

所属专题: 2018数据挖掘前沿进展专题

• 人工智能 • 上一篇    下一篇

密集异构网络中基于强化学习的流量卸载算法

王倩1,2,聂秀山1,尹义龙2   

  1. 1(山东财经大学计算机科学与技术学院 济南 250014);2(山东大学齐鲁软件学院 济南 250101) (qianwang@sdu.edu.cn)
  • 出版日期: 2018-08-01
  • 基金资助: 
    国家自然科学基金项目(61573219,61671274);山东省重点研发计划项目(2017CXGC1504);山东省自然科学基金项目(ZR2017MF053);中国博士后科学基金面上项目(2016M602141);山东省高校优势学科人才团队培育计划 This work was supported by the National Natural Science Foundation of China (61573219, 61671274), Shandong Provincial Key Research and Development Plan (2017CXGC1504), the Natural Science Foundation of Shandong Province of China (ZR2017MF053), the China Postdoctoral Science Foundation (2016M602141), and the Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

A Reinforcement Learning Algorithm for Traffic Offloading in Dense Heterogeneous Network

Wang Qian1,2, Nie Xiushan1,Yin Yilong2   

  1. 1(Department of Computer Science and Technology, Shandong University of Finance and Economics, Jinan 250014);2(Software College, Shandong University, Jinan 250101)
  • Online: 2018-08-01

摘要: 近年来互联网用户规模和网络流量呈现爆炸式的增长,不断逼近蜂窝移动通信网络的容量极限.流量卸载技术可充分利用现有网络,将蜂窝网络的部分流量卸载到空闲网络中,进行跨网协作实现对蜂窝网络容量的极大提升,可有效解决有限的无线带宽资源与海量高速业务需求的矛盾.将强化学习的思想引入流量卸载算法中,提出了一种异构网络中基于强化学习的流量卸载算法.该算法把流量卸载问题映射为一个强化学习问题.基于前一状态完成的动作,以WiFi网络吞吐量作为回报函数,准确地预测需卸载的流量,并计算当前网络的最大卸载量,寻找最佳的WiFi网络接入点(access point, AP),并推导出最优的流量卸载判决规则,达到异构网络整体吞吐量最大化.仿真结果表明:基于Q学习的流量卸载算法可有效地实现自适应流量卸载控制规则,有效地避免过度卸载引起的碰撞冲突和系统性能急剧恶化,达到跨网协作的负载均衡点,在保证WiFi用户服务质量的条件下,最大限度地提高LTE系统吞吐量,保证密集异构网络的整体性能.

关键词: 强化学习, 密集异构网络, 流量卸载, 吞吐量, 效用函数

Abstract: With the explosive growth of numbers of Internet users and network traffic, the capacity of cellular mobile communication is already limited. In order to solve the contradiction between the increasing demand for high capacity and the limited resource, traffic offloading technology makes full use of the existing network, which offloads part of traffic from the cellular network into the other network and carries on the cooperation between networks, to improve the capacity of the cellular network greatly. Traffic offloading becomes one of the hot topics in the future research of wireless communication technology. In this paper, based on reinforcement learning, we propose a novel reinforcement learning algorithm for traffic offloading in dense heterogeneous network. Based on the previous experience and performance gain of the user offloading, this algorithm considers the system throughput of each state, and finds the optimal WiFi network access point (AP) by calculating the reward value. We also derive the optimal policy of traffic offloading decision to maximize the throughput of the system. Simulation results show that the reinforcement learning for traffic offloading can effectively avoid the collision caused by over offloading and rapid deterioration of system performance. Our scheme can effectively implement the adaptive traffic offloading control policy and achieve the cooperation between LTE and WiFi network guaranteeing the quality of service for users. The overall throughput of the dense heterogeneous network also reaches the maximum.

Key words: reinforcement learning, dense heterogeneous network, traffic offloading, throughput, utility function

中图分类号: