ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2021, Vol. 58 ›› Issue (12): 2783-2797.doi: 10.7544/issn1000-1239.2021.20200366

Previous Articles     Next Articles

Adaptive Virtual Machine Consolidation Method Based on Deep Reinforcement Learning

Yu Xian1,2, Li Zhenyu1, Sun Sheng1,2, Zhang Guangxing1, Diao Zulong1, Xie Gaogang1   

  1. 1(Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190);2(University of Chinese Academy of Sciences, Beijing 100049)
  • Online:2021-12-01
  • Supported by: 
    This work was supported by the National Natural Science Foundation of China (61725206, U20A20180), and the CAS-Austria Project Plan (171111KYSB20200001).

Abstract: The problem of service quality optimization with energy consumption restriction has always been one of the big challenges for virtual machine (VM) resource management in data centers. Although existing work has reduced energy consumption and improved system service quality to a certain extent through VM consolidation technology, these methods are usually difficult to achieve long-term optimal management goals. Moreover, their performance is susceptible to the change of application scenarios, such that they are difficult to be replaced and will produce much management cost. In view of the problem that VM resource management in data center is hard to achieve long-term optimal energy efficiency and service quality, and also has poor flexibility in policy adjustment, this paper proposes an adaptive VM consolidation method based on deep reinforcement learning. This method builds an end-to-end decision-making model from data center system state to VM migration strategy through state tensor representation, deterministic action output, convolution neural network and weighted reward mechanism; It also designs an automatic state generation mechanism and an inverting gradient limitation mechanism to improve deep deterministic strategy gradient algorithm, speed up the convergence speed of VM migration decision-making model, and guarantee the approximately optimal management performance. Simulation experiment results based on real VM load data show that compared with popular VM consolidation methods in open source cloud platforms, this method can effectively reduce energy consumption and improve system service quality.

Key words: data center, VM resource management, VM consolidation, reinforcement learning, deep deterministic policy gradient (DDPG)

CLC Number: