ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2016, Vol. 53 ›› Issue (9): 2107-2131.

### Energy Consumption Modeling and Optimization Analysis for MapReduce

Liao Bin1,4, Zhang Tao2,3, Yu Jiong2,4, Yin Lutong4, Guo Gang4, Guo Binglei2,4

1. 1(School of Statistics and Information, Xinjiang University of Finance and Economics, Urumqi 830012);2(School of Information Science and Engineering, Xinjiang University, Urumqi 830046);3(College of Medical Engineering and Technology, Xinjiang Medical University, Urumqi 830011);4(School of Software, Xinjiang University, Urumqi 830008)
• Online:2016-09-01

Abstract: The continuous expansion of the cloud computing centers scale and neglect of energy consumption factors exposed the problem of high energy consumption and low efficiency. To improve the MapReduce framework utilization of energy consumption, we build an energy consumption model for MapReduce framework. First, we propose a task energy consumption model which is based on CPU utilization estimation, energy consumption accumulation of main components and the average energy consumption estimation as well as the job energy consumption model of MapReduce. Specifically, after analyzing the energy optimization under energy consumption model, we come up with three directions to optimize energy consumption of MapReduce: optimize MapReduce energy consumption of job execution, reduce MapReduce energy consumption of task waiting and improve the energy utilization rate of MapReduce cluster. We further propose the data placement policy to decrease energy consumption of task waiting under heterogeneous environment and the minimum resource allocation algorithms to improve energy utilization rate of MapReduce jobs by the deadline constraints. A large number of experiments and data analysis of energy consumption demonstrate the effectiveness of energy consumption model and optimum policy of energy consumption.

CLC Number: