Energy Consumption Modeling and Optimization Analysis for MapReduce

The continuous expansion of the cloud computing centers scale and neglect of energy consumption factors exposed the problem of high energy consumption and low efficiency. To improve the MapReduce framework utilization of energy consumption, we build an energy consumption model for MapReduce framework. First, we propose a task energy consumption model which is based on CPU utilization estimation, energy consumption accumulation of main components and the average energy consumption estimation as well as the job energy consumption model of MapReduce. Specifically, after analyzing the energy optimization under energy consumption model, we come up with three directions to optimize energy consumption of MapReduce: optimize MapReduce energy consumption of job execution, reduce MapReduce energy consumption of task waiting and improve the energy utilization rate of MapReduce cluster. We further propose the data placement policy to decrease energy consumption of task waiting under heterogeneous environment and the minimum resource allocation algorithms to improve energy utilization rate of MapReduce jobs by the deadline constraints. A large number of experiments and data analysis of energy consumption demonstrate the effectiveness of energy consumption model and optimum policy of energy consumption.