• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Liao Bin, Zhang Tao, Yu Jiong, Yin Lutong, Guo Gang, Guo Binglei. Energy Consumption Modeling and Optimization Analysis for MapReduce[J]. Journal of Computer Research and Development, 2016, 53(9): 2107-2131. DOI: 10.7544/issn1000-1239.2016.20148443
Citation: Liao Bin, Zhang Tao, Yu Jiong, Yin Lutong, Guo Gang, Guo Binglei. Energy Consumption Modeling and Optimization Analysis for MapReduce[J]. Journal of Computer Research and Development, 2016, 53(9): 2107-2131. DOI: 10.7544/issn1000-1239.2016.20148443

Energy Consumption Modeling and Optimization Analysis for MapReduce

More Information
  • Published Date: August 31, 2016
  • The continuous expansion of the cloud computing centers scale and neglect of energy consumption factors exposed the problem of high energy consumption and low efficiency. To improve the MapReduce framework utilization of energy consumption, we build an energy consumption model for MapReduce framework. First, we propose a task energy consumption model which is based on CPU utilization estimation, energy consumption accumulation of main components and the average energy consumption estimation as well as the job energy consumption model of MapReduce. Specifically, after analyzing the energy optimization under energy consumption model, we come up with three directions to optimize energy consumption of MapReduce: optimize MapReduce energy consumption of job execution, reduce MapReduce energy consumption of task waiting and improve the energy utilization rate of MapReduce cluster. We further propose the data placement policy to decrease energy consumption of task waiting under heterogeneous environment and the minimum resource allocation algorithms to improve energy utilization rate of MapReduce jobs by the deadline constraints. A large number of experiments and data analysis of energy consumption demonstrate the effectiveness of energy consumption model and optimum policy of energy consumption.
  • Related Articles

    [1]Lu Yuxuan, Kong Lanju, Zhang Baochen, Min Xinping. MC-RHotStuff: Multi-Chain Oriented HotStuff Consensus Mechanism Based on Reputation[J]. Journal of Computer Research and Development, 2024, 61(6): 1559-1572. DOI: 10.7544/issn1000-1239.202330195
    [2]Yu Xiao, Liu Hui, Lin Yuxiu, Zhang Caiming. Consensus Guided Auto-Weighted Multi-View Clustering[J]. Journal of Computer Research and Development, 2022, 59(7): 1496-1508. DOI: 10.7544/issn1000-1239.20210126
    [3]Yang Hongzhang, Yang Yahui, Tu Yaofeng, Sun Guangyu, Wu Zhonghai. Proactive Fault Tolerance Based on “Collection—Prediction—Migration—Feedback” Mechanism[J]. Journal of Computer Research and Development, 2020, 57(2): 306-317. DOI: 10.7544/issn1000-1239.2020.20190549
    [4]Wang Zuan, Tian Youliang, Yue Chaoyue, Zhang Duo. Consensus Mechanism Based on Threshold Cryptography Scheme[J]. Journal of Computer Research and Development, 2019, 56(12): 2671-2683. DOI: 10.7544/issn1000-1239.2019.20190053
    [5]Wei Songjie, Li Shuai, Mo Bing, Wang Jiahe. Regional Cooperative Authentication Protocol for LEO Satellite Networks Based on Consensus Mechanism[J]. Journal of Computer Research and Development, 2018, 55(10): 2244-2255. DOI: 10.7544/issn1000-1239.2018.20180431
    [6]Liu Yiran, Ke Junming, Jiang Han, Song Xiangfu. Improvement of the PoS Consensus Mechanism in Blockchain Based on Shapley Value[J]. Journal of Computer Research and Development, 2018, 55(10): 2208-2218. DOI: 10.7544/issn1000-1239.2018.20180439
    [7]Ye Songtao, Lin Yaping, Hu Yupeng, Zhou Siwang, You Zhiqiang. A Faulty Sensor Node Tolerance Algorithm Based on Cut Point Set[J]. Journal of Computer Research and Development, 2009, 46(12): 2117-2125.
    [8]Han Jianjun, Gan Lu, Ruan Youlin, Li Qinghua, Abbas A.Essa. Real-Time Dynamic Scheduling Algorithms for the Savings of Power Consumption and Fault Tolerance in Multi-Processor Computing Environment[J]. Journal of Computer Research and Development, 2008, 45(4): 706-715.
    [9]Cheng Xin, Liu Hongwei, Dong Jian, Yang Xiaozong. A Fault Tolerance Deadlock Detection/Resolution Algorithm for the AND-OR Model[J]. Journal of Computer Research and Development, 2007, 44(5): 798-805.
    [10]Luo Wei, Yang Fumin, Pang Liping, and Li Jun. A Real-Time Fault-Tolerant Scheduling Algorithm for Distributed Systems Based on Deferred Active Backup-Copy[J]. Journal of Computer Research and Development, 2007, 44(3).
  • Cited by

    Periodical cited type(7)

    1. 李翔硕,畅广辉,苏盛,阮冲,吴坡,李斌. 变电监控系统网络安全威胁指标研究综述与展望. 电力科学与技术学报. 2024(04): 1-10 .
    2. 高莉莉,高雪,林钰浩,吴钰博,范金鹏. 楼宇建筑空调系统设备错误连接关系自动检测算法. 制冷与空调(四川). 2022(02): 311-316+323 .
    3. 马标,胡梦娜,张重豪,周正寅,贾俊铖,杨荣举. 基于融合马尔科夫模型的工控网络流量异常检测方法. 信息安全学报. 2022(03): 17-32 .
    4. 燕敏,阮秀琴,赵阳,郑宏涛. 基于小样本学习的物联网异常状态修正算法. 计算机仿真. 2022(08): 389-393 .
    5. 张书钦,白光耀,李红,张敏智. 多源数据融合的物联网安全知识推理方法. 计算机研究与发展. 2022(12): 2735-2749 . 本站查看
    6. 陈国瑞,袁旭华. 基于HDFS开源架构的异常数据实时检测算法. 计算机仿真. 2021(08): 445-449 .
    7. 谢胜平. 石灰粉一体化加工设备状态检测与故障维修系统. 机械设计与制造工程. 2021(09): 44-48 .

    Other cited types(5)

Catalog

    Article views (1401) PDF downloads (889) Cited by(12)

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return