• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Zheng Xiaowei, Xiang Ming, Zhang Dawei, and Liu Qingkun. An Adaptive Tasks Scheduling Method Based on the Ability of Node in Hadoop Cluster[J]. Journal of Computer Research and Development, 2014, 51(3): 618-626.
Citation: Zheng Xiaowei, Xiang Ming, Zhang Dawei, and Liu Qingkun. An Adaptive Tasks Scheduling Method Based on the Ability of Node in Hadoop Cluster[J]. Journal of Computer Research and Development, 2014, 51(3): 618-626.

An Adaptive Tasks Scheduling Method Based on the Ability of Node in Hadoop Cluster

More Information
  • Published Date: March 14, 2014
  • Scheduling algorithm has played an important role in improving Hadoop cluster performance. However, the phenomenon of uneven load distribution exists in current Hadoop inherent task-level scheduling methods. In order to maintain the load status of each node in the cluster, we focus on the analysis and research about the implementation capacity of cluster nodes. An adaptive tasks scheduling method based on the node capability is proposed in this paper. According to the node history and the current load status, the method takes node performance, task characteristics and node failure rate as the parameters to calculate node executive ability. On this basis, the different amount of tasks are assigned to cluster nodes. Thus, joined nodes adaptively adjust the amount of running tasks to make the node suited for different tasks better. Finally experimental results indicate that the proposed adaptive tasks scheduling method can make the total task completion time being reduced significantly, and moreover, the load on each node gets more balanced and the node resource utilization is more reasonable.

Catalog

    Article views (961) PDF downloads (674) Cited by()
    Turn off MathJax
    Article Contents

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return