ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2018, Vol. 55 ›› Issue (6): 1308-1319.doi: 10.7544/issn1000-1239.2018.20170024

Previous Articles     Next Articles

MapReduce Back Propagation Algorithm Based on Structure Parallelism

Ren Gang1,2,3, Deng Pan2, Yang Chao2, Wu Changmao2   

  1. 1(Department of Computer Science and Technology, Henan Institute of Technology, Xinxiang, Henan 453003); 2(Laboratory of Parallel Software and Computational Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190); 3(University of Chinese Academy of Sciences, Beijing 100049)
  • Online:2018-06-01

Abstract: Back propagation (BP) algorithm is a widely used learning algorithm that is used for training multiple layer neural networks. BP algorithm based on Hadoop cluster and MapReduce parallel programming model (MRBP) shows good performance on processing big data problems. However, it lacks the capability of fine-grained parallelism. Thus, when confronted with high dimension data and neural networks with large nodes, the performance is low relatively. On the other hand, since the users can’t control the communication of Hadoop computing nodes, the existing structure parallel scheme based on clusters can’t be directly applied to MRBP algorithm. This paper proposes a structure parallelism based MRBP algorithm (SP-MRBP), which adopts layer-wise parallelism, layer-wise ensemble (LPLE) strategy to implement structure parallel computing. Also, we derive the analytical expressions of the proposed SP-MRBP algorithm and the classic MRBP algorithm, and obtain the time differences between the both algorithms as well as the optimal number of parallel structures of SP-MRBP algorithm. To the best knowledge of the authors, it is the first time to introduce the structure parallelism scheme to the MRBP algorithm. The experimental results show that, compared with the classic MRBP algorithm, our algorithm has better performance on processing efficiency when facing large neural networks.

Key words: MapReduce model, structure parallelism, back propagation (BP) algorithm, multiple layer neural networks, MapReduce back propagation (MRBP) algorithm

CLC Number: