Parallel Load-Balancing Performance Analysis Based on Maximal Ratio of Load Offset

It is well known that the measurement of load-balancing and its effect on the whole performance of massively parallel applications is a tough job. In this work, an evaluation model of load-balancing is put forward for numerical parallel computing based on local discretization schemes (finite difference, finite element, etc.). By introducing the maximal ratio of load offset (MRLO) as the index of load-balance, the quantitative relationships between the parallel efficiency and the ratio of computing to communication, parallel scale, problem size, and complexity of numerical schemes are analysed. MRLO is a basal parameter of the evaluation model. A famous global ocean circulation model named POP benchmark is used for verification of the evaluation model. Since the ratio of walltime cutdown is equal to that of efficiency improvement, the parallel efficiency increase of POP can be predicted from both walltime and the evaluation model. The evaluation model reveals that to what extent the whole performance depends upon load-balancing. Especially for large-scale parallel computing, load-balancing is more and more sensitive to parallel efficiency with the increase of problem size and parallel scale. The result shows that load-balancing is an important bottleneck of parallel computing up to tens of thousands tasks, and researchers of parallel optimization for all kinds of applications should put more emphases on load-balancing.