ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2019, Vol. 56 ›› Issue (4): 790-797.doi: 10.7544/issn1000-1239.2019.20170906

Previous Articles     Next Articles

An Overlap Store Optimization for Large-Scale Parallel Earth Science Application

Chen Jingkun Du Yunfei   

  1. (National Supercomputer Center in Guangzhou (Sun Yat-sen University), Guangzhou 510006)
  • Online:2019-04-01

Abstract: Weather forecast, atmosphere or ocean simulations have much output data during the iterative computation for the intermediate status or check point. However, an unreasonable output design limits the performance of the earth science application in large-scale parallel computation. In this paper, we propose an overlap store optimization to solve this problem. The key issue of this overlap store optimization is setting some I/O processes to hide the I/O cost. This optimization has three main advantages: first, we hide the I/O operation through the overlap of output and computing; second, we limit the cost of gather operation, break though the bottleneck of gather communication bandwidth and memory size; third, the I/O process is flexible to use different high-performance parallel I/O API. We use this method to optimize WRF, ROMS_AGRIF and GRAPES in Tianhe II super computer, and test their performance after the optimization. The result of the tests shows that we obtain about 30% to 900% improvement in the peak. We also discuss the best proportion of computer process and I/O process when the total number of processes is fixed. The optimized version is very easy to used, and the only cost is the scientists need to setup two more variables in the namelist.

Key words: overlap store, multiply I/O processes, processes distribution, earth science simulation, large-scale parallel application

CLC Number: