Correlation Statistical Modeling Reduction Method for Large-Scale Structural Grid Data

Data visual analysis is essential for large-scale numerical simulations. The storage bottleneck of high-performance computers makes it challenging to analyze and visualize data with original high-resolution. The method based on statistical modeling can significantly reduce the data storage cost, with the reconstruction uncertainty being high. Therefore, we propose a large-scale data reduction method for efficient analysis and visualizing large-scale multi-block volume data generated by massively parallel scientific simulations. The technical core of this method is to guide the statistical modeling of adjacent data blocks through the statistical representation of correlation between data blocks. By doing so, our method efficiently preserves the statistical data properties without merging data blocks stored in different parallel computing nodes and repartitioning them according to the homogeneity requirements of the visualization. Compared with exsiting methods, the original data can be reconstructed more accurately by coupling numerical distribution information, spatial distribution information, and correlation information, further reducing the visual uncertainty. The experimental tests use five sets of scientific data with the largest scale of one billion grids. The quantitative analysis results show that our method improves the data reconstruction accuracy by up to two orders of magnitude at the same data compression ratio compared with the current state-of-the-art methods.