ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2017, Vol. 54 ›› Issue (4): 844-854.doi: 10.7544/issn1000-1239.2017.20150923

• 软件技术 • 上一篇    下一篇

面向科学计算可视化的两级并行数据读取加速方法

石刘1,2,3,肖丽2,3,曹立强2,3,莫则尧2,3   

  1. 1(中国舰船研究院 北京 100101); 2(北京应用物理与计算数学研究所 北京 100088); 3(中物院高性能数值模拟软件中心 北京 100088) (shilbill@vip.sina.com)
  • 出版日期: 2017-04-01
  • 基金资助: 
    国家自然科学基金重点项目(61232012);国家重点基础研究专项经费(2011CB309702);国家“八六三”高技术研究发展计划基金项目(2012AA01A309)

Two Level Parallel Data Read Acceleration Method for Visualization in Scientific Computing

Shi Liu1,2,3, Xiao Li2,3, Cao Liqiang2,3, Mo Zeyao2,3   

  1. 1(China Ship Research and Development Academy, Beijing 100101); 2(Institute of Applied Physics and Computational Mathematics, Beijing 100088); 3(CAEP Software Center for High Performance Numerical Simulation, Beijing 100088)
  • Online: 2017-04-01

摘要: 为了匹配超级计算机的整体计算能力,超级计算机存储子系统通常具有良好的I/O性能可扩展性,表现为:应用获得存储子系统最佳性能时的I/O访问并发度,与超级计算机系统总计算核数(可达数万至数百万)通常处于同一数量级.然而,科学计算可视化应用通常使用的进程数(等于I/O访问并发度)相对较小(经验上常设为计算进程数的1%,典型值为数个至数百个),因此无法充分发挥超级计算机存储子系统的最佳I/O性能.提出了一种面向科学计算可视化的两级并行数据读取加速方法,在可视化进程内部引入多线程并行数据读取,通过进程间和进程内两级并行,增加超级计算机存储子系统的I/O访问并发度,提升可视化应用数据读取速率.测试结果表明:在不同的可视化进程规模下,两级并行比单级并行峰值数据读取速率提高33.5%~269.5%,均值数据读取速率提高26.6%~232.2%;随着科学计算应用种类以及应用规模的变化,两级并行数据读取可使可视化应用整体峰值运行速度加速19.5%~225.7%,均值运行速度加速15.8%~197.6%.

关键词: 两级并行数据读取, 科学计算可视化, 数据访问模式, 存储子系统, I/O性能特征

Abstract: In order to match the overall computing capability of super computer, the super computer’s storage subsystem usually has good I/O performance scalability, which causes that, applications’ I/O access concurrency under the best performance of the storage subsystem and the total compute core number (tens of thousands to several millions) of super computer are usually in the same order of magnitude; however, the process number (equals to the I/O access concurrency) used in visualization in scientific computing (ViSC) applications is usually relatively small (experientially set to 1% of used computing process number, typically several to hundreds). Therefore, the best I/O performance of the storage subsystem cannot be achieved. In this paper we propose a two level parallel data read-based acceleration method for ViSC applications. Multi threads parallel data accessing is introduced into the visualization process; the I/O access concurrency of the super computer’s storage subsystem has been enhanced and visualization applications’ data read rate has been promoted through the two level parallel read, i.e. the parallelism among multi processes and the parallelism among multi threads inner process. The test results show that, under various visualization process scales, the peak data read rate using two parallel mode is higher than that using single parallel mode by 33.5%-269.5%, while the mean data read rate using two parallel mode is higher than that using single parallel mode by 26.6%-232.2%; according to the diverse scientific computing applications and various process scales, based on two level parallel data read method, the overall peak running speed of visualization applications can be accelerated by 19.5%-225.7%, and the mean speed can be accelerated by 15.8%-197.6%.

Key words: two level parallel data read, visualization in scientific computing, data access pattern, storage subsystem, I/O performance characteristics

中图分类号: