Heterogeneous Programming and Optimization of Gyrokinetic Simulation Code on Tianhe Supercomputer

Li Qingfeng; Li Yueyan; Luan Zhongzhi; Zhang Wenlu; Gong Chunye; Zheng Gang; Kang Bo; Meng Xiangfei

doi:10.7544/issn1000-1239.202330872

Journal of Computer Research and Development > 2025 > Accepted Manuscript > DOI: 10.7544/issn1000-1239.202330872 CSTR: 32373.14.issn1000-1239.202330872

Li Qingfeng, Li Yueyan, Luan Zhongzhi, Zhang Wenlu, Gong Chunye, Zheng Gang, Kang Bo, Meng Xiangfei. Heterogeneous Programming and Optimization of Gyrokinetic Simulation Code on Tianhe Supercomputer[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202330872

Citation:

PDF (1034 KB)

Heterogeneous Programming and Optimization of Gyrokinetic Simulation Code on Tianhe Supercomputer

1.
National Supercomputer Center in Tianjin, Tianjin 300457
2.
School of Computer Science and Engineer, Beihang University, Beijing 100191
3.
Institute of Physics, Chinese Academy of Sciences, Beijing 100086
4.
College of Computing, National University of Defense Technology, Changsha 410073

Funds: This work was supported by the National Key Research and Development Program of China (2021YFBO300101) and the National Natural Science Foundation of China (62032023,42104078,61902411).

More Information

Author Bio:
Li Qingfeng: born in 1993. Master, engineer. His main research interests include heterogeneous high performance computing

Li Yueyan: born in 1993. PhD, engineer. His main research interests include heterogeneous high performance computing

Luan Zhongzhi: born in 1971. PhD, associate professor. His main research interests include distributed computing, computing, parallel high computing, performance computer architecture, cloud computing, and big data

Zhang Wenlu: born in 1975. PhD, professor. His main research interests include theory and numerical simulation of plasma physics

Gong Chunye: born in 1982. PhD, associate professor. His main research interests include parallel algorithms and automatic performance optimization

Zheng Gang: born in 1988. Master, senior engineer, His main research interests include high-performance computing, parallel computing, and the development and optimization of large-scale parallel programs

Kang Bo: born in 1986. PhD, senior engineer, His main research interests include intelligent computing and high performance computing applications

Meng Xiangfei: born in 1979. PhD, senior engineer, His main research interests include heterogeneous high performance computing and generative artificial intelligence, responsible for the research and development of the “Tianhe” series of supercomputers
Received Date: October 30, 2023
Revised Date: March 18, 2025
Accepted Date: April 03, 2025
Available Online: April 03, 2025

Graphical Abstract

Abstract

Abstract

The magnetic confinement fusion particle-in-cell (PIC) gyrokinetic simulation code, VirtEx, has been capable of studying the confinement and transport of the fusion product Alpha, which is the key to fusion energy realization. Alpha particle simulation relies heavily on the computational code of the kinetic ion, which has more complex memory access than the electron, and contains both non-regular accesses and atomic write-back operations, belong to memory-intensive application. MT-3000 as a new heterogeneous acceleration device, provided by Tianhe's new-generation supercomputing platform, which have powerful computational performance with its extremely high computational density. Heterogeneous porting of alpha particle simulations for this device is a great challenge, in order to fully exploit the computational power of the acceleration array in MT-3000, we combine application characteristics propose some optimization methods, such as recalculation of intermediate variables, customized software cache design, memory locality optimization, and hotspot function merging, are designed and implemented to reduce the total amount of memory accesses in the program. The medium scale benchmark with gyrokinetic ion shows an overall speedup of 4.2 times, with 10.9, 13.3 and 16.2 times of speedup on hotspot functions Push, Locate and Charge, respectively, meanwhile it shows a good scaling of scalability with 88.4% efficiency with 5 898 240 accelerator cores in 3 840 nodes.
- particle-in-cell,
- gyrokinetic simulation,
- memory intensive,
- high arithmetic intensity system,
- heterogeneous programming

FullText(HTML)

References (16)

References

[1]	Sheffield J. The physics of magnetic fusion reactors[J]. Review of Modern Physics, 1994, 66(3): 1015−1103
[2]	Lawson J D. Some criteria for a power producing thermonuclear reactor[J]. Proceedings of the Physical Society: Section B, 1957, 70(1): 6−10
[3]	Catto P J, Tang W M, Baldwin D E. Generalized gyrokinetics[J]. Plasma Physics, 1981, 23(7): 639
[4]	Frieman E A , Chen Liu . Nonlinear gyrokinetic equations for low-frequency electromagnetic waves in general plasma equilibria[J]. The Physics of Fluids, 1982, 25(3): 502-508
[5]	Brizard A J, Hahm T S. Foundations of nonlinear gyrokinetic theory[J]. Review of Modern Physics, 2007, 79(2): 421−468
[6]	Lee W W, Tang W M. Gyrokinetic particle simulation of ion temperature gradient drift instabilities[J]. The Physics of Fluids, 1988, 31(3): 612−624
[7]	Parker S E , Lee W W . A fully nonlinear characteristic method for gyrokinetic simulation[J]. Physics of Fluids B Plasma Physics, 1993, 5(1): 77-86
[8]	Lin Zhihong, Tang W M, Lee W W. Gyrokinetic particle simulation of neoclassical transport[J]. Physics of Plasmas, 1995, 2(8): 2975−2988
[9]	Meng Xiangfei, Zhu Xiaoqian, Wang Peng, et al. Heterogeneous programming and optimization of gyrokinetic toroidal code and large-scale performance test on th−1a [C] //Proc of the ISC 2013. Berlin: Springer, 2013: 81−96
[10]	Zhang Wenlu, Joubert W , Wang Peng , et al. Heterogeneous programming and optimization of gyrokinetic toroidal code using directives[C]//Proc of the 5th Int Workshop on Accelerator Programming Using Directives. Cham: Springer, 2019: 3−21
[11]	Noé O, Gheller C, Lanti E, et al. Gyrokinetic simulations on many- and multi-core architectures with the global electromagnetic particle-in-cell code ORB5[J]. Computer Physics Communications, 2021, 262: 107208
[12]	Wang Endong, Wu Shaohua, Zhang Qing , et al. The gyrokinetic particle simulation of fusion plasmas on TIANHE−2 supercomputer[C]// Proc of the 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems. Piscataway, NJ: IEEE, 2016: 25−32
[13]	Dannert T, Marek A, Rampp M. Porting large hpc applications to gpu clusters: The codes gene and vertex[J]. Advances in Parallel Computing, 2013, 25: 305−314
[14]	Lu Kai, Wang Yaohua, Guo Yang. et al. MT−3000: A heterogeneous multi-zone processor for HPC[J]. CCF Transactions on High Performance Computing. 2022, 4(2): 150–164
[15]	Madduri K, Ibrahim K Z, Williams S, et al. Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems[C/OL]// Proc of the 11th SC. Piscataway, NJ: IEEE, 2011[2024-02-25]. https://xplorestaging.ieee.org/document/6114454/metrics#citations
[16]	Feng Hongying, Zhang Wenlu, Dong Chao, et al. Verification of linear resistive tearing instability with gyrokinetic particle code virtex[J]. Physics of Plasmas, 2017, 24(10): 1−9