Study of the Effectiveness of Parallel Algorithms for Modeling the Dynamics of Collisionless Galactic Systems on GPUs

Sergei S. Khrapov; Alexander V. Khoperskov

doi:10.14529/jsfi240302

Authors

Sergei S. Khrapov Volgograd State University, Volgograd, Russian Federation https://orcid.org/0000-0003-2660-2491
Alexander V. Khoperskov Volgograd State University, Volgograd, Russian Federation https://orcid.org/0000-0003-0149-7947

DOI:

https://doi.org/10.14529/jsfi240302

Keywords:

N-body, GPUs, OpenMP-CUDA, GPUDirect, efficiency

Abstract

N-body model is a common research tool in galaxy physics and cosmology. The transition to the use of computing systems with GPUs can significantly improve the performance and quality of simulation results for gravitational systems. N-body – Particle-Particle algorithm is presented on a hybrid computing platform CPU + multi-GPUs. Using a direct method of calculating gravitational forces by summing the interactions of each particle with each other is resource-intensive, but provides the best accuracy in modeling dynamics at all scales. The main result is an analysis of the efficiency of parallel code depending on the number of GPUs and the choice of single and double precision floating-point arithmetics. The laws of conservation of energy, momentum and angular momentum are tested for a series of models, including major mergers of galaxies and the evolution of galactic stellar disc subject to the most severe gravitational instability. The general conclusion is that conservation laws are poorly implemented when using 4-byte numbers due to the accumulation of arithmetic errors. Calculations with 8-byte numbers ensure that the laws of conservation of momentum and angular momentum are satisfied to the limit of arithmetic accuracy without accumulating errors. The law of conservation of energy is determined primarily by the order of the numerical scheme for integrating the equations of motion. The additional reduction in the error of the conservation law of total energy due to the transition from 4-byte to 8-byte numbers is 1–2 orders of magnitude. Increasing the number of GPUs used helps improve the implementation of conservation laws due to a decrease in the number of particles per graphics processing unit.

References

Aarseth, S.J.: Gravitational N-Body Simulations: Tools and Algorithms. Cambridge University Press, Cambridge (2009)

Abraham, M.J., Murtola, T., Schulz, R., et al.: Gromacs: High performance molecular simulations through multi-level parallelism from laptops to supercomputers. SoftwareX 1-2, 19–25 (2015). https://doi.org/10.1016/j.softx.2015.06.001

Appleton, P.N., Emonts, B., Lisenfeld, U., et al.: The CO Emission in the Taffy Galaxies (UGC 12914/15) at 60 pc Resolution. I. The Battle for Star Formation in the Turbulent Taffy Bridge. Astrophysical Journal 931(2), 121 (2022). https://doi.org/10.3847/1538-4357/ac63b2

Belleman, R.G., Be dorf, J., Zwart, S.F.P.: High performance direct gravitational N-body simulations on graphics processing units II: An implementation in CUDA. New Astronomy 13, 103–112 (2008). https://doi.org/10.1016/j.newast.2007.07.004

Binney, J., Tremaine, S.: Galactic Dynamics. Princeton University Press, Princeton (2008)

Brasser, R., Grimm, S.L., Hatalova, P., Stadel, J.G.: Speeding up the GENGA N-body integrator on consumer-grade graphics cards. Astronomy & Astrophysics 678, A73 (2023). https://doi.org/10.1051/0004-6361/202347071

Bruno, D., Capitelli, M., Longo, S., et al.: Particle kinetic modelling of rarefied gases and plasmas. Plasma Sources Science and Technology 12(4), S89 (2003). https://doi.org/10.1088/0963-0252/12/4/024

Eckmann, J.P., Hassani, F.: The detection of relativistic corrections in cosmological N-body simulations. Celestial Mechanics and Dynamical Astronomy 132(2) (2020). https://doi.org/10.1007/s10569-019-9943-z

Fedorov, V.A., Kholina, E.G., Gudimchuk, N.B., Kovalenko, I.B.: High-performance computing of microtubule protofilament dynamics by means of all-atom molecular modeling. Supercomputing Frontiers and Innovations 10(4), 62–68 (2023). https://doi.org/10.14529/jsfi230406

Greengard, L.: The Numerical Solution of the N-Body Problem. Computers in Physics 4, 142–152 (1990). https://doi.org/10.1063/1.4822898

Grigoriev, F.V., Sulimov, V.B., Tikhonravov, A.V.: Study of thin optical films properties using high-performance atomistic simulation. Supercomputing Frontiers and Innovations 11(1), 97–108 (2024). https://doi.org/10.14529/jsfi240105

Hopkins, P.F., Nadler, E.O., Grudic, M.Y., et al.: Novel conservative methods for adaptive force softening in collisionless and multispecies N-body simulations. Monthly Notices of the Royal Astronomical Society 525(4), 5951–5977 (2023). https://doi.org/10.1093/mnras/stad2548

Ishchenko, M., Kovaleva, D.A., Berczik, P., et al.: Star-by-star dynamical evolution of the physical pair of the Collinder 135 and UBC 7 open clusters. Astronomy & Astrophysics 686, A225 (2024). https://doi.org/10.1051/0004-6361/202348978

Kamlah, A.W.H., Leveque, A., Spurzem, R., et al.: Preparing the next gravitational million-body simulations: evolution of single and binary stars in NBODY6++GPU, MOCCA, and MCLUSTER. Monthly Notices of the Royal Astronomical Society 511(3), 4060–4089. https://doi.org/10.1093/mnras/stab3748

Khan, R., Kandappan, V.A., Ambikasaran, S.: HODLRdD: A new black-box fast algorithm for N-body problems in d-dimensions with guaranteed error bounds: Applications to integral equations and support vector machines. Journal of Computational Physics 501, 112786 (2024). https://doi.org/10.1016/j.jcp.2024.112786

Khoperskov, A.V., Khrapov, S.S., Sirotin, D.S.: Formation of transitional cE/UCD galaxies through massive disc to dwarf galaxy mergers. Galaxies 12(1), 1 (2024). https://doi.org/10.3390/galaxies12010001

Khrapov, S.S., Khoperskov, A.V.: Retrograde infall of the intergalactic gas onto S-galaxy and activity of galactic nuclei. Open Astronomy 33(1), 20220231 (2024). https://doi.org/10.1515/astro-2022-0231

Khrapov, S.S., Khoperskov, A.V., Zaitseva, N.A., et al.: Formation of spiral dwarf galaxies: observational data and results of numerical simulation. St. Petersburg State Polytechnical University Journal. Physics and Mathematics 16(1.2), 395–402 (2023). https://doi.org/10.18721/JPM.161.260

Li, Y., Pinto, M.C., Holderied, F., et al.: Geometric Particle-In-Cell discretizations of a plasma hybrid model with kinetic ions and mass-less fluid electrons. Journal of Computational Physics 498, 112671 (2024). https://doi.org/10.1016/j.jcp.2023.112671

Liseykina, T.V., Dudnikova, G.I., Vshivkov, V.A., et al.: MHD-PIC Supercomputer Simulation of Plasma Injection into Open Magnetic Trap. Supercomputing Frontiers and Innovations 10(3), 11–17 (2024). https://doi.org/10.14529/jsfi230302

Navarro, C.A., Hitschfeld-Kahler, N., Mateu, L.: A Survey on Parallel Computing and its Applications in Data-Parallel Problems Using GPU Architectures. Communications in Computational Physics 15(2), 285–329 (2014). https://doi.org/10.4208/cicp.110113.010813a

Ong, B.W., Dhamankar, S.: Towards an Adaptive Treecode for N-body Problems 82, 72 (2020). https://doi.org/10.1007/s10915-020-01177-1

Rantala, A., Naab, T., Rizzuto, F.P., et al.: Bifrost: simulating compact subsystems in star clusters using a hierarchical fourth-order forward symplectic integrator code. Monthly Notices of the Royal Astronomical Society 522(4), 5180–5203 (2023). https://doi.org/10.1093/mnras/stad1360

Romeo, A.B., Horellou, C., Bergh, J.: N-body simulations with two-orders-of-magnitude higher performance using wavelets. Monthly Notice of the Royal Astronomical Society 342(2), 337–344 (2003). https://doi.org/10.1046/j.1365-8711.2003.06549.x

Smirnov, A.A., Sotnikova, N.Y., Koshkin, A.A.: Simulations of slow bars in anisotropic disk systems. Astronomy Letters 43(2), 61–74 (2017). https://doi.org/10.1134/S1063773717020062

Voevodin, V.V., Chulkevich, R.A., Kostenetskiy, P.S., et al.: Administration, Monitoring and Analysis of Supercomputers in Russia: a Survey of 10 HPC Centers. Supercomputing Frontiers and Innovations 8(3), 82–103 (2021). https://doi.org/10.14529/jsfi210305

Yokota, R., Barba, L.A.: Treecode and Fast Multipole Method for N-Body Simulation with CUDA. Springer (2011). https://doi.org/10.1016/B978-0-12-384988-5.00009-7

Zhang, H., Si, S., Hsieh, C.J.: GPU-acceleration for large-scale tree boosting. Eprint arXiv 1706.08359 (2017). https://doi.org/10.48550/arXiv.1706.08359

Zhou, K., Liu, B.: Molecular Dynamics Simulation: Fundamentals and Applications. Elsevier, Amsterdam (2022)