A Numerical Code for a Wide Range of Compressible Flows on Hybrid Computational Architectures

Anton A. Shershnev; Alexey N. Kudryavtsev; Alexander V. Kashkovsky; Georgy V. Shoev; Semyon P. Borisov; Timofey Yu. Shkredov; Danila P. Polevshchikov; Alexey A. Korolev; Dmitry V. Khotyanovsky; Yulia V. Kratova

doi:10.14529/jsfi220408

Authors

Anton A. Shershnev Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Alexey N. Kudryavtsev Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Alexander V. Kashkovsky Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Georgy V. Shoev Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Semyon P. Borisov Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Timofey Yu. Shkredov Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Danila P. Polevshchikov Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Alexey A. Korolev Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Dmitry V. Khotyanovsky Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation
Yulia V. Kratova Khristianovich Institute of Theoretical and Applied Mechanics, Siberian Branch of Russian Academy of Sciences, Novosibirsk, Russian Federation

DOI:

https://doi.org/10.14529/jsfi220408

Keywords:

Navier–Stokes equations, numerical simulation, compressible flows, DNS, thermochemical non-equilibrium, GPGPU, CUDA

Abstract

The major points in the development of the parallel multiplatform multipurpose numerical code solving the full unsteady Navier–Stokes equations are presented. The developed code is primarily designed for running on multi-GPU computational devices but can also be used on traditional multicore CPUs and even on manycore processors such as Intel Xeon Phi. Physical models include calorically perfect inert gas, single- and multi-temperature approaches for chemically reactive flows and an Euler–Euler model for gas-particle suspensions. Main details of the implementation are described. Shock capturing TVD and WENO schemes in general curvilinear coordinates are used for spatial approximation. Explicit, semi-implicit and fully implicit schemes are employed for advancing solution in time. The code is written in C++ with CUDA API and opensource libraries, such as MPI, zlib and VTK. A few examples of numerical simulations are briefly described to provide general idea of the numerical code capabilities. They include a supersonic flow past a wedge, a jet exhausting from a square nozzle, a heavy gas bubble descending in a lighter medium and a heterogeneous detonation in gas-particle suspension.

References

Surzhikov, S.T.: Radiation aerothermodynamics of the Stardust space vehicle. Journal of Applied Mathematics and Mechanics 80(1), 44–56 (2016). https://doi.org/10.1016/j.jappmathmech.2016.05.008

Abalakin, I.V., Bakhvalov, P.A., Gorobets, A.V., et al.: Parallel software package NOISETTE for large-scale computations in fluid dynamics and aeroacoustics. Num. Meth. Prog. 13(3), 110–125 (2012).

Neyland, V.Y., Bosnyakov, S.M., Glazkov, S.A., et al.: Conception of electronic wind tunnel and first results of its implementation. Progress in Aerospace Sciences 37(2), 121–145 (2001). https://doi.org/10.1016/S0376-0421(00)00013-0

Shur, M., Strelets, M., Travin, A.: High-order implicit multi-block Navier–Stokes code: Ten years experience of application to RANS/DES/LES/DNS of turbulent flows. https://cfd.spbstu.ru/agarbaruk/doc/NTS_code.pdf

Petrov, M.N., Tambova, A.A., Titarev, V.A., et al.: FlowModellium Software Package for Calculating High-Speed Flows of Compressible Fluid. Comput. Math. and Math. Phys. 58, 1865–1886 (2018). https://doi.org/10.1134/S0965542518110118

Isaev, S.A., Baranov, P.A., Usachov, A.E.: Multiblock computational technologies in the VP2/3 Package on aerothermodynamics. LAP LAMBERT Academic Publishing, Saarbrucken (2013).

Lebedev, A.B., Lyubimov, D.A., Maslov, V.P., et al.: The prediction of three-dimensional jet flows for noise applications. AIAA Paper no. 2002-2422 (2002). https://doi.org/10.2514/6.2002-2422

Smirnov, E.M., Zajtsev, D.K.: The finite-volume method in application to complexgeometry fluid dynamics and heat transfer problems. Scientific-Technical Bulletin of the St.-Petersburg State Technical University 2(36), 70–81 (2004). (in Russian)

Kozelkov, A.S., et al.: Multifunctional LOGOS software package for computing fluid dynamics and heat and mass transfer using multiprocessor computers: basic technologies and algorithms. Supercomputing and mathematical modeling: Proceedings of the XII International Workshop, Russia, Sarov, pp. 215–230 (2010). (in Russian)

Aksenov, A.A.: Flowvision: Industrial computational fluid dynamics. Computer Research and Modeling 9(1), 5–20 (2017). (in Russian) https://doi.org/10.20537/2076-7633-2017-9-5-20

Yao, Y., Yeo, K.-S.: An application of GPU acceleration in CFD simulation for insect flight. Supercomputing Frontiers and Innovations 4(2), 13–26 (2017). https://doi.org/10.14529/jsfi170202

Chaplygin, A.V., Gusev, A.V., Diansky, N.A.: High-performance shallow water model for use on massively parallel and heterogeneous computing systems. Supercomputing Frontiers and Innovations 8(4), 74–93 (2021). https://doi.org/10.14529/jsfi210407

Gorobets, A.V., Duben, A.P.: Technology for supercomputer simulation of turbulent flows in the good new days of exascale computing. Supercomputing Frontiers and Innovations 8(4), 4–10 (2022). https://doi.org/10.14529/jsfi210401

Shershnev, A.A., Kudryavtsev, A.N., Kashkovsky, A.V., Khotyanovsky, D.V.: HyCFS, a high-resolution shock capturing code for numerical simulation on hybrid computational clusters. AIP Conf. Proc. 1770, 030076 (2016). https://doi.org/10.1063/1.4964018

Kudryavtsev, A.N., Kashkovsky, A.V., Borisov, S.P., Shershnev, A.A.: A numerical code for the simulation of non-equilibrium chemically reacting flows on hybrid CPU-GPU clusters. AIP Conf. Proc. 1893, 030054 (2017). https://doi.org/10.1063/1.5007512

Borisov, S.P., Kudryavtsev, A.N., Shershnev, A.A.: Development and validation of the hybrid code for numerical simulation of detonations. J. Phys.: Conf. Ser. 1105, 012037 (2018). https://doi.org/10.1088/1742-6596/1105/1/012037

Borisov, S.P., Kudryavtsev, A.N., Shershnev, A.A.: Influence of detailed mechanisms of chemical kinetics on propagation and stability of detonation wave in H2/O2 mixture. J. Phys.: Conf. Ser. 1382, 012052 (2019). https://doi.org/10.1088/1742-6596/1382/1/012052

Alexander Burcat’s Ideal Gas Thermodynamic Data in Polynomial form for Combustion and Air Pollution Use. https://garfield.chem.elte.hu/Burcat/burcat.html

McBride, B.J., Zehe, M.J., Gordon, S.: NASA Glenn coefficients for calculating thermodynamic properties of individual species. NASA/TP-2002-211556 (2002).

Gupta, R.N.: Viscous shock-layer study of thermochemical nonequilibrium. Journal of Thermophysics and Heat Transfer 10(2), 257–266 (1996). https://doi.org/10.2514/3.801

Landau, L., Teller, E.: Theory of sound dispersion. Phys. Z. Sowjetunion 10(1), 34–43 (1936).

Kustova, E., Oblapenko, G.: Reaction and internal energy relaxation rates in viscous thermochemically non-equilibrium gas flows. Phys. Fluids 27(1), 016102 (2015). https://doi.org/10.1063/1.4906317

Fedorov, A.V., Khmel, T.A. Fomin, V.M.: Non-equilibrium model of steady detonations in aluminum particles-oxygen suspensions. Shock Waves 9, 313–318 (1999). https://doi.org/10.1007/s001930050191

Fedorov, A.V., Khmel, T.A.: Numerical simulation of formation of cellular heterogeneous detonation of aluminum particles in oxygen. Combust. Expl. Shock Waves 41(4), 435–448 (2005). https://doi.org/10.1007/s10573-005-0054-7

Fedorov, A.V., Khmel, T.A.: Formation and degeneration of cellular detonation in bidisperse gas suspensions of aluminum particles. Combust. Expl. Shock Waves 44(3), 343–353 (2008). https://doi.org/10.1007/s10573-008-0042-9

Hirschfelder, J.O., Curtiss, C.F., Bird, R.B.: Molecular Theory of Gases and Liquids. Wiley, New York (1954).

Jiang, G.-S., Shu, C.-W.: Efficient implementation of weighted ENO schemes. J. Comput. Phys. 126, 202–228 (1996). https://doi.org/10.1006/jcph.1996.0130

Kolgan, V.P.: Application of the principle of minimal values of the derivative to construction of mesh schemes to calculation of discontinuous solutions of gas dynamics. Uch. Zap. TsAGI 3(6), 68–77 (1972). (in Russian). Translated to English and reprinted in J. Comput. Phys. 230, 2384–2390 (2011). 10.1016/j.jcp.2010.12.033

van Leer, B.: Towards the ultimate conservative difference scheme. V. A second-order sequel to Godunov’s method. J. Comput. Phys. 32(1), 101–136 (1979). https://doi.org/10.1016/0021-9991(79)90145-1

Anderson, W.K., Thomas, J.L., van Leer, B.: Comparison of finite volume flux vector splittings for the Euler equations. AIAA Journal 24(9), 1453–1460 (1986). https://doi.org/10.2514/3.9465

Einfeldt, B., Munz, C.D., Roe, P.L., Sjögren, B.: On Godunov-type methods near low densities. J. Comput. Phys. 92(2), 273–295 (1991). https://doi.org/10.1016/0021-9991(91)90211-3

Toro, E.F., Spruce, M., Speares, W.: Restoration of the contact surface in the Harten–Lax–van Leer Riemann solver. Shock Waves 4, 25–34 (1994). https://doi.org/10.1007/BF01414629

Batten, P., Leschziner, M.A., Goldberg, U.C.: Average-state Jacobians and implicit methods for compressible viscous and turbulent flows. J. Comput. Phys. 137, 38–78 (1997). https://doi.org/10.1006/jcph.1997.5793

Roe, P.L.: Approximate Riemann solvers, parameter vectors, and difference schemes. J. Comput. Phys. 43(2), 357–372 (1981). https://doi.org/10.1016/0021-9991(81)90128-5

van Leer, B.: Flux-vector splitting for the Euler equations. In: Krause, E. (eds) Eighth International Conference on Numerical Methods in Fluid Dynamics. Lecture Notes in Physics, vol. 170, pp. 507–512. Springer, Berlin, Heidelberg (1982). https://doi.org/10.1007/3-540-11948-5_66

Liou, M.-S., Steffen, C.J.: A new flux splitting scheme. J. Comput. Phys. 107(1), 23–39 (1993). https://doi.org/10.1006/jcph.1993.1122.

Edwards, J.R., Liou, M.-S.: Low-diffusion flux-splitting methods for flows at all speeds. AIAA J. 36, 1610–1617 (1998). https://doi.org/10.2514/2.587

Kim, K.H., Kim, C., Rho, O.-H.: Methods for the accurate computations of hypersonic flows – I. AUSMPW+ scheme. J. Comput. Phys. 174(1), 38–80 (2001). https://doi.org/10.1006/jcph.2001.6873

Liou, M.-S.: A sequel to AUSM, Part II: AUSM+-up for all speeds. J. Comput. Phys. 214(1), 137–170 (2006). https://doi.org/10.1016/j.jcp.2005.09.020

Collins, J.P., Ferguson, R.E., Chien, K., et al.: Simulation of shock-induced dusty gas flows using various models. AIAA paper 94-2309.

Shu, C.-W., Osher, S.: Efficient implementation of essentially non-oscillatory shockcapturing schemes. J. Comput. Phys. 77(2), 439–471 (1988). https://doi.org/10.1016/0021-9991(88)90177-5

Thompson, R.J.: Improving round-off in Runge–Kutta computations with Gill’s method. Commun. ACM 13(12), 739–740 (1970). https://doi.org/10.1145/362814.362823

Zhong, X.: Additive semi-implicit Runge–Kutta methods for computing high-speed nonequilibrium reactive flows. J. Comput. Phys. 128(1), 19–31 (1996). https://doi.org/10.1006/jcph.1996.0193

Wright, M.J., Candler, G.V., McDonald, J.D.: Data-parallel lower-upper relaxation method for reacting flows. AIAA Journal 32(12), 2380–2386 (1994). https://doi.org/10.2514/3.12303

Bondar, Ye.A., Markelov, G.N., Gimelshein, S.F., Ivanov, M.S.: Numerical modeling of near-continuum flow over a wedge with real gas effects. Journal of Thermophysics and Heat Transfer 20(4), 699–709 (2006). https://doi.org/10.2514/1.18758

Shuen, J.-S.: Upwind differencing and LU factorization for chemical non-equilibrium Navier–Stokes equations. J. Comput. Phys. 99(2), 233–250 (1992). https://doi.org/10.1016/0021-9991(92)90205-D

Hornung, H.G., Smith, G.H.: The influence of relaxation on shock detachment. J. Fluid Mech. 93(2), 225–239 (1979). https://doi.org/10.1017/S0022112079001865