Direct Numerical Simulation of Stratified Turbulent Flows and Passive Tracer Transport on HPC Systems: Comparison of CPU Architectures


  • Evgeny V. Mortikov Lomonosov Moscow State University
  • Andrey V. Debolskiy Lomonosov Moscow State University Research Computing Center



turbulence, direct numerical simulation, ARM, supercomputing


In this paper we assess the influence of CPU architectures commonly used in HPC systems on the efficiency of the implementation of algorithms used for direct numerical simulation (DNS) of turbulent flows. We consider a stably stratified turbulent plane Couette flow as a benchmark problem supplemented with the additional transport of passive substances. The comparison includes the Intel Xeon, AMD Rome x86 CPU architecture processors and the Huawei Kunpeng ARM CPU processor. We discuss the role of memory-oriented optimizations on the efficiency of tracer transport implementation on each platform.


Afanasyev, I., Lichmanov, D.: Evaluating the performance of Kunpeng 920 processors on modern HPC applications. In: Parallel Computing Technologies 2021, Proceedings. pp. 301–321. Springer International Publishing (2021).

Ayala, A., Tomov, S., Haidar, A., Dongarra, J.: heFFTe: Highly efficient FFT for exascale. In: Computational Science – ICCS 2020. ICCS 2020. Lecture Notes in Computer Science. pp. 262–275. Springer, Cham (2020).

Besnard, J., Malony, A., Shende, S., et al.: An MPI halo-cell implementation for zero-copy abstraction. In: EuroMPI ’15: Proceedings of the 22nd European MPI Users’ Group Meeting. pp. 1–9. ACM Press (2015).

Brown, D., Cortez, R., Minion, M.: Accurate projection methods for the incompressible Navier-Stokes equations. J. Comp. Phys. 168, 464–499 (2001).

Dongarra, J., Heroux, M., Luszczek, P.: A new metric for ranking high performance computing systems. Nat. Sci. Rev. 3(1), 30–35 (2016).

Gladskikh, D., Stepanenko, V., Mortikov, E.: The effect of the horizontal dimensions of inland water bodies on the thickness of the upper mixed layer. Water Res. 48, 226–234 (2021).

Glazunov, A., Mortikov, E., Barskov, K., et al.: Layered structure of stably stratified turbulent shear flows. Izv., Atmos. Ocean. Phys. 55(4), 312–323 (2019).

Ibeid, H., Olson, L., Gropp, W.: FFT, FMM, and multigrid on the road to exascale: Performance challenges and opportunities. J. Parallel Distrib. Comput. 136, 63–74 (2020).

Kadantsev, E., Mortikov, E., Zilitinkevich, S.: The resistance law for stably stratified atmospheric planetary boundary layers. Q. J. R. Meteorol. Soc. 147(737), 2233–2243 (2021).

Larsson, J., Lien, F., Yee, E.: Conditional semicoarsening multigrid algorithm for the Poisson equation on anisotropic grids. J. Comp. Phys. 208, 368–383 (2005).

LeMone, M., Angevine, W., Bretherton, C., et al.: 100 years of progress in boundary layer meteorology. Meteorological Monographs 59, 9.1–9.85 (2019).

Moin, P., Mahesh, K.: Direct numerical simulation: A tool in turbulence research. Annu. Rev. Fluid Mech. 30, 539–578 (1998).

Monin, A., Yaglom, A.: Statistical fluid mechanics: The mechanics of turbulence. MIT Press, Cambridge (1971)

Morinishi, Y., Lund, T., Vasilyev, O., Moin, P.: Fully conservative higher order finite difference schemes for incompressible flows. J. Comp. Phys. 143, 90–124 (1998).

Mortikov, E.: Numerical simulation of the motion of an ice keel in a stratified flow. Izv., Atmos. Ocean. Phys. 52(1), 108–115 (2016).

Mortikov, E., Glazunov, A., Lykosov, V.: Numerical study of plane Couette flow: turbulence statistics and the structure of pressure-strain correlations. Russ. J. Numer. Analysis Math. Model. 34(2), 119–132 (2019).

Pirozzoli, S., Bernardini, M., Orlandi, P.: Turbulence statistics in Couette flow at high reynolds number. J. Fluid Mech. 758, 327–343 (2014).

Porter, A., Appleyard, J., Ashworth, M., et al.: Portable multi- and many-core performance for finite-difference or finite-element codes – application to the free-surface component of NEMO (NEMOLite2D 1.0). Geosci. Model Dev. 11, 3447–3464 (2018).

Rajovic, N., Rico, A., Puzovic, N., Adeniyi-Jones, C., Ramirez, A.: Tibidabo: Making the case for an ARM-based HPC system. Future Generation Computer Systems 36, 322–334 (2014).

Sofiev, M., Vira, J., Kouznetsov, R., et al.: Construction of the SILAM Eulerian atmospheric dispersion model based on the advection algorithm of Michael Galperin. Geosci. Model Dev. 8, 3497–3522 (2015).

Soustova, I., Troitskaya, Y., Gladskikh, D., et al.: A simple description of the turbulent transport in a stratified shear flow as applied to the description of thermohydrodynamics of inland water bodies. Izv., Atmos. Ocean. Phys. 56, 603–612 (2020).

Thorpe, S.: An introduction to ocean turbulence. Cambridge University Press, Cambridge (2007)

Tkachenko, E., Debolskiy, A., Mortikov, E.: Intercomparison of subgrid scale models in largeeddy simulation of sunset atmospheric boundary layer turbulence: computational aspects. Lobachevskii Journal of Mathematics 42, 1580–1595 (2021).

Trottenberg, U., Oosterlee, C., Schüller, A.: Multigrid. Academic Press, London (2001)

Vasilyev, O.: High order finite difference schemes on non-uniform meshes with good conservation properties. J. Comp. Phys. 157, 746–761 (2000).

Vichi, M., Pinardi, N., Masina, S.: A generalized model of pelagic biogeochemistry for the global ocean ecosystem. Part I: theory. J. Mar. Sys. 64, 89–109 (2007).

Van der Vorst, H.: Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992).

Yli-Juuti, T., Barsanti, K., Hildebrandt Ruiz, L., et al.: Model for acid-base chemistry in nanoparticle growth (MABNAG). Atmos. Chem. Phys. 13(24), 12507–12524 (2013).

Zasko, G., Glazunov, A., Mortikov, E., Nechepurenko, Y.: Large-scale structures in stratified turbulent Couette flow and optimal disturbances. Russ. J. Numer. Analysis Math. Model. 35(1), 37–53 (2020).

Zilitinkevich, S., Druzhinin, O., Glazunov, A., et al.: Dissipation rate of turbulent kinetic energy in stably stratified sheared flows. Atmos. Chem. Phys. 19, 2489–2496 (2019).

Zoric, D., Sandborn, V.: Similarity of large reynolds number boundary layers. Bound. Layer-Meteorol. 2, 326–333 (1972).




How to Cite

Mortikov, E. V., & Debolskiy, A. V. (2022). Direct Numerical Simulation of Stratified Turbulent Flows and Passive Tracer Transport on HPC Systems: Comparison of CPU Architectures. Supercomputing Frontiers and Innovations, 8(4), 50–68.