Adapting a Scientific CFD Code to Industrial Applications on Hybrid Supercomputers




heterogeneous code, computational fluid dynamics, turbulent flows, scale-resolving simulation, CPU GPU, MPI OpenMP OpenCL


The NOISEtte heterogeneous parallel code for simulating turbulent flow and aerodynamic noise is considered. In our previous works, high acceleration and parallel efficiency in scientific scale-resolving simulations using GPUs were reported. For parallelization, the MPI, OpenMP and OpenCL standards are used, the latter allows using GPUs from different vendors. However, the further transition to industrial-oriented applications brought more trouble. Instead of discussing the parallel algorithm, this work will focus on the problems that are not so obvious at first glance, which arise when developing a heterogeneous simulation code. How to deal with numerous simulation algorithm components, all those bells and whistles like wall functions, mixing plane and sliding interfaces, synthetic turbulence generators, a variety of boundary conditions, etc., that either need to be ported to the GPU side or incorporated directly from the CPU side? How to maintain and modify the OpenCL code in a growing number of source files? How to arrange the modularity of a complex heterogeneous software package? How to preserve reliability and fault tolerance, especially in the case of numerical schemes of increased accuracy, but reduced social responsibility? These issues are discussed here and some solutions will be proposed.


Alvarez, X., Gorobets, A., Trias, F., et al.: HPC2 – A fully-portable, algebra-based framework for heterogeneous computing. Application to CFD. Computers & Fluids 173, 285–292 (2018),

Bocharov, A., Evstigneev, N., Petrovskiy, V., et al.: Implicit method for the solution of supersonic and hypersonic 3D flow problems with Lower-Upper Symmetric-Gauss-Seidel preconditioner on multiple graphics processing units. Journal of Computational Physics 406, 109189 (2020),

Borrell, R., Dosimont, D., Garcia-Gasulla, M., et al.: Heterogeneous CPU/GPU co-execution of CFD simulations on the POWER9 architecture: Application to airplane aerodynamics. Future Generation Computer Systems 107, 31–48 (2020), future.2020.01.045

Gorobets, A., Bakhvalov, P.: Heterogeneous CPU+GPU parallelization for high-accuracy scale-resolving simulations of compressible turbulent flows on hybrid supercomputers. Computer Physics Communications 271, 108231 (2022),

Gorobets, A., Duben, A.: Technology for Supercomputer Simulation of Turbulent Flows in the Good New Days of Exascale Computing. Supercomputing Frontiers and Innovation 8(4), 4–10 (2021),

Niedermeier, C., Janssen, C., Indinger, T.: Massively-parallel multi-GPU simulations for fast and accurate automotive aerodynamics. In: Proceedings of the 7th European Conference on Computational Fluid Dynamics, Glasgow, Scotland, UK, June 11–15, 2018 (06 2018)

Voevodin, V., Antonov, A., Nikitenko, D., et al.: Supercomputer Lomonosov-2: Large Scale, Deep Monitoring and Fine Analytics for the User Community. Supercomput. Front. Innov. 6(2), 4–11 (2019),

Watanabe, S., Aoki, T.: Large-scale flow simulations using lattice Boltzmann method with AMR following free-surface on multiple GPUs. Computer Physics Communications 264, 107871 (2021),




How to Cite

Gorobets, A. V. (2022). Adapting a Scientific CFD Code to Industrial Applications on Hybrid Supercomputers. Supercomputing Frontiers and Innovations, 9(4), 49–54.