On an Algorithm for Decomposing Multi-Block Structured Meshes for Calculating Dynamic Wave Processes in Complex Structures on Supercomputers with Distributed Memory

Ilia N. Agrelov; Nikolay I. Khokhlov; Vladislav O. Stetsyuk; Sergey D. Agibalov

doi:10.14529/jsfi240405

Authors

Ilia N. Agrelov Moscow Institute of Physics and Technology, Dolgoprudny, Russian Federation https://orcid.org/0009-0001-1594-6988
Nikolay I. Khokhlov Moscow Institute of Physics and Technology, Dolgoprudny, Russian Federation; Scientific Research Institute for System Analysis of the Russian Academy of Sciences, Moscow, Russian Federation https://orcid.org/0000-0002-2460-0137
Vladislav O. Stetsyuk Moscow Institute of Physics and Technology, Dolgoprudny, Russian Federation
Sergey D. Agibalov Moscow Institute of Physics and Technology, Dolgoprudny, Russian Federation

DOI:

https://doi.org/10.14529/jsfi240405

Keywords:

parallel calculation, numerical modeling, wave propagation, multiple grids modeling, parallel algorithm

Abstract

The advancement of the oil and gas industry represents a key priority area for the Russian Federation. The Arctic region contains substantial hydrocarbon reserves, but the inherent difficulties in exploring these resources make them particularly challenging to access. The present paper is devoted to the numerical calculation of the dynamic impact propagation on an oil platform using parallel computing methods. To address this issue, a grid-characteristic method was employed. The substantial volume of computation necessitates the utilization of parallel computing techniques, such as Message Passing Interface (MPI). A grid model was constructed based on a real platform, and an algorithm for decomposing the computational domain was developed with the aim of reducing the message time between MPI processes and increasing speedup. A series of test calculations were performed to demonstrate the capabilities of the algorithms. Examples of calculations and the application of the developed method of decomposition are provided. The feasibility of decomposition and parallelization algorithms is currently being investigated. The conducted tests have demonstrated the potential for using the model for real calculations.

References

Ali, Z., Tucker, P.G., Shahpar, S.: Optimal mesh topology generation for CFD. Computer Methods in Applied Mechanics and Engineering 317, 431–457 (2017). https://doi.org/10.1016/j.cma.2016.12.001

Andrade, X., Alberdi-Rodriguez, J., Strubbe, D.A., et al.: Time-dependent density-functional theory in massively parallel computer architectures: the octopus project. Journal of Physics: Condensed Matter 24(23), 233202 (2012). https://doi.org/10.1088/0953-8984/24/23/233202

Basermann, A., Clinckemaillie, J., Coupez, T., et al.: Dynamic load-balancing of finite element applications with the drama library. Applied Mathematical Modelling 25(2), 83–98 (2000). https://doi.org/10.1016/S0307-904X(00)00043-3

Bulu¸c, A., Meyerhenke, H., Safro, I., et al.: Recent Advances in Graph Partitioning, pp. 117–158. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49487-6_4

Cacace, S., Cristiani, E., Falcone, M., Picarelli, A.: A patchy dynamic programming scheme for a class of Hamilton–Jacobi–Bellman equations. SIAM Journal on Scientific Computing 34(5), A2625–A2649 (2012). https://doi.org/10.1137/110841576

Cacace, S., Falcone, M., et al.: A dynamic domain decomposition for the eikonal-diffusion equation. Discret Contin Dyn S 9(1), 109–123 (2016). https://doi.org/10.3934/dcdss.2016.9.109

C¸ ataly¨urek, ¨ U., Devine, K., Faraj, M., et al.: More recent advances in (hyper) graph partitioning. ACM Computing Surveys 55(12), 1–38 (2023). https://doi.org/10.1145/3571808

Chaplygin, A.V., Gusev, A.V.: Shallow water model using a hybrid MPI/OpenMP parallel programming. Probl Inf 1(50), 65–82 (2021). https://doi.org/10.24411/2073-0667-2021-10006

Devine, K., Boman, E., Heaphy, R., et al.: Zoltan data management services for parallel dynamic applications. Computing in Science & Engineering 4(2), 90–96 (2002). https://doi.org/10.1109/5992.988653

Farrashkhalvat, M., Miles, J.P.: Basic Structured Grid Generation: With an introduction to unstructured grid generation. Elsevier (2003). https://doi.org/10.1016/B978-0-7506-5058-8.X5000-X

Favorskaya, A., Khokhlov, N., Sagan, V., Podlesnykh, D.: Parallel computations by the grid-characteristic method on Chimera computational grids in 3D problems of railway non-destructive testing. In: Russian Supercomputing Days, pp. 199–213. Springer (2022). https://doi.org/10.1007/978-3-031-22941-1_14

Favorskaya, A.V., Zhdanov, M.S., Khokhlov, N.I., Petrov, I.B.: Modelling the wave phenomena in acoustic and elastic media with sharp variations of physical properties using the grid-characteristic method. Geophysical Prospecting 66(8), 1485–1502 (2018). https://doi.org/10.1111/1365-2478.12639

Fofanov, V., Khokhlov, N.: Optimization of load balancing algorithms in parallel modeling of objects using a large number of grids. In: Supercomputing: 6th Russian Supercomputing Days, RuSCDays 2020, Moscow, Russia, September 21–22, 2020, Revised Selected Papers 6. Communications in Computer and Information Science, vol. 1331, pp. 63–73. Springer (2020). https://doi.org/10.1007/978-3-030-64616-5_6

Golovchenko, E.N., Yakobovskiy, M.V.: Parallel partitioning tool GridSpiderPar for large mesh decomposition. Numerical Methods and Programming 16(4), 507–517 (2015). https://doi.org/10.26089/NumMet.v16r448

Ivanov, A.M., Khokhlov, N.I.: Efficient inter-process communication in parallel implementation of grid-characteristic method. In: Smart Modeling for Engineering Systems: Proceedings of the Conference 50 Years of the Development of Grid-Characteristic Method. pp. 91–102. Springer (2019). https://doi.org/10.1007/978-3-030-06228-6_9

Ivanov, A.M., Khokhlov, N.I., et al.: Parallel implementation of the grid-characteristic method in the case of explicit contact boundaries. Computer research and modeling 10(5), 667–678 (2018). https://doi.org/10.20537/2076-7633-2018-10-5-667-678

Karypis, G., Kumar, V.: A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices. University of Minnesota, Department of Computer Science and Engineering, Army HPC Research Center, Minneapolis, MN 38, 7–1 (1998)

Karypis, G., Schloegel, K., Kumar, V.: Parmetis: Parallel graph partitioning and sparse matrix ordering library (1997)

Larour, E., Seroussi, H., Morlighem, M., Rignot, E.: Continental scale, high order, high spatial resolution, ice sheet modeling using the Ice Sheet System Model (ISSM). Journal of Geophysical Research: Earth Surface 117(F1) (2012). https://doi.org/10.1029/2011JF002140

Muratov, R.V., Ryabov, P.N., Dyachkov, S.A.: Dynamic domain decomposition method based on weighted Voronoi diagrams. Computer Physics Communications 290, 108790 (2023). https://doi.org/10.1016/j.cpc.2023.108790

Palmroth, M., Ganse, U., Pfau-Kempf, Y., et al.: Vlasov methods in space physics and astrophysics. Living reviews in computational astrophysics 4(1), 1 (2018). https://doi.org/10.1007/s41115-018-0003-2

Petrov, I.B., Favorskaya, A.V., Khokhlov, N.I.: Grid-characteristic method on embedded hierarchical grids and its application in the study of seismic waves. Computational Mathematics and Mathematical Physics 57, 1771–1777 (2017). https://doi.org/10.1134/S0965542517110112

Rao, A.R.M.: Distributed evolutionary multi-objective mesh-partitioning algorithm for parallel finite element computations. Computers & Structures 87(23-24), 1461–1473 (2009). https://doi.org/10.1016/j.compstruc.2009.05.006

Rao, A.R.M.: Parallel mesh-partitioning algorithms for generating shape optimised partitions using evolutionary computing. Advances in Engineering Software 40(2), 141–157 (2009). https://doi.org/10.1016/j.advengsoft.2008.03.017

Rao, A.R.M., Rao, T.A., Dattaguru, B.: A new parallel overlapped domain decomposition method for nonlinear dynamic finite element analysis. Computers & Structures 81(26-27), 2441–2454 (2003). https://doi.org/10.1016/S0045-7949(03)00312-2

Walshaw, C., Cross, M.: JOSTLE: parallel multilevel graph-partitioning software–an overview. Mesh partitioning techniques and domain decomposition techniques 10, 27–58 (2007)

Xia, Z.: Review of architecture and model for parallel programming. Applied and Computational Engineering 8, 455–461 (2023). https://doi.org/10.54254/2755-2721/8/20230225

Yang, X., Shan, J.L., Yu, F., et al.: Boundary constrained quadrilateral mesh generation based on domain decomposition and templates. Computers & Structures 295, 107275 (2024). https://doi.org/10.1016/j.compstruc.2024.107275

Zhou, Y., Cai, X., Zhao, Q., et al.: Quadrilateral mesh generation method based on convolutional neural network. Information 14(5), 273 (2023). https://doi.org/10.3390/info14050273