WAVEWATCH III Hybrid Parallelization for Azov Sea Wave Modelling

The Authors 2024. This paper is published with open access at SuperFri.org The article examines potential applications of WAVEWATCH III (WW3), the third-generation wind-wave model. This study delves into the implementation of hybrid parallelization (MPI-OpenMP) and the development of multiple-cell grids tailored for the Azov Sea region. It elucidates fundamental equations of the model, their discretization, and software execution. The multiple-cell grid strategy employs high-resolution cells within the region of interest, gradually increasing cell density in other areas to optimize memory consumption. A 6-level multiple-cell grid was speciﬁcally crafted for the Azov Sea, with an algorithm outlined for its generation incorporating two reﬁnement methods. This algorithm enables the creation of reﬁned multiple-cell grids near shorelines at varying levels, along with the capability to reﬁne grid structures in arbitrary zones. Additionally, the article presents hybrid parallelization techniques for the wave spectral component (MPI-OpenMP), assessing scalability in both MPI and hybrid deployments. The WW3 model offers a multigrid option facilitating parallel operation of subdomains akin to domain decomposition, while ensuring parallelization of each subnet via the component decomposition method.


Introduction
Modern simulations of ocean wave motion are becoming increasingly intricate and resourceintensive.Among the primary tools in this domain is WW3, an advanced wave modeling system engineered to accurately and efficiently replicate wave phenomena across oceans.
The WW3 model serves as a numerical tool for wave prediction in oceans and other aquatic environments.Developed and maintained by the National Center for Environmental Prediction (NCEP), a division of the National Oceanic and Atmospheric Administration (NOAA) in the United States, the latest iteration, WAVEWATCH III version 6.07, was released in 2019 and is accessible on GitHub.
WW3 enjoys widespread usage and recognition in forecasting ocean wave motion.However, like any modeling system, it both has its strengths and weaknesses.WW3 excels in delivering highly precise forecasts of wave motion, particularly over extended temporal and spatial scales.The model accounts for numerous factors influencing wave formation, including wind, currents, tides, and other parameters, rendering it versatile for a multitude of applications.By incorporating various wave components, such as wind waves and tidal waves, WW3 enhances realism across diverse scenarios.Additionally, WW3 offers various configurations and customization options tailored to specific tasks, encompassing global and regional forecasting endeavors [1][2][3][4][5].
The model might demand substantial computational resources, particularly when employing finer spatial resolutions or conducting global forecasts.Limitations may emerge at regional levels, particularly in regions characterized by intricate coastline geometries or proximity to continents.Additionally, WW3 exhibits a restricted prognostic horizon, potentially complicating the accurate prediction of waves over extended periods.
In summary, notwithstanding these constraints, WW3 retains its efficacy as a robust tool for wave prediction and finds broad utility across diverse domains of oceanography and maritime safety [6].
The WW3 model offers various grid configuration options tailored to different forecasting requirements and scales.The adoption of multiple-cell grids within WW3 has emerged as a pivotal factor, elevating standards of accuracy and adaptability in modeling practices.Multiplecell grids afford the potential for enhanced granularity and adaptability in regional coverage, a critical aspect for proficient wave motion forecasting.In this paper, we examine the significance of employing multiple-cell grids within WW3 and their consequential impact on augmenting spatial resolution and precision in wave modeling within the Azov Sea region [7].
The article is organized as follows.Section 1 is devoted to the description of the WW3 model, the governing equations of the model and their discretization are presented.Section 2 describes the algorithm for generating a Multiple-Cell Grid implemented in WW3, based on this algorithm, a Multiple-Cell Grid was built to simulate processes in the Azov Sea.Section 3 includes a description of The main elements of the WW3 program.Section 4 demonstrates numerical experiments conducted on the basis of WW3 for the Azov Sea, including Simulation results of the direction and period of the peak wave, maximum wave height in the Azov Sea.Section 5 describes the parallel implementation of WW3 based on the MPI-OpenMP hybrid approach.Conclusion summarizes the study and points directions for further work.

Governing Equation in WW3 Model
The principles governing quasi-homogeneous waves find localized application in scenarios where the approach involves a gradual alteration in flow and depth conditions.This application yields the subsequent relationships [7]: The variables presented are defined as follows: k represents the wavenumber vector, where k denotes the wavenumber, and θ signifies the direction perpendicular to the wave crest or spectral component, equating to the direction of k. σ = 2πf r stands for the relative or natural frequency, while ω = 2πf a represents the absolute frequency.Moreover, d signifies the average water depth, and U indicates the flow velocity, averaged over both depth and time on the scale of individual waves.
The definition of k and ω within the wave phase function implies the conservation of the number of wave crests: The density spectrum characterizing the dispersion of surface height F is contingent upon all independent phase parameters, denoted as F (k, σ, ω), exhibiting spatial and temporal variability across scales exceeding those of individual waves.The primary spectrum encompasses the wave number-directional spectrum F (k, θ).The WW3 output comprises the frequency-directional spectrum F (f r , θ).Diverse spectra can be derived from F (k, θ): where c g is the group velocity.
The formula governing wave propagation is as follows: The expression utilizes D/Dt to denote the total derivative, with S indicating the cumulative impact of sources and absorbers on the spectrum F .
The equilibrium equation governing the spectrum N (k, θ; x, t) takes the following format: where c g = (c g sinθ, c g cosθ), s is the coordinate in the direction θ, and m is the coordinate perpendicular to s [7][8].
The standard source terms employed in WW3 are delineated as follows: The components within the equation consist of the source term denoted as S, where S ln represents the linear input term facilitating model initialization and enhancing the initial wave growth's realism.Additionally, S in pertains to the interaction between the atmosphere and waves, while S nl characterizes nonlinear wave interaction.Furthermore, S ds corresponds to wave interaction with the ocean, predominantly encompassing crashing waves, and S bot denotes wave interaction with the bottom.S db serves as an additional criterion for faults during extremely low water conditions, while S tr represents the term governing triadic wave interactions.Moreover, S sc accounts for wave scattering by bottom elements, and S ref signifies the energy sources for infragravity waves [9].
Various combinations of these fundamental source terms are available, with nonlinear wave interactions feasibly modeled via discrete interaction approximation.

Discretization
If equation ( 8) is directly solved, it results in a noticeable reduction in spectral resolution in shallow water.This decrease in resolution can be mitigated by solving the equation on a grid featuring variable wavenumbers, implicitly accommodating changes in the kinematic wavenumber due to shallow depths.Such a grid of wavenumbers corresponds to a spatially and temporally consistent grid of relative frequencies.The respective local grid of wavenumbers can be computed directly from the varied frequency grid and dispersion relation (1), thereby becoming dependent on the local depth d.To facilitate cost-effective calculations of S nl and ensure proper segregation of rising frequencies, frequency sampling utilizes exponentially increasing increments.This approach ensures that the variable frequency resolution is proportionate to the local frequency [10,11]: The impact of employing a spatially variable grid will be examined in relation to equation (8).Representing a grid featuring a variable wavenumber as κ, the equilibrium equation is expressed as: In WW3, equation ( 14) is resolved utilizing the fractional step method.The initial step manages temporal alterations in depth and the corresponding adjustments in the wavenumber grid.By isolating the effects of transient changes in water level, the grid remains constant, while the depth becomes quasi-stationary for subsequent fractional steps.Additional fractional steps account for spatial propagation, intra-spectral propagation, and source conditions.
This division into multiple stages ensures efficient vectorization and parallelization concurrently.Furthermore, temporal partitioning enables the utilization of distinct partial or dynamically adaptable time intervals across various segmented stages of the model.
In the upstream 2nd order non-oscillatory advection scheme (UNO2), the wave action's interpolated value at the mid-flux (MF) point for the cell face between cell i − 1 and cell i is defined as In the context provided, i− represents the index pertaining to the cell face situated between cell i − 1 and cell i; φ denotes the propagation velocity at the boundary between points i and i−1; the indices U , C, and D signify the Upstream, Central, and Downstream cells, respectively.When φ > 0, The UNO scheme, implemented on multiple-cell grids, demonstrates stability, conservativeness, and shape retention.
The process of updating the flow and cell values is executed based on the following formulas: where F i− denotes the flux between grid points with indices i and i − 1 in φ-space, while F i+ represents the flow at the cell face between cell i and cell i + 1.Its estimation can be achieved by utilizing the average flow value, akin to equation ( 16), albeit with the substitution of i by i + 1. UNO2 schemes are characterized by conservativeness, shape preservation, and immunity to numerical fluctuations.They also maintain the component correlation of the transferred variables.UNO2 is implemented within the multiple-cell grid framework to facilitate spatial and interspectral propagation in WW3 [14].

Multiple-Cell Grid Generation Algorithm
A multiple-cell grid employs high-resolution cells in designated regions while incrementally increasing cell density elsewhere to optimize memory utilization.Let's establish the minimum length of the grid side as 1, and denote the side length of the grid at the n-th level as 2 n−1 .Consequently, a 4-level multiple-cell grid encompasses four grid tiers with side lengths of 1, 2, 4, and 8.The largest grid within WW3 is referred to as the base grid.Figure 1 illustrates the layout of the 4-level coastal refined multiple-cell grid.

Figure 1. 4-level refined multiple-cell grid
The resolution of bottom bathymetry data for generating a multiple-cell grid should adhere to the same minimal selection criteria as that of a single-level grid.
The fundamental grid must satisfy the following criteria: the grid can be scaled at a 1:2 ratio, the depth of the grid must exceed the minimum depth requirement of the WW3 model, grid points near shorelines should belong to a level 1 grid.We introduce the notion of a limited radius, representing a distance within which a ground grid of a level other than the first is prohibited.
These restrictions impose a boundary on the grid, wherein ground grids beyond a certain distance (limited radius) are disallowed, except for the level 1 grid.As illustrated in Fig. 1, the limited radius for a 4-level grid is 1+2+4 = 7, for a 5-level grid it is 15, and for a 6-level grid it is 31.It is also evident that the bounded radii of N-level grids are given by If the base grid meets the specified constraints, it will be included in the multi-cell grid file.Conversely, if the base grid fails to meet the constraints, the large grid will be evenly subdivided into four 3-level grids and assessed against the same criteria until further division is no longer feasible.
The grid generation procedure follows a recursive approach, wherein grid cells undergo continuous division until further division is unfeasible.Thus, the grid generation process can be executed using a recursive loop structure, allowing users to adjust the multi-cell grid level.
In the recursive loop, the grid generation process functions iteratively.If the grid fails to meet the specified constraints, the function will invoke itself until either the constraints are satisfied or the grid becomes unsplitable.Figure 2 illustrates the algorithm of the program designed to generate a multi-cell grid for the Azov Sea.

Figure 2. Algorithm for organizing recursive cyclic coastal refinement
To achieve a flexible form of grid refinement, we utilize a curve to delineate the area necessitating refinement, while imbuing hierarchical attributes into the curve.Consequently, grid refinement can be implemented to the specified level within any designated area (refer to Fig. 3).
In order to impose constraints on the refinement of an arbitrary area, it becomes necessary to modify the last of the three constraints in the coastal zone refinement method.We postulate that the N -level grid is encompassed by the N -level curve and lies beyond the scope of the (N − 1)-level curve.If a grid intersects with a curve, it is assumed to be encircled by the curve.Accordingly, the following constraints are appended to elucidate an arbitrary area: grids must fall within the boundaries of curves of the same level and must lie outside the boundaries of curves of a lower level, except for the base grid, the base grid must lie beyond the boundaries of any level of curve, if a curve merely traverses through a grid, the grid is also considered to be within the boundaries of the curve.
Under these constraints, refinement of any arbitrary domain can be achieved.To ensure that the multiple-cell grid scales only in ratios of 1:2, additional restrictions must be adhered to when defining a refined area using a curve: curves must not intersect, any lower-level curve must be contained within the boundaries of the upper-level curve, the distance between the nth level curve and the (N − 1)-level curve must exceed the length of the side of the n-th level grid (to prevent the two curves from being too close).
The ray method is employed to ascertain whether a grid lies within the curve.A ray, parallel to the x-axis and extending positively from the estimated grid, is introduced.If the number of intersections between the ray and the curve is odd, the estimated point lies inside the curve; if the number of intersections is even, the estimated point is outside the curve.In cases where the ray passes tangentially to the curve, no intersection is counted.
A ray is projected from the point S, which is subsequently intersected by curves, resulting in segments delineated on both sides of the curve, as depicted in Fig. 4.  In the application of the ray method, a polygon serves as an approximation for the curve.Consequently, it becomes essential to tally the intersections occurring between the ray and each edge comprising the polygon.The formula utilized to estimate these intersections is as follows: In the equation provided, (x p , y p ) represent the coordinates of the estimated point, while (x a , y a ) and (x b , y b ) denote the coordinates of the edge's endpoints.Additionally, (x i , y i ) stand for the coordinates of the intersection point where the straight line intersects the ray, and where the segment AB is situated.
The curve maintains continuity and differentiability throughout its entirety.However, the polygon is composed of vertices and edges, rendering the ray method ineffective when the ray intersects the polygon's vertex or coincides with its edge.To rectify these inaccuracies, the mathematical expression for estimating intersections is adjusted to the following equation:

WW3 Model Structure
The fundamental component of WW3 resides in its wave model subroutine, accessible via a standalone shell or any other software necessitating real-time updates of wave data.Complementary programs encompass a grid preprocessor, a tool for generating synthetic initial conditions, versatile software shells tailored for individual or multiple-cell grid applications, along with two input preprocessors and postprocessors facilitating grid and point output data manipulation.Figure 6 delineates the relational diagram depicting the foundational data flow within the system.
The data files are denoted by the .ww3file extension.For the multiple-cell grid wave module, the extension ww3 multi indicates a distinct grid.The WW3 grid preprocessor generates the mod def.ww3 model definition file, containing details regarding the date, obstacles, and parameters governing both physical and numerical methodologies.Initial conditions for the wave model are encapsulated in a restart file named restart.ww3.Additionally, the w3wave procedure produces restart files labeled restartn.ww3.
Boundary conditions are sourced from the nest.ww3file by the WW3 model and generated for subsequent runs in nestn.ww3.Output data is written to files such as out grd.ww3, out pnt.ww3, track o.ww3, and partition.ww3.In versions employing distributed memory, a test output file named testnnn.ww3is employed, with nnn representing the processor number, commencing from 1.

WW3 Simulation Results
The Azov Sea was chosen as a testing ground for evaluating the accuracy of the WW3 model.The peculiarity of the Azov Sea is its weather conditions, which are characterized by steady winds, significantly different for the winter and summer seasons.The winter season is chosen because stable winds from the north-west direction prevail on average in winter.The action of the wind along a fixed direction occurs for a long time, which makes it possible to study wave spectra at various accelerations.
The computational areas approximating the configuration of the shores and the bathymetry of real marine basins are determined in the predictive model on regular latitude-longitude grids.The bathymetry required for calculations and the corresponding land-sea mask for each of the basins are constructed using navigation maps.The resulting sampling of the spatial grid is 1.2' in both directions (approximately 2 km).The spectral resolution of the model is 24 directions.The time step for integrating the functions of sources and sinks of wave energy is 5 s, the time steps for energy transmission over the spectrum were 120 s.The wind direction varies in the range from west-northwest to northwest.
The WW3 model was verified on the basis of a regressive analysis of data obtained from calculations and observations from the database Coastal array of monthly average wave heights in the Azov Sea. Figure 7 shows the average (Hsing) and maximum wave heights (Wlen), along with the average wavelength (HsingMAX) by month in the Azov Sea.Furthermore, the directions, periods of peak waves, and maximum wave height for the Azov Sea were calculated based on WW3.The data obtained from the model aligns with the Unified State Information System on the Situation in the Oceans.Notably, the highest wave heights were recorded in the central Based on the WW3 spectral model, the forecast of wind wave parameters in the Azov Sea was carried out.The predictive meteorological fields necessary for calculations (wind speed, water and air temperature) are represented by the products of the medium-term meteorological forecast systems of the Hydrometeorological Center of Russia and NCEP/NOAA.The general grid of winds in the ww3-multi process is used as the input grid.To do this, you need to generate Wind.input and mod-def.input(Fig. 8).The calculation grid of the Azov Sea has a resolution of 2 km.Geographical characteristics of the simulated area: the westernmost point: 34.5, the easternmost point: 39.5, the northernmost point: 47.4, southernmost point: 45.2.Data on the current wind speed in the simulated area at an altitude of 10 m is used for calculation.
Prognostic maps of wind wave parameters are constructed based on calculations based on the WW3 spectral model.The predictive modeling of the height of the prevailing waves (Fig. 9), the average length (Fig. 10) and the period of wind waves (Fig. 11) in the Azov Sea at the initial moment of time, and after 3, 6 and 9 hours was carried out.The average wavelength and period of the waves are calculated as weighted averages over the entire frequency-angular spectrum resolved by the model, with weights proportional to the density of the distribution of wave energy over the spectrum.In the Azov Sea, wave heights predominantly remain below 1 meter, with a recurrence rate of approximately 75%.Waves measuring 1-2 meters occur with a frequency ranging between 20% to 45%, while those exceeding 2 meters repeat no more than 13% of the time.Within the central region of the sea, wave heights typically do not surpass 3.5 meters, with occurrences of heights reaching 4 meters being exceptionally rare [14].The average discrepancy in the computed wave height values was recorded to be between 5-15 centimeters.
Simulation of wind waves in the Azov Sea makes it possible to predict changes in the height, length and period of waves in a certain marine area, which is important for the safety of navigation and coastal protection.Predictive modeling provides an opportunity to better understand the physical processes underlying wave formation and propagation, climatic and hydrological processes in a given marine area and their impact on the environment and human activities.Thus, wave wave modeling is important both for scientific research and for practical applications in marine and coastal areas.

Hybrid Parallelization (MPI-OpenMP) WW3
Parallel launch capabilities have been significantly expanded in WW3 version 5.01, in which multithreading and parallelization using the MPI switch, pure MPI, pure OpenMP and hybrid MPI-OpenMP approaches have become available [8].
In MPI mode, the WW3 model is parallelized using the decomposition of wave spectral components (CD), which imposes a limit on the number of MPI ranks.While hybrid or combined MPI-OpenMP parallelization can enhance node usage, the scalability of OpenMP is limited to multiple threads.
The Domain Decomposition (DD) method serves as a widely utilized tool for parallelization in atmospheric and oceanic models.Presently, most DD applications are based on rectangular subdomains.Despite its simplicity in implementation using standard MPI procedures [12], domain decomposition suffers from potential load imbalances among blocks.Load balancing ensures that each processor carries out a comparable amount of work to prevent overloading on multiple processors.In a parallel environment, the processor with the highest load largely determines the overall efficiency of the model, as all processors typically await the completion of the last processor's tasks.An effective way to implement domain decomposition involves dividing the x-space into regular rectangular blocks.However, this approach may result in load imbalances in wave models due to variations in the number of sea points across such blocks, influenced by the uneven distribution of land masses.Addressing this issue requires designing the blocks to encompass approximately equal numbers of sea points.
Another method of parallelization in the WW3 model involves decomposing spectral components, abbreviated as CD, and transmitting data via the Message Passing Interface (MPI).Unlike spatial domain division, the CD method addresses the propagation of each spectral component of the wave across the entire region on a single processor, facilitating parallelization with as many processors as there are spectral components.However, the effectiveness of the CD method is limited when two or three spectral components are assigned to the same MPI rank due to variations in computational load across different frequency components.Hybrid parallelization, combining MPI and OpenMP, is a means to extend CD parallelization, theoretically allowing one MPI rank to span multiple processors within a single computing node with shared memory.Nevertheless, in practice, the scalability of the OpenMP method does not exhibit significant improvement beyond multiple OpenMP threads.
Another approach to extending parallelization in WW3 is to combine the CD method with the Domain Decomposition (DD) method.In this configuration, the entire model area is partitioned into subdomains and interconnected using boundary exchange, akin to the standard DD method.However, the combined CD-DD method treads each subdomain as a regional model and additionally parallelizes it using the CD method.
The CD-DD method offers an advantage over the pure DD method by reducing the number of subdomains and, consequently, the number of boundary points.Its scalability is nearly linear when the number of subdomains is small but levels off when the number exceeds the optimal threshold.Another challenge with the model is the memory constraint on a single computer node.Since the MPI method allocates memory across all MPI ranks, data shared by all ranks is duplicated for each rank on a single computing node, even in cases where the node has shared memory capabilities.
CD parallelization is specifically applied to advection and diffusion processes.Wind generation and nonlinear wave interactions are parallelized based on localized sea points.With the CD scheme, land points are excluded from the storage of wave spectra, and the remaining spectra of sea points are evenly distributed across MPI ranks.Terms associated with sea points are computed by iterating through local sea points within each MPI rank.In the MPI-OpenMP hybrid mode, additional parallelization of the local loop at each MPI level is achieved through additional OpenMP directives.
The spectrum in this model is sampled using 30 directions and 36 wavenumbers.Each individual process in MPI is responsible for storing data on the spectra of wave components for certain points in the sea.The process of calculating wave propagation over the entire region for a specific component of the spectrum is performed in one of the MPI processes.To use resources more efficiently, each MPI (M ) process is divided into several OpenMP (N ) threads within a single node with shared memory.
In the case of WW3, hybrid parallelization typically comprises the following elements.MPI is employed for parallelization between nodes of a computing cluster or multiple processors.Each process manages its own data region, and MPI facilitates information exchange between processes and synchronizes task execution across different nodes.OpenMP is utilized for parallelization within a node (i.e., within a processor), enabling more efficient multithreading at the processor level, which is particularly advantageous on modern multicore architectures.Hybrid parallelization can be implemented for 20×N nodes, but is most effective at M = 360 or 10 × N nodes due to different wave velocities in different parts of the computational domain.Hybrid parallelization can use up to 30 computing nodes multiplied by N , but it shows the best efficiency when using about 360 MPI processes or 10 nodes multiplied by N threads (Fig. 13).The total hybrid CPU usage is N per M cores.
Scalability levels off after 360 processes for both MPI and hybrid launches due to load balancing of various spectral components.Component decomposition methods have a limitation that is 1/3 of the spectral components (MPI processes).Hybrid parallelization also has a limit of 9 OpenMP threads.The WW3 model incorporates a multiple-cell grid feature, enabling subnets to operate in parallel akin to domain decomposition (DD), while preserving parallelization within each subnet using the component decomposition (CD) technique.This multiple-cell grid capability not only facilitates the integration of CD-DD parallelization but also allows for higher resolutions.
The multiple-cell grid option has been integrated into the multiple-cell grid module of WW3, enhancing its capabilities.
This update includes enhanced hybrid parallelization in the multiple-cell grid module of WW3, enabling it to effectively handle multiple-cell grids.The improved hybrid parallelization leads to more efficient scaling with increased OpenMP threads and can potentially halve the runtime compared to MPI-only execution.

Conclusions
The study provides a comprehensive analysis of the modern version of the WW3 model, encompassing its fundamental equations, discretization techniques, and software implementation.Additionally, it delves into the construction of a multiple-cell grid for the Azov Sea and elucidates an algorithm for its generation employing two refinement methods.
Furthermore, hybrid parallelization of the wave spectral component (MPI-OpenMP) is introduced, with scalability analysis conducted for both MPI and hybrid launches.It is observed that scalability levels out after 360 ranks for both MPI and hybrid launches due to load balancing considerations across various spectral components with disparate velocities.
The updated multiple-cell grid module in WW3 is presented, boasting improved hybrid parallelization capabilities and the capacity to handle multiple-cell grids.Enhanced hybrid parallelization yields superior scaling with increased OpenMP threads and has the potential to reduce computational time compared to MPI execution.
The findings underscore the significance of the WW3 model in the realms of hydrodynamics and supercomputing.The research highlights the efficacy of hybrid parallelization and the revamped multiple-cell grid module in enhancing model scalability and performance.Further development and optimization of WW3 model parallelization hold promise for delivering even more precise and efficient modeling outcomes in oceanography and marine dynamics.

Figure 3 .
Figure 3.The scheme of using curves to refine the grid

Figure 4 .
Figure 4.The ray method used to determine whether a point is inside a curve y s > min(y a , y b ), y s ≤ max(y a , y b ) : intersection, x s < x i .(19)

Figure 5 .
Figure 5. Multiple-cell grid for the Azov Sea

Figure 5
Figure5illustrates the multiple-cell grid constructed for the Azov Sea.The algorithm facilitates the creation of multiple-cell grids featuring varying levels of granularity, thereby enabling enhanced modeling of wave dynamics across diverse regions within the sea and coastal areas.

Figure 6 .
Figure 6.The main elements of the WW3 program

Figure 7 .
Figure 7. Simulation results of the maximum wave height in the Azov Sea

Figure 8 .
Figure 8.The wind speed at an altitude of 10 meters at the initial moment of the simulation (m/s), the arrows show the wind direction

Figure 9 .Figure 10 .
Figure 9.The results of modeling the heights of the prevailing waves (m) at various points in time -at the initial moment, after 3, 6 and 9 hours.The arrows show the average direction of wave propagation

Figure 11 .
Figure 11.The results of modeling the average wave period (sec) at various points in timeat the initial moment, after 3, 6 and 9 hours.The arrows show the average direction of wave propagation