What Makes Fidelity the Preferred Choice for Large Eddy Simulation (LES)?

To learn more about the Cadence Fidelity LES solver, watch the recording of Introducing Fidelity CharLES: GPU-Accelerated Wall-Modeled Large Eddy Simulation - Cadence (Cascade)

Computational fluid dynamics (CFD) engineers must consider turbulence when designing a car, airplane, or sailing vessel as their systems’ aerodynamics or flow field incorporate eddy effects. To do this, turbulence models are used in CFD simulations to allow for the inclusion of fluid flow turmoil occurring in real-world scenarios. More specifically, the Reynolds Averaged Navier Stokes (RANS) turbulence model is preferred when computing resources are limited and/or time is not infinite. Yet, with a RANS turbulence model, the accuracy of simulation results is compromised in certain cases/instances. To eliminate this conundrum, Large Eddy Simulation (LES) comes in handy and is especially important when dealing with combustion and chemical problems due to their nonlinear nature. But, here to, computing power limitations do exist which have largely rendered LES to not be a broadly applicable solution. Fortunately, GPU technology has enabled the compute landscape to evolve and, as such, LES is now more accessible and affordable. The Cadence Fidelity LES solver is especially suited for LES simulation because it uses nonlinearly stable numerical schemes, producing highly accurate and high-fidelity simulation results that are efficient to compute using GPUs.

History of LES Modeling at Cadence 

In 1980, Professor Parviz Moin of Stanford and founder of Cascade Technologies, now a Cadence company, conducted groundbreaking research on turbulence modeling. At the time, there were numerous studies being conducted to investigate turbulence experimentally. Figure 1 illustrates an experiment conducted using hydrogen bubbles in boundary layers, which was aimed at studying the beautiful, coherent but chaotic structure in turbulence.

        Fig 1: Simulation result (Moin & Kim, 1981) (left), experimental result (Kim, Klein & Reynolds, 1970)(right).

Parvis and his colleagues presented their simulations at the American Physical Society Conference 1981, and these simulations were run using ILLAC IV (15 MFlops machine) at NASA Ames. Their method was to photograph a screen and build the visualization, which, at the time, was unconventional. However, their simulations showed that it was possible to accurately predict the Navier-Stokes equations in time, capturing both the dynamics and the turbulence statistics. Today, computing power has advanced significantly, moving beyond mega, giga, tera, peta, and exa for even greater capabilities. Modern LES modeling impacts some real systems and is at a point where turbulence modeling can be done with more precision and speed. It is no longer limited to research centers and is increasingly used in commercial design environments.

Why LES Today?

Four differentiating technologies that enable high-fidelity large eddy simulations to be possible or rather practical today include:

1. Grid Discretization: The importance of high-quality and relatively isotropic grids cannot be overstated when it comes to time-dependent simulations. While some anisotropy near the wall can be beneficial, it is crucial that the grids maintain a consistent level of quality throughout the simulation.
2. Numerical Methods: It is essential to use robust, nonlinearly stable numerical methods and advanced physical models that accurately represent flow physics.
3. Data Analytics: As you'll be generating extensive data sets, the ability to rapidly visualize and comprehend this data is substantial.
4. Scalability: High performance and fast turn-around time are critical for LES to gain wide-spread adoption.

For many years, the philosophy for LES modeling has been that it requires low-dissipation numerical schemes. However, these low-dissipation schemes are hard to construct for multiphysics applications and complex geometries. Although the actual dissipation in high Reynolds number flow is low, the numerical dissipation in typical CFD codes is very high. With the Fidelity LES solver, however, it is possible to have low-dissipation numerical schemes that are stable.

For mesh generation, the Fidelity LES solver employs a mesh generator that produces polyhedral meshes with regions of varying resolutions and transitions between them. This mesh is essentially a 3D Voronoi diagram created around a specific set of points. When these points are introduced in an organized manner, meshes with a high degree of uniformity are generated. This makes the Fidelity LES solver suitable and convenient for generating meshes for LES.

High-Fidelity Multiphysics LES Predictions

One notable example of LES that has been successful in recent years is the study of a turbulent jet flame (Sandia D piloted jet flame). As shown in Figure 2, the methane fuel is injected from the center and mixes with air from the surrounding coflow. The jet spreads, reacts, reaches a maximum temperature, then cools as it entrains more air from the coflow. The solver’ low-dissipation schemes and efficient combustion models deliver robust results over a range of mesh sizes. In fact, changes in mesh resolution from 400 thousand cells to 16 million cells do not significantly affect the time-average and RMS of temperature along the centerline of the flame, as shown in Figure 2. This example highlights the effectiveness of LES in solving complex problems where turbulent mixing and nonlinear multiphysics are important.

 Figure 2: LES of the Sandia D jet flame. Changes in mesh resolution from 0.4M to 16M cells does not significantly impact the predicted flame structure (left) or centerline temperature profiles (right).

Another example of successful LES application is the fan noise Source Diagnostic Test (SDT), which was a benchmark developed by a cooperative effort between NASA and GE Aircraft Engines to investigate noise generation mechanisms in modern high bypass ratio turbofan engines. The full annulus of the fan, the nacelle, the low-noise outlet guide vane (OGV) and the entire test section are simulated with Fidelity LES (figure 3).

.

Figure 3: NASA SDT fan benchmark: visualization of the flow Mach number and surface shear stress on the rotor, hub, and low-noise OGV.

The LES results are compared to available experimental data in terms of aerodynamic performance, unsteady flow fields, and far-field noise, and show good overall agreement (Figures 4 and 5). The SDT configuration is also used to assess the solver scalability and increased computational throughput with GPU acceleration: for the present O(140) million cell mesh, the simulation results for 10 full rotations are obtained in approximately 6 hours on 40 standard GPUs.

Figure 4: NASA SDT fan benchmark: comparison of the laser Doppler velocimetry (LDV) experimental data and LES results for axial velocity statistics between the rotor and the low-noise OGV.

Figure 5:  NASA SDT fan benchmark: Comparisons of the sound power levels (PWL) from the experiment (black) and LES (red). The vertical dashed lines represent the blade passage frequency (BPF) and the first four harmonics.

GPU/CPU for LES 

Running an LES simulation can be expensive. For instance, performing high-fidelity simulations of combustion dynamics can cost upward of $10K per simulation using CPU-based hardware on public cloud or on-premise, with each calculation taking up to one million core hours. This can quickly become quite expensive, especially when hundreds of simulations are needed. Although codes that scale on CPUs to compute these flows in under a day have been developed, the associated high cost has hindered the widespread use of LES in industry and design environments. Thankfully, GPU technology is changing the game. With the advent of GPU acceleration, there is a significant reduction in simulation costs and an increase in computational efficiency (Figure 6).

Figure 6: GPUs provide 9X throughput for the same cost as CPU, or 9X lower cost and 17X less energy for the same throughput as CPU

It is critical for CFD designers to consider turbulence in automotive, aerospace, and marine (aka sailing vessel) applications because these systems’ aerodynamics or flow field incorporate eddy effects. However, until now, computing power limitations have limited LES as a broadly applicable solution. Cadence Fidelity makes LES simulation much more accessible through the use of GPU compute resources that provide both an increase in computational efficiency and a significant reduction in simulation costs.

To learn more about the history and technology behind Cadence Fidelity LES solver, watch the recording of Introducing Fidelity CharLES: GPU-Accelerated Wall-Modeled Large Eddy Simulation - Cadence (Cascade)