3.2.1. Symmetries and invariants

Lambert (Reference Lambert1997) has examined the possible reasons behind the effectiveness in mathematics in his book ‘Is the efficiency of mathematics unreasonable?’, inspiring the title of this very subsection. Within this work, he explores various philosophical theories, such as Pythagoreanism, Platonism, empiricism etc. intended to explain the effectiveness of mathematics and, in general, to every scientific methods. Among them, he mentioned formalism, which stipulates that mathematics is effective because it relies on symbols and rules to manipulate them. Nevertheless, Lambert (Reference Lambert1997) argues that this point of view can explain the effectiveness of mathematics only as a kind of miracle, as it is difficult to see how a ‘game’ governed solely by constraints of internal coherence could correspond to a reality external to thought, such as physical phenomena. However, this theory gave birth to relationism, which considers mathematics as the formalised representation of all possible conceptual relationships, including numerical ones. For instance, according to Bohr (Reference Bohr1958), ‘we shall not consider pure mathematics as a separate branch of knowledge, but rather as a refinement of general language, supplementing it with appropriate tools to represent relations for which ordinary verbal expression is imprecise or cumbersome’. Here, mathematics is not just a language with symbols and rules, but serves as a means of formally translating and elucidating the relationships between concepts. This might explain why physics is well mathematised, since most results can be expressed through structured relationships between the fundamental quantities of the system, structures that mathematics can faithfully capture and elaborate. According to Vandamme (Reference Vandamme1976), ‘The knowledge in general, and even more so the scientific knowledge, is characterised by its structures’. Nevertheless, it is worth noting that not all relationships are useful. Corry (Reference Corry1996) notes that mathematicians are not interested in cataloguing all possible structures, but rather to investigate ones that possess rich invariants. This is consistent with physics, where an ‘element of reality’ cannot be described properly without recourse to invariants preserved under the symmetries of the system. This need of invariance is motivated by our partial observation of reality that requires invariants to conclude without exhaustive observations, which is often impractical or cumbersome. In other words, reality in physics must be robust to a change of point of view. This is exactly what research in physics is doing, but more generally how humans and animals perceive the surrounding world. For instance, Florack et al. (Reference Florack, Romeny, Koenderink and Viergever1994) and Lindeberg (Reference Lindeberg2013) have demonstrated that human vision and cognitive functions rely on some invariants and symmetries: ‘It is apparently the case that, at least up to some approximation, many visual tasks can be solved on the basis of such invariants only. This observation should affect the way computer-vision and image-analysis tasks are handled as well’ (Florack et al. Reference Florack, Romeny, Koenderink and Viergever1994).

Modern science, and mathematics, now heavily rely on the study of symmetry, as illustrated by Anderson (Reference Anderson1972) who noted that ‘it is only slightly overstating the case to say that physics is the study of symmetry’. Early on, Curie (Reference Curie1984) proposed his principle of symmetry, based on ideas from F. Neumann and B. Minnigerode, where ‘the symmetries of the causes are to be found in the effects’. This highlights the profound relevance of symmetries to understand the surrounding world. Interestingly, symmetries in physics are intricately related to invariants, i.e. conserved quantities, also known as first integrals, as elegantly elucidated by Noether’s theorem (Noether Reference Noether1918). As an example, rotational symmetry yields the conservation of angular momentum, and invariance under a translation in time leads to the conservation of energy. Moreover, symmetries and invariances are much more intricate and influential in modern physics.

In the realm of fluid mechanics, the rigorous investigation of symmetry and invariance has also been studied for a long time. For instance, Golubitsky & Stewart (Reference Golubitsky and Stewart1986) studied the specific symmetries acting on the stability of Taylor–Couette flows. They demonstrate that symmetries inherent in bifurcating systems have a strong influence on their behaviour, which motivates their study. The effect of symmetries on flow instability has been recently revisited by Sierra-Ausin, Fabre & Knobloch (Reference Sierra-Ausin, Fabre and Knobloch2024), focusing on the $O(2)$ symmetry applied to wake instability. Similarly, combustion instabilities in annular combustion chambers are governed by the azimuthal symmetry. When this symmetry is broken because of different flame types (Bauerheim et al. Reference Bauerheim, Salas, Nicoud and Poinsot2014b ), azimuthal mean flow (Bauerheim et al. Reference Bauerheim, Cazalens and Poinsot2014a ) or stochasticity (Bauerheim et al. Reference Bauerheim, Ndiaye, Constantine, Moreau and Nicoud2016a ), modal splitting occurs which affects the whole stability of the system. Recent works in thermoacoustics still involves this notion of symmetry breaking (Aguilar et al. Reference Aguilar, Dawson, Schuller, Durox, Prieur and Candel2021; Faure-Beaulieu et al. Reference Faure-Beaulieu, Indlekofer, Dawson and Noiray2021). Symmetries have also been exploited in fluid mechanics to build turbulence models. Indeed, Grebenev & Oberlack (Reference Grebenev and Oberlack2007) and Fakhar et al. (Reference Fakhar, Hayat, Yi and Zhao2009) studied the symmetry of the incompressible Navier–Stokes equations (Cantwell Reference Cantwell1978). Based on these symmetries, Razafindralandy, Hamdouni & Oberlack (Reference Razafindralandy, Hamdouni and Oberlack2007) and later Sayed et al. (Reference Sayed, Hamdouni, Liberge and Razafindralandy2010) derived new turbulence modelling which preserve those symmetries. They also analysed the existing turbulence models available at that time, including the Smagorinsky model (Smagorinsky Reference Smagorinsky1963), showing that only its dynamic version (Germano et al. Reference Germano, Piomelli, Moin and Cabot1991) allows symmetry preservation.

These examples reveal the uncanny effectiveness of mathematics because of symmetry and invariance, highlighting its capacity to shape our understanding of reality while existing purely within the realm of abstract ideas. This fundamental concept of symmetries and invariance is also embedded in the very construction of modern AI methods for classification and regression tasks (Gens & Domingos Reference Gens and Domingos2014; Weiler et al. Reference Weiler, Geiger, Welling, Boomsma and Cohen2018; Weiler & Cesa Reference Weiler and Cesa2019; Wang, Walters & Yu Reference Wang, Walters and Yu2021). Typical examples are CNN built upon the translation equivariance principle or graph networks based on permutation invariance. Recently, the community devoted to AI for science has been exploring the necessity of incorporating symmetries or structures directly into neural network architectures (Önder & Liu Reference Önder and Liu2023; Otto et al. Reference Otto, Zolman, Kutz and Brunton2023), not only as a design principle, but also to align with the intrinsic demands of science, which fundamentally relies on these principles. Similarly to the Erlangen programme proposed by F. Klein in 1972 to unify geometry as the study of invariants or symmetries using the language of group theory, a recent objective in AI is to unify the various network architectures into a general framework, known as geometric deep learning, based also on symmetries or invariants (Bronstein et al. Reference Bronstein, Bruna, Cohen and Velickovic2021). As a consequence, symmetry and invariants constitute the first pillar to make AI effective in CFD, as further discussed technically in § 4.1.

3.2.2. Scale separation

Furthermore, while investigating invariants for image recognition, Florack et al. (Reference Florack, Romeny, Koenderink and Viergever1992) also emphasised the scale dependence of these invariants. This notion of scale is crucial in physics, where the physical laws governing complex systems exhibit a sort of separation between the various levels of structures associated with spatial and temporal scales. For instance, while the classical Euclidean space is intuitive at spatial scales we encounter in our everyday life, this framework fails to explain phenomena at very low (quantum mechanics) and very large (gravitation) scales. Einstein (Reference Einstein1920) challenged this view by introducing the concept of curved space–time, effective at large scales, but still failing to explain the transition between the continuous world we experience and the quantised energy realm of quantum mechanics. Similarly in fluid mechanics, working at the macroscopic (Navier–Stokes equations) or mesoscopic (lattice Boltzmann method) scales does not require a detailed knowledge of the fine microscopic scales, such as the interactions between the fundamental particles. This scale separation, when available, allows dedicated physical analyses and methodologies to be performed, introducing its own abstraction. For instance, the notion of pressure at the macroscopic scale in a fluid is intuitive, while being much more complex, or even meaningless, at much lower scales.

These considerations are essential not only in physics, but also in the development of AI, as well as across a broad spectrum of disciplines, including sociology, history etc. This idea aligns closely with the epistemological perspective emphasised by Bachelard (Reference Bachelard1938), who considered that the main challenge of science is to push humans beyond their intuitive sense of scale. Additionally, Simondon (Reference Simondon1970) named scale rupture this transition between ordinary scales, where familiar schemes like particles or Euclidean space remain valid, and the extreme scales studied by quantum mechanics and astrophysics, where these schemes inherited from ordinary perception are no longer adequate. It suggests that the effectiveness of mathematics in physics derives from its ability to conceptualise and study phenomena occurring at very different scales compared with those the researcher, as a human being, is used to in a daily-life.

This principle is fundamental in CFD, where applications span an enormous range of scales, from microscopic flows around biological cells (O’Connor et al. Reference O’Connor, Day, Mandal and Revell2016) to astrophysical phenomena, such as supernova-induced turbulence and other large-scale flows in the universe (Müller Reference Müller2020). Early on, engineers and researchers recognised the necessity of classifying these flows based on characteristic non-dimensional parameters, such as the Reynolds or Mach numbers, among others. It is well established that flow behaviour changes drastically as these parameters vary, leading to the emergence of complex phenomena like shock waves or turbulence. These flow regimes can be highly localised, coexist within the same system, interact nonlinearly or even exhibit fractal characteristics (Meneveau & Sreenivasan Reference Meneveau and Sreenivasan1987), making them inherently multi-scale and challenging to capture and analyse.

Consequently, scale separation is central to many advancements in fluid mechanics. For instance, separation between the slowly varying large-scale perturbations from the rapidly varying small-scale of the flow is essential to derive a simplified effective description of the dynamics at large scales, and justify the concept of eddy viscosity (Dubrulle & Frisch Reference Dubrulle and Frisch1991). Similar multiple scale approaches are employed by Ovenden (Reference Ovenden2005) to model sound transmission through a slowly varying duct of arbitrary cross-section, by Faure-Beaulieu & Noiray (Reference Faure-Beaulieu and Noiray2022) to extract the dynamics of the slow-flow variables in an annular thermo-acoustic system, and by Magri et al. (Reference Magri, See, Tammisola, Ihme and Juniper2016) to formulate a set of equations in the low-Mach number limit for linear stability analysis.

Moreover, this multi-scale character of fluid flows lies at the core of many challenges in CFD. To address these, high-order numerical schemes (Bassi & Rebay Reference Bassi and Rebay1997; Nicoud Reference Nicoud2004) and lattice Boltzmann methods (LBMs) (Marié et al. Reference Marié, Ricot and Sagaut2009; Coratger et al. Reference Coratger, Farag, Zhao, Boivin and Sagaut2021) have been developed, specifically to preserve the propagation of fine-scale turbulent structures over long distances and to efficiently capture interactions across a wide range of scales. Similarly, identifying regions with extreme length scales, such as thin shock waves, flames or boundary layers, has led to the development of specialised numerical treatments (Colin et al. Reference Colin, Ducros, Veynante and Poinsot2000; Nicoud et al. Reference Nicoud, Baggett, Moin and Cabot2001). The critical role of scales in CFD is further highlighted by the widespread use of post-processing techniques, such as Fourier and wavelet transforms (Grizzi & Camussi Reference Grizzi and Camussi2012), proper orthogonal decomposition (POD), and other spectral analysis tools. These methods help to extract spatial and temporal scales where energy is generated or dissipated, providing deeper insight into the underlying flow physics.

This inherent multi-scale nature of fluid flows presents a significant challenge for AI, which is not initially specifically designed to handle such drastic scale separation with order of magnitudes. A clear example lies in denoising filters (Pratt Reference Pratt1972), used in some generative models, which often treat high-frequency components as noise and discard them (e.g. Wiener’s filter). In CFD, however, small-scale structures play a crucial role in accurately capturing overall flow behaviour and cannot simply be removed. Recognising this limitation, recent research has increasingly focused on equipping deep networks with multi-scale (Nabian Reference Nabian2014; Mathieu, Couprie & LeCun Reference Mathieu, Couprie and LeCun2016) and multi-resolution (Catalani et al. Reference Catalani, Agarwal, Bertrand, Tost, Bauerheim and Morlier2024; Cheng et al. Reference Cheng, Morel, Allys, Ménard and Mallat2024) capabilities, particularly for flow applications. These advancements open new avenues for improving deep learning methods in fluid mechanics by enforcing specialised treatments at different scales, as further discussed in § 4.2.

3.2.3. Sparsity

Pertaining directly to the second form of effectiveness as defined by Wigner (Reference Wigner1960), i.e. the capability to provide explainable structures, mathematics is based on concise equations (sparsity), facilitating explicability: sparsity requires abstraction, generating a semantics for physics. Semantics refers to the branch of linguistics and logic concerned with meaning. Sparsity and parsimony, as suggested by William of Occam, are of true interest since all models are wrong: ‘Pluralitas non est ponenda sine necessitate’ (Plurality is not to be posited without necessity). Among models with equivalent accuracy, the most effective are those that are sparse. For instance, the compactness of the expressions describing the Navier–Stokes equations is truly remarkable when considering the numerous physical fluid phenomena they govern: these applications range from the dynamics of blood circulation within the human heart (Chnafa et al. Reference Chnafa, Mendez, Moreno and Nicoud2015) to the airflow patterns around an aircraft, and even extend to the turbulent storms on Jupiter or the cataclysmic events of supernovae (Müller Reference Müller2020). How feasible are such compact expressions encapsulating so much physics? Occam’s razor suggests that sparsity is at the root of explicability, but nothing yet allows us to justify that these expressions will be so general and so effective in physics. This fundamental concept of sparsity is central to many approaches in recent AI for physics, such as discovering physical laws by neural networks. This is often achieved through sparsity-promoting algorithms (Brunton et al. Reference Brunton, Proctor and Kutz2016a , Reference Brunton, Proctor and Kutzb ; Silva et al. Reference Silva, Higdon, Brunton and Kutz2020) allowing to focus the explainability of the model in a few terms, and hence be more general by construction.

This explainability induced by sparsity can be illustrated with one remarkable example: the evolution of Maxwell’s equations. Initially, Maxwell (Reference Maxwell1861) formulated a set of $20$ equations to describe the behaviour of electric and magnetic fields. The quest for sparsity and elegance in Maxwell’s equations culminated in the realisation that the equations could be unified and simplified through the introduction of vector and tensor calculus. This reformulation has lead to a compact set of the four well-known equations, which simplifies interpretability through the divergence and curl operators, and manipulation such as deriving Poynting’s theorem for the energy conservation. However, electric ( $\boldsymbol E$ ) and magnetic ( $\boldsymbol B$ ) fields still appear as separate entities, while in fact they can be unified into the same compact theory. This paradigm shift occurred when the equations were again reformulated, but now using the language of differential geometry. This leads to only two compact equations, where the electric and magnetic fields appear as components of a single electromagnetic tensor ( $F_{\mu \nu }$ ), where $F_{0 i} = -E_i/c$ and $F_{\textit{ij}} =- \epsilon _{\textit{ijk}} B_k$ . This unification of electric and magnetic fields is only possible because space and time are themselves combined into a single four-dimensional space–time, so that this sparse formalism naturally incorporates the principles of relativity. Beyond its compactness and elegance, this tensorial representation makes the fundamental Lorentz invariants of the electromagnetic field emerging directly from simple tensor contractions, highlighting the deep geometric structure underlying electromagnetism.

Similar illustrations of sparsity also arise in fluid dynamics, for instance, from the Boltzmann equation. Instead of solving directly the full set of nonlinear Navier–Stokes equations for mass, momentum and energy conservation, LBM reformulates fluid dynamics in terms of a single mesoscopic transport equation for the particle distribution function $F(x, t, v)$ . This sparse formulation makes conservation laws not imposed explicitly, but emerging solely from the moments of this distribution function. This is governed by the symmetries of the velocity space that ensures that only a handful of velocity moments are conserved, while the rest decay. This explains why mass, momentum and energy are the universal conservation laws in fluid flows. In addition, while the velocity space is continuous, its discretisation into a small set of representative velocities (e.g. D2Q9, D3Q19) is sufficient to preserve the essential hydrodynamic moments. This reduction transforms the complexity of the full PDE system into a simple stream-and-collide process, which is computationally effective. In this way, sparsity provides: (i) a compact representation; (ii) a clear physical interpretation; and (iii) a computational effective method.

Another example of sparsity is provided by turbulence modelling. While DNS resolves all turbulent scales, a task that rapidly becomes intractable at high Reynolds numbers, RANS circumvents this by replacing the fluctuating motions with the Reynolds stress tensor, which contains only six independent unknowns. On one hand, Reynolds stress models (RSMs) solve transport equations for each component, offering high fidelity but little interpretability. On the other hand, eddy-viscosity models collapse the entire tensor to a single scalar coefficient, capturing turbulence as an effective viscosity. Despite its simplicity, it provides valuable physical insight by revealing turbulence’s essential role as a mechanism for momentum diffusion and enhanced mixing.

As a consequence, much like the compact form of Maxwell’s equations revealed the underlying structure of electromagnetism, sparse formulations and models of fluid motion provide deeper physical insight by exposing the fundamental role of key quantities in organising flow behaviour. In other words, sparsity promotes explicability. While sparsity often requires new assumptions, sometimes at the expense of predictive accuracy, these models typically retain a remarkable degree of expressive power despite being very low-parametric. However, what about the whole CFD pipeline? In practice, CFD employs high-dimensional inputs such as meshes, yielding algorithms that are not sparse in a formal sense, but nevertheless remain interpretable. I argue this interpretability arises in part because CFD is not monolithic: it is built from an aggregation of models and numerical schemes, each designed for specific objectives (e.g. turbulence closure, discretisation, boundary conditions). Such modularity allows a ‘divide and conquer’ strategy when diagnosing errors, debugging or analysing sensitivities of particular components of the modelling chain. This notion is referred as decomposability by Lipton (Reference Lipton2016), where ‘each part of the model […] admits an intuitive explanation’. In contrast, modern deep learning models tend to be monolithic, highly over-parametrised and opaque: predictive success is achieved only through massive parametrisation, but with interpretability being sacrificed (Doshi-Velez & Kim Reference Doshi-Velez and Kim2017). It highlights that the pursuit of sparsity in AI may involve trade-offs distinct from those in traditional mathematical tools. This also suggests that, while sparsity is indeed a critical pillar of effectiveness in scientific modelling, it may not be sufficient on its own: not all sparse methods are necessarily either effective or explicable, as further discussed in the AI context in § 4.3.

3.2.4. Semantic significance

In science, explicability is crucial, as it constitutes a necessary condition for constructive critical thinking. However, critical thinking can only be exercised in relation to an informed opinion, exhibiting the path and reasoning behind it. This process is often slower and more laborious than the rapid predictions of AI models. Yet, would we be willing to adhere to a result or a decision produced by an unexplainable AI, either in biology and medicine, but also in engineering? Maybe if the gap between AI and humans is substantial. This difference between slow reasoning and ultrafast predictions is characteristic of the distinction between conscious reasoning versus intuition, where the latter is ‘the ability to acquire knowledge, without recourse to conscious reasoning or needing an explanation’ (Patterson & Eggleston Reference Patterson and Eggleston2017).

A similar distinction is often invoked in artificial intelligence: on one hand, symbolic learning methods attempt to mimic conscious reasoning by manipulating explicit rules, logical relations or symbolic representations; on the other hand, statistical learning methods, such as neural networks, rely on detecting patterns in large datasets to provide rapid predictions without explicit reasoning. Mathematics aligns closely with the symbolic side, where logic and reasoning are the angular stones. In the realm of mathematics, we want to reason in an argumentative way to uncover truths. Therefore, mathematicians not only construct reality, but the mathematical notions created directly speak to reason (Bachelard Reference Bachelard1934). This forces mathematics, and thus physics, to be more than just a language with syntax, but also to possess semantics, where the structures and the relationships between mathematical objects are essential: ‘The knowledge, indeed, does not consist in a mere collection of data. On the contrary, it is well known that something can be considered as a data precisely thanks to its relationships with the structure, more or less abstract, of knowledge’ (Vandamme Reference Vandamme1976). In this context, semantics allows the reduction of information to manageable quantities with underlying structure and relationships, presented in a format or symbols (i.e. using the syntax) which facilitates their evaluations. This is precisely what mathematics accomplishes.

A typical example of semantic significance emerges in the construction of the various mathematical sets of real numbers ( $\mathbb{R}$ ), complex numbers ( $\mathbb{C}$ ) and quaternions ( $\mathbb{H}$ ). Consider the case of a real squared number being negative, which has no meaning in $\mathbb{R}$ . By expanding our realm of exploration to include not only $\mathbb{R}$ , but also a new set $\mathbb{C}$ , we invent a new concept (‘i’, the pure imaginary number), which restores the validity of the initial proposition. However, what does this extension bring in terms of semantics? Is the transition from $\mathbb{R}$ to $\mathbb{C}$ merely an artefact to enforce the validity of otherwise meaningless expressions or does $\mathbb{C}$ introduce a new layer of meaning? Additionally, critically, does this meaningful transition only exist as an abstract idea or does it translate into tangible objects when analysing physical phenomena? To answer these questions, we can observed that expanding these sets inevitably alters their underlying syntactic rules. For instance, $\mathbb{R}$ is a complete ordered field, whereas in $\mathbb{C}$ , we lose ordering and in $\mathbb{H}$ , commutativity. These syntactic rules, i.e. constraints governing how mathematical operations are performed, are central to the structure of these sets (Matsumoto & Nakai Reference Matsumoto and Nakai2023), which yields profound semantics much broader than their initial simple definitions. Indeed, each set provides a model for mathematical and physical systems where analogous rules are present or absent. Rotations offer a relevant example: while rotations in two dimensions are commutative, those in three dimensions are not. Consequently, unitary complex numbers can represent two-dimensional (2-D) rotations, and unitary quaternions rotations in three dimensions, since possessing the same structure. This connection between these sets and physical rotations can be formalised through the more general Lie theory, where complex numbers naturally encode planar rotations, while quaternions provide the algebraic structure underlying three-dimensional (3-D) rotations. In other words, syntactic rules and semantics are intertwined. This interconnection stems from the ability of mathematics to provide structures and relationships among diverse objects, either abstract entities as in pure mathematics (e.g. set of numbers) or tangible entities as in physics (e.g. rotations). These mathematical structures serve as a bridge between abstract ideas and real-world physical phenomena, thereby generating profound semantics to mathematics.

As a result, semantics extends beyond the mere need for explainability. Incorporating semantics into a model introduces an inherent structure and sparsity, which in turn reduces computational costs, minimises data requirements, enhances generalisation and fosters greater acceptance within the scientific community. While symbolic AI methods naturally embed such structured reasoning, the current trend remains largely driven by statistical learning approaches, especially deep neural networks. This divergence raises fundamental questions about the future direction of AI in CFD. Should we continue relying solely on data-driven techniques or should we advocate for a stronger integration of symbolic AI? Given that symbolic methods also face significant challenges in tackling real-world problems, could hybrid approaches mixing symbolic AI and deep networks provide a viable alternative for CFD? Addressing these questions requires a deeper reflection on the practical means of embedding semantic structures within modern deep learning frameworks, further discussed in § 4.4.

3.2.5. Four pillars of effectiveness (PoEs)

As a summary, the effectiveness of mathematics in physics can be attributed to four main aspects, hereafter named the ‘four pillars of effectiveness’ (PoEs): (i) symmetries providing internal structures to equations; (ii) scales separation, allowing a specific treatment of the physics for each scale and their interactions; (iii) sparsity, which promotes explicability; and finally (iv) semantic significance, which gives mathematics not a purely grammatical structure, but also a meaning, facilitating abstraction and reasoning. Those aspects will be further analysed in the specific context of the development of artificial methods for CFD, which also applies more generally to computational physics.

Yet, as claimed by Jebeile (Reference Jebeile2024), the actual usability of a model is not determined solely by its accuracy or effectiveness, but also depends on its credibility, which is its ability to earn trust and foster adoption within a community. In mathematics, credibility is inherently tied to rigour and logical proof, but in AI, credibility is contingent: it depends on empirical validation, the reliability and trust in data, and the interpretability of models. For example, Levins (Reference Levins1966) argued that credibility can emerge from the ‘robustness across models’, where consistent conclusions drawn from an ensemble of different models provide confidence to shared results, an idea closely connected with epistemic uncertainties. Furthermore, Simon (Reference Simon1962, Reference Simon1969) emphasised near-decomposability and hierarchical organisation in the study of complex systems, arguing that modular structures enhance both tractability and interpretability. This insight has more recently been explored by Jebeile (Reference Jebeile2024), who mentioned that decomposability plays a key role in establishing the credibility of scientific models. This suggests that beyond the proposed PoEs, AI for CFD may also require explicit safeguards of credibility, not as a condition for effectiveness itself, but as a prerequisite for its widespread use. This fifth pillar will be further discussed in § 4.5.

4.1.1. Physics-aware deep learning

One of the most critical challenges in AI for CFD is ensuring that the models respect the underlying physical laws, such as conservation of mass, momentum and energy. Standard neural networks (figure 1 a), while powerful, often lack built-in mechanisms to preserve these physical principles. This has led to the development of PINNs (figure 1 b and table 2), which incorporate physical laws directly into the loss function. By doing so, PINNs ensure that the network’s predictions are consistent with the governing equations $\mathcal{R}$ of fluid dynamics (Cai et al. Reference Cai, Mao, Wang, Yin and Karniadakis2021; Lu et al. Reference Lu, Meng, Mao and Karniadakis2021). Moreover, compared with CFD working on discretised equations, PINN approximates the solution of the continuous PDE, thus providing in theory solution with an infinite resolution. Yet, in practice, this property may be limited by the sampling strategy used during training (Wu et al. Reference Wu, Zhu, Tan, Kartha and Lu2023). So far, only simple flows have been tackled by PINN (Moser et al. Reference Moser, Fenz, Thumfart, Ganitzer and Giretzlehner2023). To my knowledge, no examples of efficient PINN in complex 3-D flows have been shown. As a consequence, future works to improve the present framework allowing to deploy PINN in complex configurations better than classical CFD still seem illusive.

Nevertheless, several ideas behind PINN have been reused in more recent works. For instance, enforcing the physical equations into the loss function of more classical deep neural networks have shown benefits even on 3-D flows. As an example, Tompson et al. (Reference Tompson, Schlachter, Sprechmann and Perlin2017) enforced the incompressible mass conservation equation $\boldsymbol{\nabla }\boldsymbol{\cdot }\boldsymbol u = 0$ during training, stabilising the solution in time. It also allows a non-supervised approach, acting as a data augmentation technique, or as a manner to control the network error when hybridised with a classical solver (Illarramendi et al. Reference Illarramendi, Alguacil, Bauerheim, Cuenot and Benazerra2020a ). Strongly enforcing physical laws have also been proposed by Trask, Huang & Hu (Reference Trask, Huang and Hu2022), for instance, by exploiting exterior calculus. Similar ideas to enforce physics have been widely investigated to improve neural networks predictions of dynamical systems (Doan, Polifke & Magri Reference Doan, Polifke and Magri2020). For instance, Greydanus, Dzamba & Yosinski (Reference Greydanus, Dzamba and Yosinski2019) introduced Hamiltonian neural networks (HNN) and then Cranmer et al. (Reference Cranmer, Greydanus, Hoyer, Battaglia, Spergel and Ho2020a ) introduced Lagrangian neural networks (LNN) to learn and respect exact conservation laws in an unsupervised manner. In HNN, the Hamiltonian equations ${{\rm d}q}/{{\rm d}t} = {\partial \mathcal{H}}/{\partial p}$ and $ {{\rm d}p}/{{\rm d}t} = - ({\partial \mathcal{H}}/{\partial q})$ are encapsulated in the loss function when learning $\mathcal{H}$ to weakly enforce the symplectic structure of the Hamiltonian system. These techniques are built in a similar way to classical integrators for chaotic dynamical systems developed decades before, such as the symplectic integrators (Yoshida Reference Yoshida1990). Similarly, Hernandez et al. (Reference Hernandez, Badias, Chinesta and Cueto2023) proposed the concept of thermodynamics-informed neural networks to enforce a metriplectic structure of dissipative Hamiltonian systems onto the latent space network, which propagates the reduced state in an auto-regressive manner. A metriplectic structure is a framework that combines both conservative (Hamiltonian, symplectic structure) and dissipative (entropy-producing) behaviours within a single system. This structure is used to model non-Hamiltonian systems that involve energy dissipation. Note that the benefits of enforcing a priori physical knowledge is not limited to AI-based methods. For instance, Baddoo et al. (Reference Baddoo, Herrmann, McKeon, Kutz and Brunton2023) revealed that standard dynamic mode decomposition can also be enriched by enforcing physical properties on the matrix $A$ employed to learn the linear mapping that advances one snapshot of the system into the next. While usually $A$ is searched as a low-rank matrix (Schmid Reference Schmid2010), Baddoo et al. (Reference Baddoo, Herrmann, McKeon, Kutz and Brunton2023) proposed to search this matrix $A$ as tri-diagonal, block matrices, unitary or upper triangular, depending on the physical property of the system (locality, multi-dimensional, conservative or causal).

Figure 2.

(a) CNN auto-encoder (AE) architecture to predict a dynamical system (Usandivaras et al. Reference Usandivaras, Bauerheim, Cuenot and Urbano2024b ), where a network $\phi _e$ encodes the snapshot $y_t$ , producing a latent code $z_t$ . This code is propagated in time by a network $\zeta$ such that $z_{t+\Delta t} = \zeta (z_t)$ . This new code is then decoded by the network $\phi _d$ , providing the high-dimensional snapshot $t_{t+\Delta t}$ of the system. (b) The AE algorithm shows excellent results compared with CFD, while classical POD fails to effectively reproduce the turbulent dynamics. (c) Sketch of the fluid–structure problem where a flexible drone flies over a gust (transverse velocity $U_{{gust}}$ ) (Colombo, Morlier & Bauerheim Reference Colombo, Morlier and Bauerheim2023). This problem is formulated through a graph, where elements of the structures are connected together. Thus, graph networks can be trained to replicate the forced fluid–structure dynamics. (d) Results show that a simple graph network (GN, blue) produces slight inaccuracy on node displacements, yielding large errors on the global displacements. This issue can be limited by hybridising graph networks and neural ODE (GDEC, red).

As an example, recent research works focused on evaluating and adapting these AI methods to predict complex dynamical systems. Usandivaras et al. (Reference Usandivaras, Bauerheim, Cuenot and Urbano2024b ) employed a spatial convolutional auto-encoder (figure 2 a) coupled with a temporal multilayer perceptron (MLP) to predict the space–time evolution of a turbulent flame in a rocket engine. The architecture resembles the work proposed by Rywik et al. (Reference Rywik, Zimmermann, Eder, Scoletta and Polifke2023) on laminar flames. The proposed method was outperforming classical techniques (figure 2 b), such as POD, in this turbulent case with fewer modes ( $20$ for AI, compared with more than $100$ modes for POD). However, generalisation beyond the time horizon of the training dataset was failing, because of a spectral collapse not yet elucidated. Novel architectures such as HNN and LNN might be an approach to solve this issue. Similarly, Colombo et al. (Reference Colombo, Morlier and Bauerheim2023) employed a neural ODE (Chen et al. Reference Chen, Rubanova, Bettencourt and Duvenaud2018) to constrain in time a graph neural network to predict the dynamical response of flexible drone subjected to gust (figure 2 c). In contrast with simpler long-term-losses (LTLs) proposed by Tompson et al. (Reference Tompson, Schlachter, Sprechmann and Perlin2017), which only learn the discretised dynamics indirectly by penalising long-term errors, neural ODE (NODE, see table 2) learns the instantaneous dynamics of the output $y(t)$ :

(4.1) \begin{align} \frac {{\rm d}y}{{\rm d}t} = f_\theta (y, t). \end{align}

NODE training is performed by differentiating directly through the time integrator, which ensures that the network learns the local dynamics rather than only trajectory statistics. As a result, long-term drift is significantly reduced, since errors remain tied to the underlying ODE structure rather than accumulating arbitrarily in time. In practice, figure 2(d) demonstrates that coupling NODE with graph networks (GDEC) yields more accurate predictions than classical graph networks (GNs) trained with a simpler long-term loss, illustrating the benefits of embedding physical structure within deep learning frameworks. Similar techniques to constrain prediction of dynamical systems in time have been proposed for generative networks by Xie et al. (Reference Xie, Franz, Chu and Thuerey2018), based on a temporal discriminator and physics-aware data augmentation.

4.1.2. Enforcing symmetries and physical constraints in neural architectures

Another emerging trend is designing AI architectures that inherently respect symmetries, such as rotational or translational invariance, which are crucial in fluid dynamics. As definitions, a transformation $g$ is a symmetry of $f$ if $f$ remains invariant under its action, i.e. $f{\kern-1pt}(g(x)) = f(x)$ . Instead, if $f$ commutes with the action, i.e. $f{\kern-1pt}(g(x))=g(f(x))$ , then $f$ is said to be equivariant to the symmetry $g$ . For instance, equivariant/invariant neural networks are designed to preserve the symmetry transformations of the data, ensuring that the model’s outputs change in a predictable way under transformations like rotations or reflections. Note that regular convolutional layers are, by construction, translational equivariant, while pooling layers are approximately translational invariants. In fact, as mentioned by Bronstein et al. (Reference Bronstein, Bruna, Cohen and Velickovic2021), most network architectures are built upon specific symmetries or invariance, as reported in table 3. Note that we could extend this list with INR equipped with Fourier features (see § 4.2.3 and figure 5 b) that are stable under translation and scale transformations, or NODE that are invariant to time reparametrisation. Similarly, previous Hamiltonian (HNN) and Lagrangian (LNN) also preserve specific structures.

Table 3.

Various well-known neural architectures, and their respective domain and underlying symmetry. The first part of the table is extracted from Bronstein et al. (Reference Bronstein, Bruna, Cohen and Velickovic2021). Note that table 1 details how aggregation functions must be adapted for each type of network presented here.

Table 3 also highlights that many symmetries are not inherently encoded in standard neural architectures. For example, rotational invariance is typically absent and has often been approximated through data augmentation, where inputs are randomly rotated so that the network learns this transformation only implicitly. More recent research, however, has focused on designing architectures that explicitly enforce rotational symmetry and, more generally, group-based symmetries, leading to models with stronger inductive biases and improved generalisation. In the pioneering work of Gens & Domingos (Reference Gens and Domingos2014), deep symmetry networks were introduced, significantly reducing the number of training examples needed compared with standard convolutional networks. These networks are designed to explicitly capture symmetries present in datasets, such as rotations in the MNIST-rot dataset (Larochelle et al. Reference Larochelle, Erhan, Courville, Bergstra and Bengio2007). To illustrate the idea, consider a simple image classification task, where the input is an image $X$ and the output is a label $Y$ . Some transformations, such as rotations or reflections, do not change the label $Y$ . Instead of learning a separate mapping for each transformed version of an image, deep symmetry networks exploit these transformations directly. They effectively ‘share’ knowledge across all transformed versions of the same image, achieving similar accuracy with far fewer training examples: for instance, only 100 samples instead of 1000 for a standard convolutional classifier on MNIST-rot (Gens & Domingos Reference Gens and Domingos2014).

In fluid mechanics, Önder & Liu (Reference Önder and Liu2023) enforced the symmetries governing the curved interface of multiphase flows into MLP, designed to replace volume of fluid (VOF) methods. A similar demonstration was achieved for level-set methods (Buhendwa, Bezgin & Adams Reference Buhendwa, Bezgin and Adams2022). Here, they simply preserve symmetry by mirroring and rotating the neural network input with regards to all symmetries, and subsequently averaging over the predictions. This is feasible only for discrete symmetries. For continuous symmetry, enforcing the symmetries of the Navier–Stokes equations by equivariant convolution operators is an emerging line of research which could help to substantially improve the generalisation accuracy of CNNs. Partly inspired by recent advances in symmetry-preserving NNs for image segmentation (Weiler & Cesa Reference Weiler and Cesa2019), Wang et al. (Reference Wang, Walters and Yu2021) incorporated Galilean, rotation and scale symmetries into CNN models through E(2)-equivariant steerable convolutions (Weiler & Cesa Reference Weiler and Cesa2019). Similarly, Siddani, Balachandar & Fang (Reference Siddani, Balachandar and Fang2021) leveraged three-dimensional steerable CNNs (Weiler et al. Reference Weiler, Geiger, Welling, Boomsma and Cohen2018) to embed rotation equivariance into a CNN model for inferring the velocity field in the steady flow around stationary particles. Recently, Otto et al. (Reference Otto, Zolman, Kutz and Brunton2023) proposed a seminal work dedicated to enforcing or discovering symmetries in machine learning. A complete and comprehensive work on symmetry-enforcing and gauge theory of convolutional networks has also been proposed by Weiler et al. (Reference Weiler, Forré, Verlinde and Welling2023). Despite the promising results of symmetry-preserving CNNs for flow inference, their design and implementation add complexity, which may be limiting their fast development and adoption (Lino et al. Reference Lino, Fotiadis, Bharath and Cantwell2023).

The unconstrained nature of neural networks can also lead to solutions that are physically inconsistent or lack generalisation. Beyond symmetries, other specific constraints can be integrated in the network’s architecture design to address these limitations. A key approach involves Lipschitz-constrained neural networks (LCNNs, see table 1), where the Lipschitz constant of the network function is bounded. LCNNs have shown promise in applications requiring stability, monotonicity and robustness, making them particularly relevant to fluid mechanics. A function $f$ is defined as K-Lipschitz continuous if there exists a Lipschitz constant $K \geqslant 0$ such that for all pairs of inputs $x_1$ and $x_2$ , $\| f(x_2)-f(x_1) \| \leqslant K \| x_2 - x_1 \|$ . This equation highlights that an LCNN ensures bounded sensitivity of the neural network to perturbations in input, which is critical for systems governed by partial differential equations. It therefore limits the divergence of predicted quantities, which thus respects the bounded energy of physical systems. It also avoids the network from being too sensitive to noisy inputs $x_2 = x_1 + \epsilon$ , since the output will be of the order of $\| \delta f \| = \| f(x_2) - f(x_1)\| \leqslant K \epsilon$ . LCNNs have been used for robust network predictions (Béthune et al. Reference Béthune, Boissin, Serrurier, Mamalet, Friedrich and Gonzalez-Sanz2022). $1$ -Lipschitz networks (i.e. the Lipschitz constant of the network is $K \leqslant 1$ ) trained with hinge-Kantorovitch–Rubinstein loss (Serrurier et al. Reference Serrurier, Mamalet, Gonzalez-Sanz, Boissin, Loubes and Barrio2021, hKR) are also well suited to learn signed distance functions (Coiffier & Béthune Reference Coiffier and Béthune2024). It is also employed for robust RL algorithms (Barbara, Wang & Machester Reference Barbara, Wang and Machester2024), as well as a necessary properties to ease theoretical proof associated with neural networks, in particular when dealing with generalisation and out-of-distribution capabilities (Bobbia & Picard Reference Bobbia and Picard2024). Note however that building LCNN with spectral normalisation or Lipschitz regularisation may increase training time, as well as reducing the network expressivity for highly nonlinear or sharp behaviours. To my knowledge, LCNN has not yet been used in fluid mechanics. I however believe that LCNN will attract some attention to ensure robustness and generalisation capabilities for deep networks in physics in the near future, for both regression and control tasks.

4.2.1. Learning across scales

Fluid flows are characterised by their inherent nonlinearity and multi-scale behaviour, particularly in high-Reynolds-number regimes where turbulence dominates. In the Navier–Stokes equations, the nonlinear term $(\boldsymbol v \boldsymbol{\cdot }\boldsymbol{\nabla }) \boldsymbol v$ governs interactions between these scales. Traditional numerical methods discretise the gradient term using numerical stencils, regardless of the specific physical phenomena or scales present. To address this, the underlying computational mesh must be carefully designed in advance to capture flow scales. Estimation of these scales a priori is thus required, mostly based on the user’s experience on the considered problem. Adaptive mesh refinement (AMR) was developed (Berger & Oliger Reference Berger and Oliger1984) as an iterative process to dynamically refine the mesh in regions where smaller scales emerge, reducing the human intervention required for the mesh improvement. However, when the required resolution is computationally prohibitive, approximations are necessary. These include turbulence models to represent unresolved scales, or artificially thickened representations of shock waves (Lodato Reference Lodato2019) or thin flames (Colin et al. Reference Colin, Ducros, Veynante and Poinsot2000), which are activated only when a manually designed sensor indicates they are needed. Traditionally, such sensors rely on classical operators (e.g. gradient, divergence, curl) to detect relevant physics and generate metrics for model adjustment. However, these approaches often require complex, ad hoc formulations to ensure well-posedness, generalisation and efficiency.

Here, AI offers a compelling alternative by leveraging flow topology directly. Convolutional neural networks (CNNs), for instance, excel at detecting and classifying flow structures, even on unresolved or coarse grids. Deep networks have demonstrated significant improvements in robustness and accuracy for tasks like shock detection without user intervention (Beck et al. Reference Beck, Zeifang, Schwarz and Flad2020). Similarly, they have been employed to model unresolved structures, such as those encountered in thin turbulent flames (Xing & Lapeyre Reference Xing and Lapeyre2023). In essence, AI, drawing from advancements in computer vision, enables flow simulations to adapt to the observed flow topology rather than rigidly applying numerical stencils agnostic to the underlying phenomena. This capability is particularly advantageous for irregular and noisy flow patterns, which often appear on coarse grids. Building on this principle, Alguacil et al. (Reference Alguacil, Bauerheim, Jacob and Moreau2022) successfully trained AI models on LBM data to capture the behaviour of acoustic waves propagating through complex media. Their models exhibited good performance, especially on coarser grids where traditional numerical schemes tend to fail. Similarly, Bar-Sinai et al. (Reference Bar-Sinai, Hoyer, Hickey and Brenner2019) introduced data-driven discretization to learn optimised approximations of PDEs working on coarse grids. Drozda et al. (Reference Drozda, Mohanamuraly, Cheng, Lapeyre, Daviller, Realpe, Adler, Staffelbach and Poinsot2023) used deep learning to learn an optimised Taylor–Galerkin convection scheme by examining both the local physical field and the local mesh topology.

4.2.2. Learning multi-scale dependencies in fluid flows

Figure 3.

(a) Sketch of the experiment to evaluate empirically the receptive field (RF) of a neural network (NN). The network is randomly initialised and the central pixel of a constant image is perturbed, here in white. The size of the non-uniform zone of the output highlights the RF value. (b) This test is done on three models: (1) a network with one convolutional layer with kernel size $3 \times 3$ ; (2) a network with five sequential convolutional layers with kernel size $3 \times 3$ ; and (3) a UNet architecture containing also five convolutional layers with kernel size $3 \times 3$ , but dispatch over three branches, acting at different resolutions (N, N/2 and N/4). Convolutional layers are marked with a blue rectangle, D refers to down-sampling, U to up-sampling and + to concatenation. Output of the three models are given in panel (c), showing clearly the advantage of UNet to provide larger RF. This empirical RF value is given for the three models: 3, 11 and 26.

Despite their promise, early studies also highlight the difficulty of neural networks, such as CNN, in addressing multi-scale problems. First, because CFD simulations typically use unstructured meshes while CNNs require data on a uniform Cartesian grid, a preprocessing interpolation step is needed to convert the CFD fields into a format suitable for the CNN. This interpolation often leads to large errors in the flow containing small-scale variations, as encountered for instance at the trailing edge of aerofoils. In this regard, Catalani et al. (Reference Catalani, Costero, Bauerheim, Zampieri, Chapin, Gourdain and Baqué2023) evaluate this limitation of convolutional layers for multi-scale problems, such as transonic aerodynamics where smooth pressure variations coexist with sharp shock waves. Here, graph neural networks (GNNs) show better results, mainly because of their capability to perform convolutions directly on the CFD mesh, thus eliminating interpolation errors.

Nevertheless, CNNs operate on the principle of locality, characterised by their receptive field (RF, figure 3). The RF defines the extent of neighbouring pixels used for prediction, effectively limiting the network’s capacity to handle large-scale interactions that exceed the RF length (Cheng et al. Reference Cheng, Illarramendi, Bogopolsky, Bauerheim and Cuenot2021). Figure 3(a) presents a test to obtain empirically the RF length of a network. It consists in perturbing the central pixel (here in white) over a constant input (here in black). Since the RF only depends on the network’s architecture and not on the actual values of its weights, the network can be randomly initialised. The output will show a region of non-uniform pixels, corresponding to pixels actually influenced by the perturbed white input: the RF length is the size of this region, a kind of ‘angle of view’ of the network. It serves as typical length scale controlling the capability of the network architecture to search for interactions: two pixels in the input image separated by a length $\lambda \gt RF$ will be treated separately by the network, while if $\lambda \lt RF$ , the network will see them both simultaneously, allowing to capture possible interactions to predict the output. This notion is therefore crucial for AI in physics, but not sufficiently considered or investigated in practical applications.

A general rule is that for feedforward CNN using $C$ convolutions with kernel size $K_c \times K_c$ , the RF is theoretically

(4.2) \begin{align} RF = 1+ \sum _{c=1}^C (K_c-1). \end{align}

For example, a simple CNN with one convolutional layer (model 1) used $C=1$ and $K=3$ , which yields $RF = 1+1\times 2 = 3$ , while increasing its depth to $5$ convolutional layers (model 2) gives only $RF = 1+5\times 2 = 11$ , theoretical values well captured by the empirical test provided in figure 3(a). As a consequence, achieving a large receptive field with such a simplistic structure requires very deep networks (large $C$ ), which increases the risk of overfitting, leads to a high memory footprint and makes the model more prone to vanishing gradient issues. Consequently, while large RF is beneficial in physics to capture long-distance relationships between inputs, figure 3 and equation (4.2) demonstrate that constructing such networks using plain feedforward architectures often necessitates either extremely deep networks or large kernel sizes, which can lead to inefficiencies.

However, for CNNs, an effective alternative is to employ multi-scale architectures that use downsampled branches, obtained by the pooling operations that transform an input at resolution $N$ into a smaller resolution $N/2$ , thus discarding the smallest wavelength of the image and keeping only the largest structures. This typical architecture extends the RF without requiring excessively deep networks. A notable example is the well-known UNet architecture, depicted in figure 3(b) (model 3), which incorporates multiple branches (here $3$ ) dedicated to progressively downsampled versions of the input. This design allows UNet to effectively handle multi-scale phenomena, as illustrated in figure 4, where a comparison is provided between a feedforward CNN (MONO), a UNet CNN (MULTI) and the reference CFD solution. Note that the two networks have the same number of parameters and only differ by their architecture. This highlight the effectiveness of employing multi-scale networks in CFD. In addition, recent works (Fanaskov Reference Fanaskov2024; Gode et al. Reference Gode, Kruse, Angersbach, Köstler, Bauerheim and Rüde2024) have bridged the gap between multi-scale neural architectures, like UNet, and traditional multigrid methods, commonly used in linear solvers for instance.

Figure 4.

(a) Results obtain by a deep neural network to solve the Poisson equation embedded in an incompressible solver (Illarramendi et al. Reference Illarramendi, Bauerheim and Cuenot2020b ). Training is performed on non-viscous flows. Test results on viscous flows at various Reynolds numbers ( $Re = 100$ , $300$ and $1000$ ) are presented for a feedforward CNN (MONO), UNet architecture (MULTI) and the reference CFD solutions. Note that MONO and MULTI have the same number of weights, they only differ by their architecture, revealing the excellent performance of multi-scale networks like UNet in CFD. (b) Experiment on a test consisting in copying a value of a source node ( $\star$ ) in all other nodes of an unstructured mesh. Results are provided for a GNN with 3 layers (GNN 3) and 12 layers (GNN 12), as well as with a graph network with global attention (GNN ATT). This illustrates the locality of shallow network (GNN 3), the difficulty of training large graph networks (GNN 12) and the excellent behaviours of graph when equipped to capture long-distance relationships (GNN ATT).

Figure 5.

(a) Illustration of a FNO layer, which combines a learned linear operator mixing channels (black arrows, matrix $R$ ) in the Fourier space with a local linear map in physical space (green arrows, matrix $W$ ) acting through a skip connection. This allows to incorporate long-distance relationships between pixels, since spectral modes are specifically designed to separate and transform features at each scale. (b) INR implicitly learns the long-distance relationship by outputing directly the desired quantity $F_\theta (x, y)$ at a query point $(x, y)$ . However, training can be limited by spectral bias, which makes high frequencies more difficult to learn. To avoid this, spectral embeddings are incorporated (Sitzmann et al. Reference Sitzmann, Martel, Bergman, Lindell and Wetzstein2020). A drawback of INR is its difficulty to input field, which requires specific compression and representation techniques (latent variable $z$ ), hyper-networks ( $\phi$ ) that modulate the weight $\theta = \phi (z)$ of the main network, and meta-learning strategies.

In contrast, graph neural networks face unique challenges when attempting to extend their RF or multi-scale capabilities. Simply scaling the depth or width of GNNs is insufficient due to optimisation difficulties, such as representation oversmoothing (node features converge towards similar values) or oversquashing (long path to propagate information compresses relevant signals and reduces their influence) (Li, Han & Wu Reference Li, Han and Wu2018; Rusch, Bronstein & Mishra Reference Rusch, Bronstein and Mishra2023). Unlike CNNs, pooling-based approaches are not universally applicable in GNNs, making it difficult to adapt the effective UNet-like architectures (as model 3 in figure 3) for graph-based domains. This limitation suggests the need for more sophisticated strategies to address the multi-scale nature of fluid flows on general grids (Wu et al. Reference Wu, Jain, Wright, Mirhoseini, Gonzalez and Stoica2021). A promising solution to this problem is demonstrated in the seminal GraphCast framework (Lam et al. Reference Lam, Sanchez-Gonzalez, Willson, Wirnsberger, Fortunato, alet, Ravuri, Ewalds, Eaton-Rosen and Battaglia2023), which employs a multi-mesh message-passing strategy to overcome the limitations of traditional GNNs. By leveraging multiple resolution scales within the graph structure, GraphCast achieves superior performance in capturing multi-scale behaviours while maintaining stability and efficiency. Similarly, the classical message-passing (MP) strategy employed in graph networks can be replaced by attention mechanisms (Hussain, Zaki & Subramanian Reference Hussain, Zaki and Subramanian2022) that help capturing long-distance relationships between nodes (see attention mechanism in table 1). While MP approaches mimic numerical schemes on unstructured meshes, they strongly suffer from locality and oversmoothing. This is illustrated in figure 4(b) on a simple example consisting in copying the value of a source node ( $\star$ ) to all nodes of the unstructured mesh. While a shallow graph network (GNN 3, with 3 MP layers) exhibits a limited zone of action, highlighting its difficulty of capture long-distance relationships, deeper graph networks (GNN 12, with 12 MP layers) completely fail. This can be overcome by a graph network with global attention mechanism (GNN ATT) which learn the correlation between each nodes of the mesh, irrespective of their distance. Results in figure 4(b) shows that GNN ATT successively address the task, while other MP-based graph networks fail. Note that since message-passing is achieved on all edges of nodes, its cost scales linearly, as $\mathcal{O}(kN)$ , where $N$ is the number nodes and $k$ the number of edge per node. However, global attention requires the correlation between all nodes, thus scaling quadratically, as $\mathcal{O}(N^2)$ , which can explode for large meshes. This difficulty can be mitigated since attention mechanisms rely on dense matrix multiplications that are well-optimised and highly parallelised on GPUs. These developments highlight the importance of specialised architectural innovations to tackle the inherent multi-scale challenges in fluid dynamics, particularly when working with diverse grid structures, irregular domains and long-distance relationships between variables.

4.2.3. Overcoming scale separation with neural operators

While previous graph architectures explicitly models interactions between nodes, emerging architectures like DeepONet (Lu, Jin & Karniadakis Reference Lu, Jin and Karniadakis2019), Fourier neural operators (FNO, figure 5 a) (Li et al. Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021), wavelet neural operators (WNO) and more recently implicit neural representations (INR, figure 5 b) with Fourier features (Sitzmann et al. Reference Sitzmann, Martel, Bergman, Lindell and Wetzstein2020; Luigi et al. Reference Luigi, Cardace, Spezialetti, Ramirez, Salti and Stefano2023; Yin et al. Reference Yin, Kirchmeyer, Franceschi, Rakotomamonjy and Gallinary2023) are advancing the ability of AI models to tackle multi-scale challenges in fluid dynamics. Operator learning frameworks (see table 2) are particularly well suited for tasks involving differential operators and physics-based systems. These architectures map functions to functions, making them ideal for modelling complex phenomena where traditional methods struggle.

On one hand, FNO and related spectral approaches operate directly in the frequency domain, enabling more global and efficient representations of fluid dynamics. By leveraging Fourier transforms, FNOs extend traditional CNNs to learn operators in spectral space, effectively capturing the non-local interactions characteristic of multi-scale fluid phenomena (figure 5 a). However, FNOs face a trade-off: the number of retained Fourier modes directly impacts both computational cost and accuracy, as each mode corresponds to a specific spatial scale. While standard FNOs include a local linear map in physical space (green arrows in figure 5 a), they remain limited in their ability to represent strongly localised or spatially varying transformations. Wavelet neural operators (WNOs) address this limitation by using wavelet transforms to provide simultaneous localisation in space and scale, enabling more flexible treatment of both global and local dynamics (Tripura & Chakraborty Reference Tripura and Chakraborty2023). Recent advances have further refined these concepts. For example, Lütjens et al. (Reference Lütjens, Crawford, Watson, Hill and Newman2022) introduced multi-scale neural operators that separate scales through a filtering process similar to LES in CFD. These models achieve quasi-linear complexity, i.e. $\mathcal{O}(N \mathrm{log} N)$ .

Table 4.

Mean $L^2$ relative error between the true and predicted results using various operator learning approaches, among which Graph Neural Operator (GNO), DeepONet, Fourier Neural Operator (FNO) and Wavelet Neural Operator (WNO). Best results are highlighted in bold. Table is extracted from Tripura & Chakraborty (Reference Tripura and Chakraborty2023).

Building on these ideas, Wang et al. (Reference Wang, Li, Yuan, Peng, Liu and Wang2024) combined FNOs with UNet-like architectures, achieving improved accuracy across scales in turbulent flow simulations. This hybrid approach outperformed both classical subgrid models, such as dynamic Smagorinsky (Germano et al. Reference Germano, Piomelli, Moin and Cabot1991; Meneveau & Lund Reference Meneveau and Lund1997) and WALE (Nicoud & Ducros Reference Nicoud and Ducros1999), as well as standard AI models based on plain FNOs. Table 4 compares the performance of various operator learning techniques, such as graph neural operators (GNO), DeepONet, FNO and WNO, across diverse test cases. These include Burger’s equation, Darcy’s equation on rectangular (rect.) and triangular (tri.) domains, 2-D Navier–Stokes equations, and a 1-D time-dependent wave advection problem. Note that standard FNO (Li et al. Reference Li, Kovachki, Azizzadenesheli, Liu, Bhattacharya, Stuart and Anandkumar2021) cannot handle complex domains like the triangular test case for Darcy’s equation. Results highlight the benefits of addressing spatial scale challenges when solving flow-related PDEs, with spectral approaches like FNO and WNO consistently outperforming others.

However, implicit neural representations (INRs) provide a framework for representing individual continuous functions using neural networks (figure 5 b). In this approach, an MLP maps spatial coordinates $\xi = (x, y)$ to field values $F_\theta (\xi )$ , yielding a continuous resolution-independent representation of the solution. To handle families of flows depending on physical or geometric parameters (e.g. boundary conditions, operating points, geometry descriptors, denoted $z$ in figure 5 b), the network weights $\theta$ are generated by a hypernetwork $\phi$ conditioned on these parameters $z$ . Concretely, the hypernetwork takes $z$ as input and outputs a specific set of weights $\theta (z)$ , thereby defining a dedicated spatial function $F_{\theta (z)}(\xi )$ . Training is performed by backpropagating the loss $\mathcal{L}$ through the INR into the hypernetwork, i.e. ${\partial \mathcal{L}}/{\partial \phi } = ( {\partial \mathcal{L}}{\partial \theta })( {\partial \theta }/{\partial \phi })$ , and only the hypernetwork parameters $\phi$ are updated to learn how solution functions vary across the parameter space. This design differs from simply concatenating parameters with spatial coordinates: instead of forcing a single network to represent all solutions at once, the hypernetwork learns a structured family of solution functions, improving expressivity and generalisation across regimes. As such, this formulation enables parametrised PDE learning and high-fidelity reconstruction of flow fields.

Nevertheless, vanilla INRs suffer from spectral bias, which makes high-frequency components more difficult to learn. By leveraging Fourier features (spectral embeddings), INRs can circumvent this issue and effectively capture multi-scale structures (Grattarola & Vandergheynst Reference Grattarola and Vandergheynst2022; Catalani et al. Reference Catalani, Agarwal, Bertrand, Tost, Bauerheim and Morlier2024), making them a highly efficient alternative to traditional grid-based methods. To do so, spectral embeddings do not feed $\xi = (x, y)$ directly into the network, but instead, the input is lifted into a higher-dimensional Fourier embedding $\gamma (x) \in \mathbb{R}^{2N}$ , defined as

(4.3) \begin{align} \gamma (\xi ) = \big [ \cos (2\pi b_1^\top \xi ), \ldots , \cos (2\pi b_N^\top \xi ), \sin (2\pi b_1^\top \xi ), \ldots , \sin(2\pi b_N^\top \xi ) \big ], \end{align}

where each frequency vector $b_k \in \mathbb{R}^2$ is sampled independently from a Gaussian distribution $b_k \sim \mathcal{N}(0, \sigma ^2 I_2)$ . Consequently, each new component acts like a Fourier mode at some random frequency and, together, they form a random Fourier basis. Feeding these oscillatory inputs to the MLP gives the network the capacity to represent high-frequency structure much more effectively. To go further, Tancik et al. (Reference Tancik, Srinivasan, Mildenhall, Fridovich-Keil, Raghavan, Singhal, Ramamoorthi, Barron and Ng2020) employs the neural tangent kernel (NTK) theory, which describes how a large neural network $f_\theta$ behaves like a kernel method, with kernel $K(\xi , \xi ') = \boldsymbol{\nabla} _\theta f_\theta (\xi )^T\boldsymbol{\nabla} _\theta f_\theta (\xi ')$ for two inputs $\xi$ and $\xi '$ . They show that with Fourier features $\gamma (\xi )$ , the NTK behaves like a shift-invariant kernel $K(\xi , \xi ') \approx K(\xi -\xi ')$ . As a result, the learning dynamics becomes translation-equivariant: the network learns spatial patterns independently of their absolute location, without this symmetry being explicitly imposed. Therefore, this bridges the gap between our PoE-i on invariance and PoE-ii on scale separation. It yields that training such a network is equivalent to doing kernel regression with a well-behaved kernel, giving predictability, interpretability and generalisation guarantees.

The remaining hyper-parameter is the variance $\sigma ^2$ , which controls the characteristic spectral scale emphasised during training. Although frequencies are sampled from a broad Gaussian distribution, NTK analysis shows that the resulting kernel concentrates learning around spatial frequencies $k \sim \sigma$ , while attenuating both lower and higher frequencies. As a result, a single $\sigma$ value biases learning towards a narrow frequency band. Multi-scale Fourier encodings, obtained by combining multiple values of $\sigma$ (e.g. $\sigma = (1, 5)$ in figure 6 b), effectively superimpose kernels acting at different spectral scales, enabling faster and more balanced learning of both global and localised features. Figure 6(b) illustrates the advantage of multi-scale spectral encoding on a simple 1-D signal, where the choice $\sigma = (1, 5)$ significantly outperform single-scale encodings ( $\sigma = (1)$ or $\sigma = (5)$ ), demonstrating superior ability to capture complex features.

Finally, INRs equipped with Fourier features exhibit near-resolution invariance and enable rapid training on coarser grids without sacrificing accuracy. This property has been successfully applied to realistic aerodynamic scenarios by Catalani et al. (Reference Catalani, Agarwal, Bertrand, Tost, Bauerheim and Morlier2024), who demonstrated that multi-scale INR methods outperform classical graph neural networks (MGN) and Gaussian processes in terms of accuracy, robustness and resolution invariance on 2-D transonic aerofoils and 3-D wings (see figures 6 c and 6 d). Recent advancements in INR techniques further enhance their capabilities. For instance, local neural fields equipped with attention mechanisms, such as the AROMA architecture (Serrano et al. Reference Serrano, Wang, Naour, Vittaut and Gallinari2024), improve multi-scale predictions by focusing computational resources on critical regions. Additionally, INRs have been adapted to operate on non-Euclidean manifolds with intrinsic neural fields (Koestler et al. Reference Koestler, Grittner, Moeller, Cremers and Lähner2022), expanding their applicability to complex geometries in fluid dynamics and beyond.

Figure 6.

(a) Illustration of a multi-scale INR architecture with various embedding vectors $\gamma _\sigma$ . (b) Experimental results for fitting a single signal using a 3-layer INR with different Fourier encodings. This experiment can be reproduced from our Git repository. (c) Results obtained by CFD, graph network (MGN) and INR on 3-D wing pressure predictions, with (d) a quantitative comparison on the pressure coefficient $C_p$ predictions.

These methods are relatively recent and are not yet fully mature when applied to complex, multiscale three-dimensional fluid flows. A deeper theoretical understanding of how these innovative deep learning networks capture the nonlinear interactions of multiple scales is essential. For instance, in INRs, Fourier features are currently set manually without clear guidelines grounded in the physical scales of the flow. Developing such guidelines could significantly enhance their applicability and performance. Recent alternatives, such as the instant-NGP approach proposed by Müller et al. (Reference Müller, Evans, Schied and Keller2022), encode coordinates into several grids of different resolutions. This method offers low-storage solutions for multi-resolution highly detailed volumetric data, suggesting potential pathways for improvements. Adopting a more structured framework with explainable guidelines, similar to the well-established practices in CFD, could lead to faster training, improved robustness and broader adoption within the fluid dynamics community.

4.2.4. Capitalising on multi-fidelity data

In computational fluid dynamics, achieving high-resolution simulations is often computationally expensive. Multi-fidelity methods aim to bridge the gap between low-fidelity (LF), computationally inexpensive models, and high-fidelity (HF), accurate simulations. These methods leverage the inherent distinctions between models, often related to the scales they resolve. For example, potential flow models assume zero vorticity, effectively eliminating viscosity and thus avoiding the formation of thin boundary layers. This simplification significantly reduces computational cost by bypassing the need to resolve small-scale flow features present in realistic scenarios, in contrast with high-fidelity simulations like RANS or LES. In other words, exploiting multi-fidelity strategies is by essence exploiting the scale separation principle (PoE-ii).

The core idea behind multi-fidelity methods is to exploit the strengths of various models: leveraging LF tools for rapid exploration of the design space, while reserving HF simulations for refining regions of interest. These approaches are particularly valuable in uncertainty quantification (Peherstorfer, Willcox & Gunzburger Reference Peherstorfer, Willcox and Gunzburger2018) and optimisation problems (Thelen et al. Reference Thelen, Bryson, Stanford and Beran2022) in high-dimensional spaces, where the cost of exclusively using HF simulations would be prohibitive. Unlike multi-level methods, which rely on a strict hierarchy between models, multi-fidelity approaches are more flexible, accommodating heterogeneous models without assuming a predefined relationship. Several strategies have emerged for constructing multi-fidelity frameworks, including: (i) incorporating information from HF simulations to refine or correct LF predictions; (ii) fusing outputs by combining predictions from LF and HF models to generate an integrated solution; and (iii) selective refinement, by using HF models only in critical regions or where higher accuracy is essential. The main goal of these strategies is to ensure that the final output achieves the same level of accuracy as if only HF models were used, while significantly reducing computational cost.

AI in CFD faces challenges similar to those encountered in uncertainty quantification (UQ) and optimisation tasks, particularly the need to collect expensive data samples in high-dimensional spaces. Consequently, integrating, adapting or reimagining multi-fidelity strategies within AI becomes essential. The potential benefits are twofold: (i) efficient exploration and exploitation of the high-dimensional input space by leveraging models with varying fidelity levels; and (ii) effective handling of flow scales, with different models naturally capturing specific flow features depending on their fidelity. A compelling example of this concept is seen in multi-scale INRs previously described in § 4.2.3 and figure 6(a), which fall into the fusion category by combining outputs from multiple scales $\sigma _k$ : as illustrated in figure 6(a), a shared MLP is applied to inputs encoded with distinct Fourier features $\gamma _{\sigma _k}$ , producing scale-specific representations $H_k$ , which are then fused through a final layer to predict the target quantity $P$ .

In deep learning, multi-fidelity strategies are extensively explored in the fields of domain adaptation (Farahani et al. Reference Farahani, Voghoei, Rasheed and Arabnia2020; Liu et al. Reference Liu, Yoo, Xing, Oh, Fakhri, Kang and Woo2022b , DA) and transfer learning (Bozinovski & Fulgosi Reference Bozinovski and Fulgosi1976; Pan & Yang Reference Pan and Yang2010, TL). These approaches enable the transfer of knowledge across different fidelity levels and scales, allowing for rapid and accurate predictions while minimising the need for costly high-resolution simulations for every new problem. For instance, Liu & Wang (Reference Liu and Wang2019) introduced a physics-constrained neural network (PCNN) trained on LF data and subsequently refined using a secondary network fed with HF data to learn the discrepancies between the two fidelity levels. This framework was successfully demonstrated in a two-dimensional heat-transfer and phase transition problem. Similarly, Minisci & Vasile (Reference Minisci and Vasile2013) employed artificial neural networks to correct aerodynamic forces predicted by an LF model using HF CFD data as a reference.

Among these strategies, transfer learning is particularly promising. It involves training a model on abundant LF data and then fine-tuning it on limited HF simulations or experimental datasets, thereby enhancing generalisability and reducing computational costs. This approach has been applied in various physics-based scenarios. Usandivaras et al. (Reference Usandivaras, Bauerheim, Cuenot and Urbano2024a ) demonstrated transfer learning in turbulent combustion simulations for rocket engines (figure 7 a). By training on hundreds of under-resolved inaccurate LES and fine-tuning on a small set of highly resolved LES, the method achieved strong generalisation to unseen configurations (figure 7 c). Note that figure 7(a) clearly illustrates the inherent scale differences between LF and HF simulations. Similarly, Lyu et al. (Reference Lyu, Zhao, Gong, Kang and Yao2023) employed transfer learning with FNO to perform multi-fidelity predictions on heat transfer and 3-D aerodynamic applications.

Figure 7.

(a) Multi-fidelity configuration where fine (HF, top) and coarse (LF, bottom) LES have been performed by Usandivaras et al. (Reference Usandivaras, Bauerheim, Cuenot and Urbano2024a ). (b) A 2-D portion of the dataset, highlighting the LF and HF datasets when the recess length ( $l_r$ ) and chamber diameter ( $d_c$ ) are varied. (c) Results reveal that the LF AI model (green) is inaccurate compared with the true HF LES data (CFD, blue), while the multi-fidelity model trained on LF and fine-tuned on HF is in good agreement with HF LES data.

4.5.1. From limited data to robust models

Robustness in AI for CFD is constrained by the quality and diversity of the training data. Indeed, credibility cannot be achieved if a model is trained on biased or poorly representative datasets. This challenge is even more pronounced in CFD, where each sample often requires significant computational resources, making exhaustive coverage of the parameter space impractical. As a result, improving credibility through robust models must be directly linked to strategies that prioritise the collection of the most relevant and informative samples. An effective solution lies in the development of automatic re-sampling strategies, which iteratively select the most informative samples to maximise metrics such as space coverage or surrogate model performance. By prioritising high-value data points, these strategies significantly reduce the number of samples needed to train a model.

In fluid dynamics, this approach can guide high-fidelity simulations or experiments, focusing computational resources where they are most impactful. Such methods have been successfully applied in traditional machine learning for fluid dynamics, where re-sampling strategies have been combined with Gaussian processes (GPs) for surrogate modelling and optimisation (Roy et al. Reference Roy, Segui, Jouhaud and Gicquel2018; Campet et al. Reference Campet, Roy, Cuenot, Riber and Jouhaud2020). Another complementary approach involves low-dimensionality reduction techniques, such as principal component analysis (PCA), POD or active subspaces (Constantine, Dow & Wang Reference Constantine, Dow and Wang2014; Bauerheim et al. Reference Bauerheim, Nicoud and Poinsot2016b ). These methods identify specific directions in the high-dimensional input space, enabling targeted sampling and efficient exploration. By reducing the complexity of the input domain, they help to guide sampling efforts towards regions that are most relevant for the problem at hand, thereby improving model training while minimising computational costs. These strategies not only enhance surrogate model accuracy, but also pave the way for more effective data-driven approaches in CFD, offering a scalable path for high-dimensional and computationally intensive applications.

Figure 9.

Illustration of the active learning approach introduced by Corban, Bauerheim & Jardin (Reference Corban, Bauerheim and Jardin2023). (a) Sketch of the framework where data from CFD (i) are collected in an iterative dataset (ii) to train a deep neural network (iii) predicting the lift ( $C_L$ ), power ( $C_P$ ) and efficiency ( $\eta = C_L/C_P$ ) of a 3-D flapping wing depending on its kinematics. NSGA II reuses this network to optimise the kinematics (iv), obtaining a predicted Pareto front (magenta). To guarantee its accuracy, new data are collected along this Pareto front ( $\circ$ ) by an acquisition function $\mathcal{F}$ (v), and evaluated by CFD, providing new training data ( $\triangle$ ). The Pareto front obtained at the first (top) and third (bottom) steps of this iterative active learning are displayed (panel b), showing a good convergence. At each step, suboptimal test data are also displayed (right), showing the mismatch between the CFD ( $\bullet$ ) and network prediction (line).

Similarly, in deep learning, sampling strategies play a vital role in identifying regions of the parameter space that are uncertain or under-explored. By focusing learning efforts on these areas, models can prioritise data that has the greatest potential to improve performance. This approach, known as active learning, is particularly effective in optimising space coverage or refining solutions along a Pareto front (figure 9). A central ingredient of active learning is the acquisition function $\mathcal{F}$ , which assigns a score to each candidate sample $x$ , reflecting its expected usefulness for improving the model. New samples $x$ are then drawn from regions where $\mathcal{F}(x)$ is high, while the network parameters are trained to minimise their predictive loss. The design of the acquisition function is flexible and can integrate multiple criteria relevant to CFD (figure 9v):

(4.4) \begin{align} \mathcal{F} = \mathcal{F} \left ( \epsilon (x), \sigma (x), nov(x), \mathcal{O}(x), -\mathcal{C}(x) \right ), \end{align}

where $\epsilon (x)$ is the predicted error, $\sigma (x)$ the model uncertainty, $\mathcal{O}(x)$ determines the alignment with an optimisation objective and $\mathcal{C}(x)$ evaluates the computational cost in multi-fidelity settings (§ 4.2.4). More recent approaches have further enriched acquisition functions with notions of novelty, diversity and even intrinsic motivation ( $nov$ ), which encourage exploration of under-represented regions of the parameter space (Michel et al. Reference Michel, Cvjetko, Hamon, Oudeyer and Moulin-Frier2025).

For multi-objective optimisation problems, active learning can be implemented using straightforward techniques, such as simply resampling the predicted Pareto front: selection is only guided by the predicted objective, i.e. $\mathcal{F}(\mathcal{O})$ , to gather additional data until convergence is achieved. For example, Corban et al. (Reference Corban, Bauerheim and Jardin2023) successfully applied this approach to discover optimal wing kinematics using deep learning (figure 9 a). This method has proven effective in revealing entirely novel solutions. Testing data ( $\bullet$ in figures 9 a(iv) and 9 b), where kinematics were sampled randomly, illustrate that random datasets often fail to capture optimal configurations, especially regarding $C_L$ , the lift coefficient (figure 9 b, step 1, where maximum $C_L \approx 0.9$ ). In contrast, active learning identified iteratively kinematics capable of producing a maximum lift coefficient close to $1.4$ , significantly higher than the $C_L \approx 0.9$ found in both training and testing datasets based on random sampling (figure 9 b, step 3). This demonstrates that active learning can discover novel solutions outside the scope of the original training data.

However, a key limitation of this approach is its lack of full exploration across the input space. This underscores the need to address the exploration–exploitation dilemma inherent in active learning strategies. Recent advances in learning theory offer promising routes to tackle this challenge. For instance, Bobbia & Picard (Reference Bobbia and Picard2024) proposed mathematically explicit ratios to balance exploration (searching broadly across the parameter space) and exploitation (focusing on known high-performance regions). In addition, exploration is closely tied to the notions of novelty and diversity (Michel et al. Reference Michel, Cvjetko, Hamon, Oudeyer and Moulin-Frier2025). Instead of measuring distances directly in the input or output domains, where differences may be dominated by noise or superficial variations, recent approaches evaluate distances in feature or latent spaces. These learned representations capture the underlying physical structures and task-relevant patterns, enabling a more meaningful selection of candidate samples for exploration. Applying such strategies to realistic flow scenarios, such as those studied by Corban et al. (Reference Corban, Bauerheim and Jardin2023), could further enhance the robustness and discovery potential of active learning frameworks.

While the current implementation has shown promising results in optimisation tasks, active learning can also leverage model’s uncertainty ( $\sigma$ in (4.4)), similar to classical sampling strategies employed with Gaussian processes. However, predicting uncertainties in deep networks is far from trivial. Although AI models can achieve impressive predictive accuracy, quantifying uncertainty remains a significant challenge, particularly for high-dimensional systems like fluid flows. A recent review by Koumoutsakos (Reference Koumoutsakos2024) highlights the need for AI models not only to provide point estimates, but also to offer reliable uncertainty bounds.

4.5.2. Learning to doubt: quantifying aleatoric and epistemic uncertainties

Uncertainties in predictions typically arise from two distinct sources: (i) epistemic uncertainty, which stems from the incomplete understanding of the underlying process being modelled; and (ii) aleatoric uncertainty, which refers to the inherent randomness or noise in the data itself, arising from stochastic processes or factors that are intrinsically unpredictable. While Bayesian approaches like Gaussian processes naturally predict the probability density function of the output variable, deep networks need additional ingredients to predict similar quantities.

On one hand, aleatoric uncertainties can be incorporated by assuming that this uncertainty follows a normal distribution $p(y | x) = \mathcal{N}(\mu , \sigma ^2)$ . Mean and variance can then be estimated by a neural network outputting both quantities ( $\mu _\theta$ and $\sigma ^2_\theta$ ). To train the model, a negative log-likelihood (NLL) loss function can be employed, which for a Gaussian distribution with estimated mean and variance becomes

(4.5) \begin{align} \mathcal{L}_{\textit{NLL}}(y, \mu _\theta (x), \sigma ^2_\theta (x)) = \frac {1}{2} \log \sigma ^2_\theta (x) + \frac {(y - \mu _\theta (x))^2}{2 \sigma ^2_\theta }. \end{align}

Alternatively, a $\beta$ -NLL loss function can be employed during training, as proposed by Seitzler et al. (Reference Seitzler, Tavakoli, Antic and Martius2022), which allows for better calibration of uncertainty estimates, particularly in scenarios where the model might be overconfident, as well as out-of-distribution scenarios.

Figure 10.

(a) Illustration of classical approaches for uncertainty estimations through a variance $\mathbb{V}(y)$ : deep ensemble (DE, left) and dropout (DO, right). (b) Sketch of the zigzag (ZZ) methodology introduced by Durasov et al. (Reference Durasov, Oner, Donier, Le and Fua2024). It consists in a network $f_\theta (x, y)$ which is fed inputs $x$ , as well as by its own prediction $y$ . A two step approach, starting from $y = 0$ , provides a first answer $y^1$ , which is then fed for a second step, providing a new output $y^2$ . The distance $\| y^2 - y^1 \|$ measures the uncertainty of the model, and thus allows to discriminate between ID or OOD scenario. (c) Uncertainty estimations by deep ensemble (DE), Monte Carlo dropout (DO), zigzag (ZZ) and Gaussian process (GP) applied to the trailing edge noise dataset (Brooks, Pope & Marcolini Reference Brooks, Pope and Marcolini1989; Markelle, Longjohn & Nottingham Reference Markelle, Longjohn and Nottingham2014). It shows the mean prediction $\mu _\theta$ , as well as the aleatoric and total uncertainty (confidence intervals, CI).

On the other hand, methods such as deep ensemble (Lakshminarayanan, Pritzel & Blundell Reference Lakshminarayanan, Pritzel and Blundell2017, DE) can be employed to model epistemic uncertainty, which arises from the model’s lack of knowledge about the underlying process. This is achieved by training multiple instances of the same model with different random initialisations (figure 10 a, left). The resulting uncorrelated models can be used to estimate both the mean ( $\mathbb{E}(y)$ ) and variance ( $\mathbb{V}(y)$ ) of the predicted outputs. However, the significant drawback of this approach is the substantial computational cost due to the need for multiple training runs and inferences. An alternative strategy is to generate multiple models from a single training process (figure 10 a, right), such as using dropout techniques (Gal & Ghahramani Reference Gal and Ghahramani2016). This approach drastically reduces computational cost by leveraging the stochasticity inherent in dropout layers, which randomly deactivate neurons during training. However, while this reduces the computational burden, it also leads to highly correlated models, which can compromise the quality of the uncertainty estimates, particularly the variance predictions. Other promising alternatives include deep Bayesian networks, which explicitly model uncertainty by placing a prior distribution over the model’s parameters. However, these networks are more difficult to train effectively and are computationally demanding (Mackay Reference Mackay1995).

Recently, new indirect approaches have emerged, such as zigzag (see figure 10 b) introduced by Durasov et al. (Reference Durasov, Oner, Donier, Le and Fua2024). This method modifies a standard neural network $y = f_\theta (x)$ into a reentrant network where the output $y^{k+1} = f_\theta (x, y^k)$ is iteratively refined through multiple stages. The first iteration $y^0$ is typically initialised to $0$ . For successful training, the network must satisfy $f_\theta (x, 0) \approx f_\theta (x, y) \approx y$ for all training pairs $(x, y)$ . Recent theoretical results (Foglia et al. Reference Foglia, Bobbia, Durasov, Bauerheim, Fua, Moreau and Jardin2025) show that the difference $\| y^2 - y^1 \|$ serves as an effective proxy for total uncertainty. Compared with dropout (DO), deep ensemble (DE) or Gaussian process (GP), this method provides accurate uncertainty estimations while reducing the training and inference costs: only one training and two inferences (figure 10 c). The idea behind the method is to evaluate if the network changes its answer when fed by its own prediction, as a human would do when ‘thinking twice’. In addition, when for the same input $x$ , several outputs $y$ are available (e.g. repeated experiment, CFD with different models), aleatoric and epistemic uncertainties can be distinguished.

Nevertheless, while these methods show promise, their evaluation has mostly been limited to toy problems. More comprehensive benchmarks, particularly on real-world high-dimensional fluid dynamics problems, are needed to validate their effectiveness in practical applications. A notable example of such current evaluation is the quantification of uncertainties in auto-encoders for 2-D aerodynamic predictions (Saetta, Tognaccini & Iaccarino Reference Saetta, Tognaccini and Iaccarino2024). Another example is provided by Alguacil et al. (Reference Alguacil, Bauerheim, Moreau, Lavoie and Bolduc-Teasdale2024), where active learning is combined with UQ estimations to build sample-efficient network models of the acoustic cascade response, which involves singularities difficult to capture by random sampling.

In addition, identifying out-of-distribution (OOD) data is critical for evaluating the confidence of a model’s predictions. OOD refers to data that the model has not encountered during training. Since testing pairs rarely match exactly with those observed during training, the goal is to identify features that are either unique or significantly different from the initial training data. OOD data can be assessed indirectly by predicting uncertainties, as OOD inputs are more likely to yield uncertain predictions. In this context, zigzag can again be used, as it distinguishes between in-distribution (ID) and OOD data: for ID data, $y^2 \approx y^1$ (as shown in figure 10 b, left), whereas for OOD data, $y^2 \neq y^1$ (figure 10 b, right). Further development and evaluation of OOD detection methods are needed in future research. This is especially relevant in optimisation pipelines that employ deep neural networks, as optimal solutions are often OOD never seen during training.

4.6.1. Generative models

In contrast with surrogate networks that learn a deterministic mapping $y = f(x)$ , generative models aim instead to approximate the full conditional distribution $p(y | x)$ , from which new consistent samples $y^* \sim p(y | x)$ can be drawn. The remarkable success of generative AI in areas like natural language processing (Brown et al. Reference Brown, Larochelle, Ranzato, Hadsell, Balcan and Lin2020; Touvron et al. Reference Touvron2023) and computer vision (Yuan et al. Reference Yuan2021) has inspired growing interest in their applications to physics, particularly in fluid dynamics. Generative models, such as generative adversarial networks (GANs), variational autoencoders (VAEs) and diffusion models (DMs), offer innovative approaches (see table 2) to create high-fidelity fluid simulations by learning the underlying data distributions $p(y | x)$ . These models provide a compelling alternative to traditional simulation methods, enabling the generation of new flow fields or turbulence patterns. For instance, Xie et al. (Reference Xie, Franz, Chu and Thuerey2018) employed a temporally consistent GAN to achieve super-resolution fluid flows for movie animations. Similarly, Drygala et al. (Reference Drygala, Winhart, di Mare and Gottschalk2022) demonstrated that deep convolutional GANs (DCGANs) outperformed classical GANs and Wasserstein GANs (WGANs) in generating turbulent fields in a blade cascade configuration. Further advancements in this domain include Li et al. (Reference Li, Buzzicotti, Biferale and Bonaccorso2023), who showcased the effectiveness of CNNs and GANs in inferring velocity components in rotating turbulent flows. More recently, Shu, Li & Farimani (Reference Shu, Li and Farimani2023) used a physics-guided denoising diffusion probabilistic model (DDPM) to generate turbulence fields. Building on this foundation, Qiu et al. (Reference Qiu, Huang, Zhang, Lin, Pan, Liu and Miao2024) extended the approach to generate space–time fluid flows, demonstrating its applicability to complex scenarios such as the Von Karman vortex street and 3-D brain artery flows.

Figure 11.

(a) Results obtained by Wei et al. (Reference Wei, Dufour, Pelletier, Fua and Bauerheim2024) with a conditioned DM (CLSDM) to generate possible aerofoil geometries. (b) Visualisation of the results using a t-sne map revealed that CLSDM can generate more diverse blade better than CGAN, especially when only limited training data are available (here, 100 training aerofoils). (c) Example of 3-D turbomachinery blades and their 2-D projection generated by CLSDM, when constrained on thickness (top) and twist (bottom).

Similarly, Wei et al. (Reference Wei, Dufour, Pelletier, Fua and Bauerheim2024) explored the generation of new geometries, such as 2-D aerofoils and 3-D turbomachinery blades, using a conditional latent space diffusion model (CLSDM, figure 11). The conditioning of the model, which guides sample generation with geometrical constraints, was enforced during the denoising steps. Results demonstrated the exceptional efficiency of CLSDM in generating both diverse and novel geometries, even with a limited number of training examples (e.g. only 100 shapes for panels a and b). A comparative analysis with other techniques, such as CGANs, highlights the superiority of CLSDM. For example, t-SNE mapping (figure 11 b) showed that CLSDM generates a broader and more novel range of shapes compared with CGANs, particularly when training data are limited (Wei et al. Reference Wei, Dufour, Pelletier, Fua and Bauerheim2024). These generated shapes were particularly effective in building training datasets for surrogate modelling in shape optimisation tasks, such as predicting and optimising the lift-to-drag ratio ( $L/D$ ). Similarly, CLSDM is able to generate realistic 3-D blades (figure 11 c), with physical constraints as thickness (top) or twist distribution (bottom). These generated blades have been integrated in a simplified pre-design framework to optimise the isentropic efficiency of a turbomachinery, revealing both their good performance and their adherence to physical constraints. Thus, while diffusion models are still recent and underexplored in fluid mechanics, their potential merits deeper investigation. For example, turbulence is inherently stochastic, yet most simulation tools are deterministic. In contrast, generative models can capture the full data distribution, thereby extending beyond the classical deterministic paradigm of CFD. Future research should focus on enhancing our understanding of these methods and applying it to more realistic configurations and scenarios.

4.6.2. Foundation models

Foundation models consist in large pre-trained models that can be fine-tuned for specific tasks. They offer significant potential for applications in physics (Subramanian et al. Reference Subramanian, Harrington, Keutzer, Bhimji, Morozov, Mahoney and Gholami2023), particularly in fluid mechanics. These models provide a versatile and powerful framework for tackling a wide range of fluid dynamics problems, from aerodynamics to weather forecasting, serving as valuable tools for both researchers and engineers. Unlike traditional approaches that incorporate semantics (PE-iv) through inductive biases in deep learning networks, foundation models often rely on transformer architectures. Their key principle is to eliminate the constraints imposed by inductive biases, instead leveraging vast amounts of data to pre-train networks that reconstruct partially masked inputs (in-painting). This enables the multi-head attention mechanism of transformers to learn contextualised, meaningful embeddings relevant to the data. However, as Jakubik et al. (Reference Jakubik2023) cautioned, ‘it would be unwise to anticipate superior performance (of foundation models) across all application categories compared with state-of-the-art models’.

Nonetheless, foundation models show clear advantages in scenarios with limited labelled data, a strength also shared with other pre-trained and fine-tuning strategies discussed in § 4.2.4. Achieving the full potential of foundation models presents new challenges beyond our pillars of effectiveness, particularly in providing easy and fast access to large and diverse datasets. Initiatives like the Johns Hopkins Turbulence Database (JHTDB), BlastNet and, more recently, The Well (Ohana et al. Reference Ohana2024), aim to facilitate access to datasets relevant for physics and fluid mechanics. Additionally, this effort necessitates the harmonisation of database formats, variables and structures, as well as the development of open-source pre-processing tools to efficiently manipulate these datasets. First demonstrations of foundation models have been realised recently by Liu et al. (Reference Liu, Sun, He, Pinney, Zhang and Schaeffer2024), Tierz et al. (Reference Tierz, Iparraguirre, Alfaro, Gonzalez, Chinesta and Cueto2024) and finally Feng et al. (Reference Feng, Zheng, Hu, Liu, Yu, Wei, Feng, Duan, Fan and Wu2025), who proposed FluidZero, a pre-trained generative model able to address diverse tasks such as predictions, shape optimisation, parameter identification or control problems. This shows the potential of versatile foundation models in fluid mechanics, opening possibly new path to solve fluid problems in the near future.

4.6.3. Rethinking effectiveness for generative and foundation models

It is important to emphasise that generative and foundation models do not necessarily incorporate the four aspects of mathematical effectiveness discussed in this article. Indeed, they both represent a distinct paradigm within AI that differs significantly from traditional physics-inspired modelling and CFD. This raises the question of whether the four pillars of effectiveness can also be applied to such models. In principle, certain pillars already find partial implementation. For instance, scale separation is central to the architecture of UNets, which are widely used in diffusion models or foundation models. Similarly, symmetries can be enforced in generative frameworks through equivariant architectures or data augmentation (Xie et al. Reference Xie, Franz, Chu and Thuerey2018). However, other pillars appear to contradict the foundational philosophy of these models. Sparsity, for example, is opposed to the overparametrisation that underlies their success and semantics is never explicitly encoded, but rather should emerge implicitly, if at all, from large-scale training. This suggests that while generative and foundation models can accommodate some of these principles, their integration is selective and often indirect, highlighting a fundamental difference between models designed around effectiveness and explicability, and those designed around universality and scale.

Consequently, this raises a critical question: are these approaches suitable for physics? Relying on such vast datasets, which are often challenging to harmonise, store, validate and share, may not always be practical, especially when compared with simpler multi-fidelity pipelines tailored for specific tasks. For example, while foundation models in fields like natural language processing are justified by the requirement to capture long dependencies and intricate contexts, the case for their use in physics is less clear. Physical phenomena are often characterised by spatial or temporal locality, and governed by well-defined equations that provide inherent structure and constraints, such as conservation laws and boundary conditions. In addition, training foundation models requires immense computational resources and energy, which may not always justify the gains in accuracy or flexibility, particularly when simpler, task-specific models can achieve comparable performance. Note however that this conclusion might be different with experimental data subject to noise and experimental biases.

Nevertheless, if generative and foundation models indeed represent a paradigm shift in CFD, this suggests that the present framework of epistemological pillars for effectiveness may need to be revisited in the near future. In particular, pillars derived from mathematics in the context of physics might not fully capture the requirements of these new approaches. For these models, statistics for example, an aspect of mathematics not scrutinised in this work, or alternative conceptual references, could provide a more suitable foundation for defining effectiveness. As an illustration, while mathematical logic in this work has motivated the principle of rules with semantics and explainability, future perspectives might reconsider this principle through the lens of causality going beyond mere correlations, which is more aligned with the statistical paradigm behind generative models. Exploring such alternative formulations constitutes an important direction for future research, opening the way towards new epistemological pillars adapted to these specific AI paradigms that emphasise universality, scale and stochasticity.