No CrossRef data available.
Article contents
Dimensional homogeneity constrained gene expression programming for discovering governing equations
Published online by Cambridge University Press: 18 April 2024
Abstract
Data-driven discovery of governing equations is of great significance for helping us understand intrinsic mechanisms and build physical models. Recently, numerous highly innovative algorithms have emerged, aimed at inversely discovering the underlying governing equations from data, such as sparse regression-based methods and symbolic regression-based methods. Along this direction, a novel dimensional homogeneity constrained gene expression programming (DHC-GEP) method is proposed in this work. The DHC-GEP simultaneously discovers the forms and coefficients of functions using basic mathematical operators and physical variables, without requiring preassumed candidate functions. The constraint of dimensional homogeneity is capable of filtering out the overfitting equations effectively. The key advantages of DHC-GEP compared with the original gene expression programming, including being more robust to hyperparameters, the noise level and the size of datasets, are demonstrated on two benchmark studies. Furthermore, DHC-GEP is employed to discover the unknown constitutive relations of two representative non-equilibrium flows. Galilean invariance and the second law of thermodynamics are imposed as constraints to enhance the reliability of the discovered constitutive relations. Comparisons, both quantitative and qualitative, indicate that the derived constitutive relations are more accurate than the conventional Burnett equations in a wide range of Knudsen numbers and Mach numbers, and are also applicable to the cases beyond the parameter space of the training data.
JFM classification
- Type
- JFM Papers
- Information
- Copyright
- © The Author(s), 2024. Published by Cambridge University Press
References
Alves, E.P. & Fiuza, F. 2022 Data-driven discovery of reduced plasma physics models from fully kinetic simulations. Phys. Rev. Res. 4 (3), 033192.CrossRefGoogle Scholar
Baydin, A.G., Pearlmutter, B.A., Radul, A.A. & Siskind, J.M. 2018 Automatic differentiation in machine learning: a survey. J. Machine Learning Res. 18, 1–43.Google Scholar
Beetham, S. & Capecelatro, J. 2020 Formulating turbulence closures using sparse regression with embedded form invariance. Phys. Rev. Fluids 5 (8), 084611.CrossRefGoogle Scholar
Bergen, K.J., Johnson, P.A., de Hoop, M.V. & Beroza, G.C. 2019 Machine learning for data-driven discovery in solid earth geoscience. Science 363 (6433), eaau0323.CrossRefGoogle ScholarPubMed
Bird, G.A. 1981 Monte-carlo simulation in an engineering context. Prog. Astronaut. Aeronaut. 74, 239–255.Google Scholar
Bird, G.A. 1994 Molecular Gas Dynamics and the Direct Simulation of Gas Flows. Clarendon.CrossRefGoogle Scholar
Bobylev, A.V. 1982 The Chapman-Enskog and grad methods for solving the Boltzmann equation. Akad. Nauk SSSR Dokl. 262 (1), 71–75.Google Scholar
Bongard, J. & Lipson, H. 2007 Automated reverse engineering of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 104 (24), 9943–9948.CrossRefGoogle ScholarPubMed
Boyd, I.D., Chen, G. & Candler, G.V. 1995 Predicting failure of the continuum fluid equations in transitional hypersonic flows. Phys. Fluids 7 (1), 210–219.CrossRefGoogle Scholar
Brunton, S.L., Noack, B.R. & Koumoutsakos, P. 2020 Machine learning for fluid mechanics. Annu. Rev. Fluid Mech. 52, 477–508.CrossRefGoogle Scholar
Brunton, S.L., Proctor, J.L. & Kutz, J.N. 2016 Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113 (15), 3932–3937.CrossRefGoogle ScholarPubMed
Burnett, D. 1936 The distribution of molecular velocities and the mean motion in a non-uniform gas. Proc. Lond. Math. Soc. 2 (1), 382–435.CrossRefGoogle Scholar
Byrd, R.H., Lu, P., Nocedal, J. & Zhu, C. 1995 A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16 (5), 1190–1208.CrossRefGoogle Scholar
Callaham, J.L., Koch, J.V., Brunton, B.W., Kutz, J.N. & Brunton, S.L. 2021 Learning dominant physical processes with data-driven balance models. Nat. Commun. 12 (1), 1–10.CrossRefGoogle ScholarPubMed
Champion, K., Lusch, B., Kutz, J.N. & Brunton, S.L. 2019 a Data-driven discovery of coordinates and governing equations. Proc. Natl Acad. Sci. USA 116 (45), 22445–22451.CrossRefGoogle ScholarPubMed
Champion, K.P., Brunton, S.L. & Kutz, J.N. 2019 b Discovery of nonlinear multiscale systems: sampling strategies and embeddings. SIAM J. Appl. Dyn. Syst. 18 (1), 312–333.CrossRefGoogle Scholar
Chapman, S. & Cowling, T.G. 1990 The Mathematical Theory of Non-Uniform Gases: An Account of the Kinetic Theory of Viscosity, Thermal Conduction and Diffusion in Gases. Cambridge University Press.Google Scholar
Comeaux, K. 1995 An Evaluation of the Second Order Constitutive Relations for Rarefied Gas Dynamics based on the Second Law of Thermodynamics. Stanford University.Google Scholar
Economon, T.D., Palacios, F., Copeland, S.R., Lukaczyk, T.W. & Alonso, J.J. 2016 Su2: an open-source suite for multiphysics simulation and design. AIAA J. 54 (3), 828–846.CrossRefGoogle Scholar
Ferreira, C. 2001 Gene expression programming: a new adaptive algorithm for solving problems. arXiv:cs/0102027.Google Scholar
Ferreira, C. 2006 Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence. Springer.CrossRefGoogle Scholar
Gallis, M.A., Bitter, N.P., Koehler, T.P., Torczynski, J.R., Plimpton, S.J. & Papadakis, G. 2017 Molecular-level simulations of turbulence and its decay. Phys. Rev. Lett. 118 (6), 064501.CrossRefGoogle ScholarPubMed
Guastoni, L., Güemes, A., Ianiro, A., Discetti, S., Schlatter, P., Azizpour, H. & Vinuesa, R. 2021 Convolutional-network models to predict wall-bounded turbulence from wall quantities. J. Fluid Mech. 928, A27.CrossRefGoogle Scholar
Gurevich, D.R., Reinbold, P.A.K. & Grigoriev, R.O. 2019 Robust and optimal sparse regression for nonlinear pde models. Chaos 29 (10), 103113.CrossRefGoogle ScholarPubMed
Gurevich, D.R., Reinbold, P.A.K. & Grigoriev, R.O. 2021 Learning fluid physics from highly turbulent data using sparse physics-informed discovery of empirical relations (spider). arXiv:2105.00048.Google Scholar
Hadjiconstantinou, N.G. 2000 Analysis of discretization in the direct simulation Monte Carlo. Phys. Fluids 12 (10), 2634–2638.CrossRefGoogle Scholar
Han, G.-F., Liu, X.-L., Huang, J., Nawnit, K. & Sun, L. 2020 Alternative constitutive relation for momentum transport of extended Navier–Stokes equations. Chin. Phys. B 29 (12), 124701.CrossRefGoogle Scholar
Han, J., Ma, C., Ma, Z. & E, W. 2019 Uniformly accurate machine learning-based hydrodynamic models for kinetic equations. Proc. Natl Acad. Sci. USA 116 (44), 21983–21991.CrossRefGoogle ScholarPubMed
Heinbockel, J.H. 2001 Introduction to Tensor Calculus and Continuum Mechanics, vol. 52. Trafford.Google Scholar
Hornik, K., Stinchcombe, M. & White, H. 1989 Multilayer feedforward networks are universal approximators. Neural Netw. 2 (5), 359–366.CrossRefGoogle Scholar
Huang, J., Ma, Z., Zhou, Y. & Yong, W.-A. 2021 Learning thermodynamically stable and galilean invariant partial differential equations for non-equilibrium flows. J. Non-Equilib. Thermodyn. 46 (4), 355–370.CrossRefGoogle Scholar
Huber, P.J. 1992 Robust estimation of a location parameter. In Breakthroughs in Statistics: Methodology and Distribution (ed. S. Kotz & N.L. Johnson), pp. 492–518. Springer.CrossRefGoogle Scholar
Juniper, M.P. 2023 Machine learning for thermoacoustics. In Machine Learning and its Application to Reacting Flows: ML and Combustion (ed. N. Swaminathan & A. Parente), pp. 307–337. Springer.CrossRefGoogle Scholar
Kaheman, K., Kutz, J.N. & Brunton, S.L. 2020 Sindy-pi: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics. Proc. R. Soc. A 476 (2242), 20200279.CrossRefGoogle ScholarPubMed
Kaiser, E., Kutz, J.N. & Brunton, S.L. 2018 Sparse identification of nonlinear dynamics for model predictive control in the low-data limit. Proc. R. Soc. A 474 (2219), 20180335.CrossRefGoogle ScholarPubMed
Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S. & Yang, L. 2021 Physics-informed machine learning. Nat. Rev. Phys. 3 (6), 422–440.CrossRefGoogle Scholar
Keijzer, M. 2003 Improving symbolic regression with interval arithmetic and linear scaling. In Genetic Programming: 6th European Conference, EuroGP 2003 Essex (ed. C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli & E. Costa), pp. 70–82. Springer.CrossRefGoogle Scholar
Kingma, D.P. & Ba, J. 2014 Adam: a method for stochastic optimization. arXiv:1412.6980.Google Scholar
Koch-Janusz, M. & Ringel, Z. 2018 Mutual information, neural networks and the renormalization group. Nat. Phys. 14 (6), 578–582.CrossRefGoogle Scholar
Li, Z., Dong, B. & Wang, Y. 2021 Learning invariance preserving moment closure model for Boltzmann-bgk equation. arXiv:2110.03682.Google Scholar
Ling, J., Kurzawski, A. & Templeton, J. 2016 Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155–166.CrossRefGoogle Scholar
Loiseau, J.-C. & Brunton, S.L. 2018 Constrained sparse galerkin regression. J. Fluid Mech. 838, 42–67.CrossRefGoogle Scholar
Loiseau, J.-C., Noack, B.R. & Brunton, S.L. 2018 Sparse reduced-order modelling: sensor-based dynamics to full-state estimation. J. Fluid Mech. 844, 459–490.CrossRefGoogle Scholar
Long, Z., Lu, Y. & Dong, B. 2019 Pde-net 2.0: learning pdes from data with a numeric-symbolic hybrid deep network. J. Comput. Phys. 399, 108925.CrossRefGoogle Scholar
Long, Z., Lu, Y., Ma, X. & Dong, B. 2018 PDE-net: learning PDEs from data. In Proceedings of the 35th International Conference on Machine Learning (ed. J. Dy & A. Krause), 80, pp. 3208–3216. PMLR.Google Scholar
Lusch, B., Kutz, J.N. & Brunton, S.L. 2018 Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9 (1), 1–10.CrossRefGoogle ScholarPubMed
Ma, W., Zhang, J. & Yu, J. 2021 Non-intrusive reduced order modeling for flowfield reconstruction based on residual neural network. Acta Astronaut. 183, 346–362.CrossRefGoogle Scholar
Mangan, N.M., Brunton, S.L., Proctor, J.L. & Kutz, J.N. 2016 Inferring biological networks by sparse identification of nonlinear dynamics. IEEE Trans. Mol. Biol. Multi-Scale Commun. 2 (1), 52–63.CrossRefGoogle Scholar
Oran, E.S., Oh, C.K. & Cybyk, B.Z. 1998 Direct simulation Monte Carlo: recent advances and applications. Annu. Rev. Fluid Mech. 30, 403.CrossRefGoogle Scholar
Park, J. & Choi, H. 2021 Toward neural-network-based large eddy simulation: application to turbulent channel flow. J. Fluid Mech. 914, A16.CrossRefGoogle Scholar
Raissi, M. 2018 Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Machine Learning Res. 19 (1), 932–955.Google Scholar
Reinbold, P.A.K. & Grigoriev, R.O. 2019 Data-driven discovery of partial differential equation models with latent variables. Phys. Rev. E 100 (2), 022219.CrossRefGoogle ScholarPubMed
Reinbold, P.A.K., Gurevich, D.R. & Grigoriev, R.O. 2020 Using noisy or incomplete data to discover models of spatiotemporal dynamics. Phys. Rev. E 101 (1), 010203.CrossRefGoogle ScholarPubMed
Reinbold, P.A.K., Kageorge, L.M., Schatz, M.F. & Grigoriev, R.O. 2021 Robust learning from noisy, incomplete, high-dimensional experimental data via physically constrained symbolic regression. Nat. Commun. 12 (1), 1–8.CrossRefGoogle ScholarPubMed
Rudy, S.H., Brunton, S.L., Proctor, J.L. & Kutz, J.N. 2017 Data-driven discovery of partial differential equations. Sci. Adv. 3 (4), e1602614.CrossRefGoogle ScholarPubMed
Sahoo, S., Lampert, C. & Martius, G. 2018 Learning equations for extrapolation and control. In Proceedings of the 35th International Conference on Machine Learning (ed. J. Dy & A. Krause), 80, pp. 4442–4450. PMLR.Google Scholar
Schaeffer, H. 2017 Learning partial differential equations via data discovery and sparse optimization. Proc. R. Soc. A 473 (2197), 20160446.CrossRefGoogle ScholarPubMed
Schmid, P.J., Li, L., Juniper, M.P. & Pust, O. 2011 Applications of the dynamic mode decomposition. Theor. Comput. Fluid Dyn. 25 (1), 249–259.CrossRefGoogle Scholar
Schmidt, M. & Lipson, H. 2009 Distilling free-form natural laws from experimental data. Science 324 (5923), 81–85.CrossRefGoogle ScholarPubMed
Sengupta, U., Amos, M., Hosking, S., Rasmussen, C.E., Juniper, M. & Young, P. 2020 Ensembling geophysical models with Bayesian neural networks. Adv. Neural Inf. Process. Syst. 33, 1205–1217.Google Scholar
Shavaliyev, M.S. 1993 Super-Burnett corrections to the stress tensor and the heat flux in a gas of Maxwellian molecules. J. Appl. Math. Mech. 57 (3), 573–576.CrossRefGoogle Scholar
Singh, N., Jadhav, R.S. & Agrawal, A. 2017 Derivation of stable Burnett equations for rarefied gas flows. Phys. Rev. E 96 (1), 013106.CrossRefGoogle ScholarPubMed
Stefanov, S., Roussinov, V. & Cercignani, C. 2002 Rayleigh–Bénard flow of a rarefied gas and its attractors. I. Convection regime. Phys. Fluids 14 (7), 2255–2269.CrossRefGoogle Scholar
Struchtrup, H. & Torrilhon, M. 2003 Regularization of grad's 13 moment equations: derivation and linear analysis. Phys. Fluids 15 (9), 2668–2680.CrossRefGoogle Scholar
Sun, Q. & Boyd, I.D. 2002 A direct simulation method for subsonic, microscale gas flows. J. Comput. Phys. 179 (2), 400–425.CrossRefGoogle Scholar
Torrilhon, M. & Struchtrup, H. 2004 Regularized 13-moment equations: shock structure calculations and comparison to Burnett models. J. Fluid Mech. 513, 171–198.CrossRefGoogle Scholar
Vaddireddy, H., Rasheed, A., Staples, A.E. & San, O. 2020 Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data. Phys. Fluids 32 (1), 015113.CrossRefGoogle Scholar
Wagner, W. 1992 A convergence proof for bird's direct simulation monte carlo method for the Boltzmann equation. J. Stat. Phys. 66 (3), 1011–1044.CrossRefGoogle Scholar
Weatheritt, J. & Sandberg, R. 2016 A novel evolutionary algorithm applied to algebraic modifications of the rans stress–strain relationship. J. Comput. Phys. 325, 22–37.CrossRefGoogle Scholar
Weinan, E. 2021 The dawning of a new era in applied mathematics. Not. Am. Math. Soc. 68 (4), 565–571.Google Scholar
Xing, H., Zhang, J., Ma, W. & Wen, D. 2022 Using gene expression programming to discover macroscopic governing equations hidden in the data of molecular simulations. Phys. Fluids 34 (5), 057109.CrossRefGoogle Scholar
Xu, H., Chang, H. & Zhang, D. 2019 Dl-pde: deep-learning based data-driven discovery of partial differential equations from discrete and noisy data. arXiv:1908.04463.Google Scholar
Yu, C., Yuan, Z., Qi, H., Wang, J., Li, X. & Chen, S. 2022 Kinetic-energy-flux-constrained model using an artificial neural network for large-eddy simulation of compressible wall-bounded turbulence. J. Fluid Mech. 932, A23.CrossRefGoogle Scholar
Zhang, J., Fan, J. & Fei, F. 2010 Effects of convection and solid wall on the diffusion in microscale convection flows. Phys. Fluids 22 (12), 122005.CrossRefGoogle Scholar
Zhang, J. & Ma, W. 2020 Data-driven discovery of governing equations for fluid dynamics based on molecular simulation. J. Fluid Mech. 892, A5.CrossRefGoogle Scholar
Zheng, P., Askham, T., Brunton, S.L., Kutz, J.N. & Aravkin, A.Y. 2018 A unified framework for sparse relaxed regularized regression: Sr3. IEEE Access 7, 1404–1423.CrossRefGoogle Scholar
Zhong, X., MacCormack, R.W. & Chapman, D.R. 1993 Stabilization of the Burnett equations and application to hypersonicflows. AIAA J. 31 (6), 1036–1043.CrossRefGoogle Scholar