Hostname: page-component-f554764f5-nt87m Total loading time: 0 Render date: 2025-04-22T05:19:19.566Z Has data issue: false hasContentIssue false
  • English
  • Français

Filtered partial differential equations: a robust surrogate constraint in physics-informed deep learning framework

Published online by Cambridge University Press:  15 November 2024

Dashan Zhang Open the ORCID record for Dashan Zhang [Opens in a new window] ,
Yuntian Chen Open the ORCID record for Yuntian Chen [Opens in a new window]  and
Shiyi Chen
Dashan Zhang
Affiliation:
Department of Mechanics and Aerospace Engineering, Southern University of Science and Technology, Shenzhen 518055, PR China
Yuntian Chen*
Affiliation:
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo 315200, PR China Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology, Ningbo 315200, PR China
Shiyi Chen*
Affiliation:
Department of Mechanics and Aerospace Engineering, Southern University of Science and Technology, Shenzhen 518055, PR China Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo 315200, PR China
*
†Email addresses for correspondence: cyt_cn@126.com, chensy@sustech.edu.cn
†Email addresses for correspondence: cyt_cn@126.com, chensy@sustech.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Embedding physical knowledge into neural network (NN) training has been a hot topic. However, when facing the complex real world, most of the existing methods still strongly rely on the quantity and quality of observation data. Furthermore, the NNs often struggle to converge when the solution to the real equation is very complex. Inspired by large eddy simulation in computational fluid dynamics, we propose an improved method based on filtering. We analysed the causes of the difficulties in physics-informed machine learning, and proposed a surrogate constraint (filtered partial differential equation, FPDE) of the original physical equations to reduce the influence of noisy and sparse observation data. In the noise and sparsity experiment, the proposed FPDE models (which are optimized by FPDE constraints) have better robustness than the conventional PDE models. Experiments demonstrate that the FPDE model can obtain the same quality solution with 100 % higher noise and 12 % quantity of observation data of the baseline. Besides, two groups of real measurement data are used to show the FPDE improvements in real cases. The final results show that the FPDE still gives more physically reasonable solutions when facing the incomplete equation problem and the extremely sparse and high-noise conditions. The proposed FPDE constraint is helpful for merging real-world experimental data into physics-informed training, and it works effectively in two real-world experiments: simulating cell movement in scratches and blood velocity in vessels.

JFM classification

Mathematical Foundations: Machine learning Mathematical Foundations: Computational methods
Type
JFM Papers
Information
Journal of Fluid Mechanics , Volume 999 , 25 November 2024 , A40
Check for updates
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Cai, S., Wang, Z., Fuest, F., Jeon, Y.J., Gray, C. & Karniadakis, G.Em. 2021 a Flow over an espresso cup: inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks. J. Fluid Mech. 915, A102.CrossRefGoogle Scholar
Cai, S., Wang, Z., Wang, S., Perdikaris, P. & Karniadakis, G.Em. 2021 b Physics-informed neural networks for heat transfer problems. Trans. ASME J. Heat Transfer 143 (6), 060801.CrossRefGoogle Scholar
Chen, F., Sondak, D., Protopapas, P., Mattheakis, M., Liu, S., Agarwal, D. & Di Giovanni, M. 2020 Neurodiffeq: a python package for solving differential equations with neural networks. J. Open Source Softw. 5 (46), 1931.CrossRefGoogle Scholar
Chen, Y., Huang, D., Zhang, D., Zeng, J., Wang, N., Zhang, H. & Yan, J. 2021 a Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method. J. Comput. Phys. 445, 110624.CrossRefGoogle Scholar
Chen, Y., Luo, Y., Liu, Q., Xu, H. & Zhang, D. 2022 Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE). Phys. Rev. Res. 4 (2), 023174.CrossRefGoogle Scholar
Chen, Y. & Zhang, D. 2022 Integration of knowledge and data in machine learning. arXiv:2202.10337.Google Scholar
Chen, Z., Liu, Y. & Sun, H. 2021 b Physics-informed learning of governing equations from scarce data. Nat. Commun. 12 (1), 1–13.Google ScholarPubMed
Cheng, C., Meng, H., Li, Y.-Z. & Zhang, G.-T. 2021 Deep learning based on pinn for solving 2 DOF vortex induced vibration of cylinder. Ocean Engng 240, 109932.CrossRefGoogle Scholar
Chilcott, M.D. & Rallison, J.M. 1988 Creeping flow of dilute polymer solutions past cylinders and spheres. J. Non-Newtonian Fluid Mech. 29, 381–432.CrossRefGoogle Scholar
Clark Di Leoni, P., Agarwal, K., Zaki, T.A., Meneveau, C. & Katz, J. 2023 Reconstructing turbulent velocity and pressure fields from under-resolved noisy particle tracks using physics-informed neural networks. Exp. Fluids 64 (5), 95.CrossRefGoogle Scholar
Cui, G.X., Xu, C.X. & Zhang, Z.S. 2004 Progress in large eddy simulation of turbulent flows. Acta Aerodyn. Sin. 22 (2), 121–129.Google Scholar
Cuomo, S., Di Cola, V.S., Giampaolo, F., Rozza, G., Raissi, M. & Piccialli, F. 2022 Scientific machine learning through physics–informed neural networks: where we are and what's next. J. Sci. Comput. 92 (3), 88.CrossRefGoogle Scholar
DebRoy, T., Mukherjee, T., Wei, H.L., Elmer, J.W. & Milewski, J.O. 2021 Metallurgy, mechanistic models and machine learning in metal printing. Nat. Rev. Mater. 6 (1), 48–68.CrossRefGoogle Scholar
Du, M., Chen, Y. & Zhang, D. 2022 AutoKE: an automatic knowledge embedding framework for scientific machine learning. IEEE Trans. Artif. Intell. 4 (6), 1564–1578.Google Scholar
Du, Y., Wang, M. & Zaki, T.A. 2023 State estimation in minimal turbulent channel flow: a comparative study of 4DVar and PINN. Intl J. Heat Fluid Flow 99, 109073.CrossRefGoogle Scholar
Duraisamy, K. 2021 Perspectives on machine learning-augmented Reynolds-averaged and large eddy simulation models of turbulence. Phys. Rev. Fluids 6 (5), 050504.CrossRefGoogle Scholar
Duraisamy, K., Iaccarino, G. & Xiao, H. 2019 Turbulence modeling in the age of data. Annu. Rev. Fluid Mech. 51, 357–377.CrossRefGoogle Scholar
Fukushima, K. & Miyake, S. 1982 Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. In Competition and Cooperation in Neural Nets, pp. 267–285. Springer.CrossRefGoogle Scholar
Glowinski, R. & Pironneau, O. 1992 Finite element methods for Navier–Stokes equations. Annu. Rev. Fluid Mech. 24 (1), 167–204.CrossRefGoogle Scholar
Goodell, J.W., Kumar, S., Lim, W.M. & Pattnaik, D. 2021 Artificial intelligence and machine learning in finance: identifying foundations, themes, and research clusters from bibliometric analysis. J. Behav. Exp. Fin. 32, 100577.CrossRefGoogle Scholar
Goodfellow, I.J., Vinyals, O. & Saxe, A.M. 2014 Qualitatively characterizing neural network optimization problems. arXiv:1412.6544.Google Scholar
Goswami, S., Anitescu, C., Chakraborty, S. & Rabczuk, T. 2020 Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theor. Appl. Fract. Mech. 106, 102447.CrossRefGoogle Scholar
Hartman, E.J., Keeler, J.D. & Kowalski, J.M. 1990 Layered neural networks with Gaussian hidden units as universal approximations. Neural Comput. 2 (2), 210–215.CrossRefGoogle Scholar
Hasanuzzaman, G., Eivazi, H., Merbold, S., Egbers, C. & Vinuesa, R. 2023 Enhancement of PIV measurements via physics-informed neural networks. Meas. Sci. Technol. 34 (4), 044002.CrossRefGoogle Scholar
Jin, W., Shah, E.T., Penington, C.J., McCue, S.W., Chopin, L.K. & Simpson, M.J. 2016 Reproducibility of scratch assays is affected by the initial degree of confluence: experiments, modelling and model selection. J. Theor. Biol. 390, 136–145.CrossRefGoogle ScholarPubMed
Kissas, G., Yang, Y., Hwuang, E., Witschey, W.R., Detre, J.A. & Perdikaris, P. 2020 Machine learning in cardiovascular flows modeling: predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks. Comput. Meth. Appl. Mech. Engng 358, 112623.CrossRefGoogle Scholar
Li, T., Buzzicotti, M., Biferale, L., Bonaccorso, F., Chen, S. & Wan, M. 2022 Data reconstruction of turbulent flows with gappy POD, extended POD and generative adversarial networks. arXiv:2210.11921.Google Scholar
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. & Anandkumar, A. 2020 Fourier neural operator for parametric partial differential equations. arXiv:2010.08895.Google Scholar
Littmann, M. et al. 2020 Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nat. Mach. Intell. 2 (1), 18–24.CrossRefGoogle Scholar
Lu, L., Meng, X., Mao, Z. & Karniadakis, G.Em. 2021 DeepXDE: a deep learning library for solving differential equations. SIAM Rev. 63 (1), 208–228.CrossRefGoogle Scholar
Maulik, R., San, O., Rasheed, A. & Vedula, P. 2019 Subgrid modelling for two-dimensional turbulence using neural networks. J. Fluid Mech. 858, 122–144.CrossRefGoogle Scholar
Misyris, G.S., Venzke, A. & Chatzivasileiadis, S. 2020 Physics-informed neural networks for power systems. In 2020 IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5. IEEE.CrossRefGoogle Scholar
Piomelli, U. 1999 Large-eddy simulation: achievements and challenges. Prog. Aerosp. Sci. 35 (4), 335–362.CrossRefGoogle Scholar
Raissi, M., Perdikaris, P. & Karniadakis, G.E. 2019 Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707.CrossRefGoogle Scholar
Raissi, M., Yazdani, A. & Karniadakis, G.Em. 2020 Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science 367 (6481), 1026–1030.CrossRefGoogle ScholarPubMed
Rao, C., Sun, H. & Liu, Y. 2020 Physics-informed deep learning for incompressible laminar flows. Theor. Appl. Mech. Lett. 10 (3), 207–212.CrossRefGoogle Scholar
Rasheed, A., San, O. & Kvamsdal, T. 2020 Digital twin: values, challenges and enablers from a modeling perspective. IEEE Access 8, 21980–22012.CrossRefGoogle Scholar
Rong, M., Zhang, D. & Wang, N. 2022 A Lagrangian dual-based theory-guided deep neural network. Complex Intell. Syst. 8 (6), 4849–4862.CrossRefGoogle Scholar
Ruder, S. 2016 An overview of gradient descent optimization algorithms. arXiv:1609.04747.Google Scholar
Rudy, S.H., Brunton, S.L., Proctor, J.L. & Kutz, J.N. 2017 Data-driven discovery of partial differential equations. Sci. Adv. 3 (4), e1602614.CrossRefGoogle ScholarPubMed
Sag, I.A., Baldwin, T., Bond, F., Copestake, A. & Flickinger, D. 2002 Multiword expressions: a pain in the neck for NLP. In International Conference on Intelligent Text Processing and Computational Linguistics, pp. 1–15. Springer.CrossRefGoogle Scholar
Saw, J.G., Yang, M.C.K. & Mo, T.C. 1984 Chebyshev inequality with estimated mean and variance. Am. Stat. 38 (2), 130–132.CrossRefGoogle Scholar
Schäfer, M., Turek, S., Durst, F., Krause, E. & Rannacher, R. 1996 Benchmark computations of laminar flow around a cylinder. In Flow Simulation with High-Performance Computers II, pp. 547–566. Springer.CrossRefGoogle Scholar
Shukla, K., Di Leoni, P.C., Blackshire, J., Sparkman, D. & Karniadakis, G.Em. 2020 Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks. J. Nondestruct. Eval. 39 (3), 1–20.CrossRefGoogle Scholar
Smagorinsky, J. 1963 General circulation experiments with the primitive equations: I. The basic experiment. Mon. Weath. Rev. 91 (3), 99–164.2.3.CO;2>CrossRefGoogle Scholar
Temam, R. 1995 Navier–Stokes Equations and Nonlinear Functional Analysis. SIAM.CrossRefGoogle Scholar
Vernon, D. 1991 Machine Vision: Automated Visual Inspection and Robot Vision. Prentice-Hall, Inc.Google Scholar
Wang, C.Y. 1991 Exact solutions of the steady-state Navier–Stokes equations. Annu. Rev. Fluid Mech. 23 (1), 159–177.CrossRefGoogle Scholar
Wang, C., Bentivegna, E., Zhou, W., Klein, L. & Elmegreen, B. 2020 a Physics-informed neural network super resolution for advection-diffusion models. arXiv:2011.02519.Google Scholar
Wang, C. & Peng, K. 2023 AI experience predicts identification with humankind. Behav. Sci. 13 (2), 89.CrossRefGoogle ScholarPubMed
Wang, H., Liu, Y. & Wang, S. 2022 a Dense velocity reconstruction from particle image velocimetry/particle tracking velocimetry using a physics-informed neural network. Phys. Fluids 34 (1), 017116.Google Scholar
Wang, N., Zhang, D., Chang, H. & Li, H. 2020 b Deep learning of subsurface flow via theory-guided neural network. J. Hydrol. 584, 124700.CrossRefGoogle Scholar
Wang, S., Teng, Y. & Perdikaris, P. 2021 Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM J. Sci. Comput. 43 (5), A3055–A3081.CrossRefGoogle Scholar
Wang, S., Yu, X. & Perdikaris, P. 2022 b When and why PINNs fail to train: a neural tangent kernel perspective. J. Comput. Phys. 449, 110768.CrossRefGoogle Scholar
Wille, R. 1960 Kármán vortex streets. Adv. Appl. Mech. 6, 273–287.CrossRefGoogle Scholar
Xu, H., Chang, H. & Zhang, D. 2019 DL-PDE: deep-learning based data-driven discovery of partial differential equations from discrete and noisy data. arXiv:1908.04463.Google Scholar
Yuan, Z., Xie, C. & Wang, J. 2020 Deconvolutional artificial neural network models for large eddy simulation of turbulence. Phys. Fluids 32 (11), 115106.CrossRefGoogle Scholar
Zhang, H., Goodfellow, I., Metaxas, D. & Odena, A. 2019 Self-attention generative adversarial networks. In International Conference on Machine Learning, pp. 7354–7363. PMLR.Google Scholar
Zhou, K., Li, J., Hong, J. & Grauer, S.J. 2023 Stochastic particle advection velocimetry (SPAV): theory, simulations, and proof-of-concept experiments. Meas. Sci. Technol. 34 (6), 065302.CrossRefGoogle Scholar