Hostname: page-component-586b7cd67f-vdxz6 Total loading time: 0 Render date: 2024-11-28T17:11:13.379Z Has data issue: false hasContentIssue false
  • English
  • Français

Learning physics-based models from data: perspectives from inverse problems and model reduction

Published online by Cambridge University Press:  04 August 2021

Omar Ghattas  and
Karen Willcox
Omar Ghattas
Affiliation:
Oden Institute for Computational Engineering & Sciences, Departments of Geological Sciences and Mechanical Engineering, The University of Texas at Austin, TX78712, USA E-mail: [email protected]
Karen Willcox
Affiliation:
Oden Institute for Computational Engineering & Sciences, Department of Aerospace Engineering & Engineering Mechanics, The University of Texas at Austin, TX78712, USA E-mail: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

This article addresses the inference of physics models from data, from the perspectives of inverse problems and model reduction. These fields develop formulations that integrate data into physics-based models while exploiting the fact that many mathematical models of natural and engineered systems exhibit an intrinsically low-dimensional solution manifold. In inverse problems, we seek to infer uncertain components of the inputs from observations of the outputs, while in model reduction we seek low-dimensional models that explicitly capture the salient features of the input–output map through approximation in a low-dimensional subspace. In both cases, the result is a predictive model that reflects data-driven learning yet deeply embeds the underlying physics, and thus can be used for design, control and decision-making, often with quantified uncertainties. We highlight recent developments in scalable and efficient algorithms for inverse problems and model reduction governed by large-scale models in the form of partial differential equations. Several illustrative applications to large-scale complex problems across different domains of science and engineering are provided.

Type
Research Article
Information
Acta Numerica , Volume 30 , May 2021 , pp. 445 - 554
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press

References

Adavani, S. S. and Biros, G. (2008), Multigrid algorithms for inverse problems with linear parabolic PDE constraints, SIAM J. Sci. Comput. 31, 369–397.CrossRefGoogle Scholar
Adavani, S. S. and Biros, G. (2010), Fast algorithms for source identification problems with elliptic PDE constraints, SIAM J. Imag. Sci. 3, 791–808.CrossRefGoogle Scholar
Akçelik, V., Bielak, J., Biros, G., Epanomeritakis, I., Fernandez, A., Ghattas, O., Kim, E. J., Lopez, J., O’Hallaron, D. R., Tu, T. and Urbanic, J. (2003a), High resolution forward and inverse earthquake modeling on terascale computers, in SC ’03: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, IEEE, p. 52.CrossRefGoogle Scholar
Akçelik, V., Biros, G. and Ghattas, O. (2002), Parallel multiscale Gauss–Newton–Krylov methods for inverse wave propagation, in SC ’02: Proceedings of the 2002 ACM/IEEE Conference on Supercomputing, IEEE, pp. 41–41.Google Scholar
Akçelik, V., Biros, G., Drăgănescu, A., Ghattas, O., Hill, J. and Bloemen Waanders, B. van (2005), Dynamic data-driven inversion for terascale simulations: Real-time identification of airborne contaminants, in SC ’05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, IEEE, p. 43.Google Scholar
Akçelik, V., Biros, G., Ghattas, O., Hill, J., Keyes, D. and van Bloemen Waanders, B. (2006), Parallel PDE-constrained optimization, in Parallel Processing for Scientific Computing (Heroux, M., Raghaven, P. and Simon, H., eds), Society for Industrial and Applied Mathematics (SIAM).Google Scholar
Akçelik, V., Biros, G., Ghattas, O., Long, K. R. and van Bloemen Waanders, B. (2003b), A variational finite element method for source inversion for convective-diffusive transport, Finite Elem. Anal. Des. 39, 683–705.CrossRefGoogle Scholar
Alexanderian, A. (2020), Optimal experimental design for Bayesian inverse problems governed by PDEs: A review. Available at arXiv:2005.12998.Google Scholar
Alexanderian, A. and Saibaba, A. K. (2018), Efficient D-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems, SIAM J. Sci. Comput. 40, A2956–A2985.CrossRefGoogle Scholar
Alexanderian, A., Gloor, P. J. and Ghattas, O. (2015), On Bayesian A- and D-optimal experimental designs in infinite dimensions, Bayesian Anal. 11, 671–695.Google Scholar
Alexanderian, A., Petra, N., Stadler, G. and Ghattas, O. (2014), A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized l 0-sparsification, SIAM J. Sci. Comput. 36, A2122–A2148.CrossRefGoogle Scholar
Alexanderian, A., Petra, N., Stadler, G. and Ghattas, O. (2016), A fast and scalable method for A-optimal design of experiments for infinite-dimensional Bayesian nonlinear inverse problems, SIAM J. Sci. Comput. 38, A243–A272.CrossRefGoogle Scholar
Alexanderian, A., Petra, N., Stadler, G. and Ghattas, O. (2017), Mean-variance risk-averse optimal control of systems governed by PDEs with random parameter fields using quadratic approximations, SIAM/ASA J. Uncertain. Quantif. 5, 1166–1192.CrossRefGoogle Scholar
Alexanderian, A., Petra, N., Stadler, G. and Sunseri, I. (2021), Optimal design of large-scale Bayesian linear inverse problems under reducible model uncertainty: Good to know what you don’t know, SIAM/ASA J. Uncertain. Quantif. 9, 163–184.CrossRefGoogle Scholar
Alger, N., Rao, V., Meyers, A., Bui-Thanh, T. and Ghattas, O. (2019), Scalable matrix-free adaptive product-convolution approximation for locally translation-invariant operators, SIAM J. Sci. Comput. 41, A2296–A2328.CrossRefGoogle Scholar
Alger, N., Villa, U., Bui-Thanh, T. and Ghattas, O. (2017), A data scalable augmented Lagrangian KKT preconditioner for large scale inverse problems, SIAM J. Sci. Comput. 39, A2365–A2393.CrossRefGoogle Scholar
Alghamdi, A., Hesse, M., Chen, J. and Ghattas, O. (2020), Bayesian poroelastic aquifer characterization from InSAR surface deformation data, I: Maximum a posteriori estimate, Water Resour . Water Resour. Res. 56, e2020WR027391.CrossRefGoogle Scholar
Alghamdi, A., Hesse, M., Chen, J., Villa, U. and Ghattas, O. (2021), Bayesian poroelastic aquifer characterization from InSAR surface deformation data, II: Quantifying the uncertainty, Water Resour. Res. Submitted.CrossRefGoogle Scholar
Allgower, E. L., Böhmer, K., Potra, F. A. and Rheinboldt, W. C. (1986), A mesh-independence principle for operator equations and their discretizations, SIAM J. Numer. Anal. 23, 160– 169.CrossRefGoogle Scholar
Ambartsumyan, I., Boukaram, W., Bui-Thanh, T., Ghattas, O., Keyes, D., Stadler, G., Turkiyyah, G. and Zampini, S. (2020), Hierarchical matrix approximations of Hessians arising in inverse problems governed by PDEs, SIAM J. Sci. Comput. 42, A3397–A3426.CrossRefGoogle Scholar
Amsallem, D. and Farhat, C. (2008), Interpolation method for the adaptation of reduced-order models to parameter changes and its application to aeroelasticity, AIAA J. 46, 1803–1813.CrossRefGoogle Scholar
Amsallem, D. and Farhat, C. (2011), An online method for interpolating linear parametric reduced-order models, SIAM J. Sci. Comput. 33, 2169–2198.CrossRefGoogle Scholar
Amsallem, D., Cortial, J., Carlberg, K. and Farhat, C. (2009), A method for interpolating on manifolds structural dynamics reduced-order models, Internat. J. Numer. Methods Engng 80, 1241–1258.CrossRefGoogle Scholar
Amsallem, D., Zahr, M. J. and Farhat, C. (2012), Nonlinear model order reduction based on local reduced-order bases, Internat. J. Numer. Methods Engng 92, 891–916.CrossRefGoogle Scholar
Antil, H., Kouri, D. P., Lacasse, M. and Ridzal, D., eds (2018), Frontiers in PDE-Constrained Optimization, Springer.CrossRefGoogle Scholar
Antoulas, A. C., Beattie, C. A. and Gugercin, S. (2020), Interpolatory Methods for Model Reduction, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Arian, E. and Ta’asan, S. (1999), Analysis of the Hessian for aerodynamic optimization: Inviscid flow, Comput . Fluids 28, 853–877.Google Scholar
Arian, E., Fahl, M. and Sachs, E. (2002), Trust-region proper orthogonal decomposition models by optimization methods, in Proceedings of the 41st IEEE Conference on Decision and Control, pp. 3300–3305.Google Scholar
Arridge, S., Maass, P., Öktem, O. and Schönlieb, C.-B. (2019), Solving inverse problems using data-driven models, in Acta Numerica, Vol. 28, Cambridge University Press, pp. 1–174.Google Scholar
Asch, M., Bocquet, M. and Nodet, M. (2016), Data Assimilation: Methods, Algorithms, and Applications, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Ascher, U. M. and Haber, E. (2003), A multigrid method for distributed parameter estimation problems, Electron . Trans. Numer. Anal. 15, 1–17.Google Scholar
Aster, R. C., Borchers, B. and Thurbur, C. H. (2013), Parameter Estimation and Inverse Problems, Academic Press.Google Scholar
Astrid, P., Weiland, S., Willcox, K. and Backx, T. (2008), Missing point estimation in models described by proper orthogonal decomposition, IEEE Trans. Automat. Control 53, 2237– 2251.CrossRefGoogle Scholar
Atkinson, K. E. (1997), The Numerical Solution of Integral Equations of the Second Kind, Cambridge University Press.CrossRefGoogle Scholar
Attia, A., Alexanderian, A. and Saibaba, A. K. (2018), Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems, Inverse Problems 34, 095009.CrossRefGoogle Scholar
Axelsson, O. and Karatson, J. (2007), Mesh independent superlinear PCG rates via compact-equivalent operators, SIAM J. Numer. Anal. 45, 1495–1516.CrossRefGoogle Scholar
Ba, Y., de Wiljes, J., Oliver, D. S. and Reich, S. (2021), Randomised maximum likelihood based posterior sampling. Available at arXiv:2101.03612.Google Scholar
Babaniyi, O., Nicholson, R., Villa, U. and Petra, N. (2021), Inferring the basal sliding coefficient field for the Stokes ice sheet model under rheological uncertainty, Cryosphere 15, 1731–1750.CrossRefGoogle Scholar
Narayanan, V. A. Badri and Zabaras, N. (2004), Stochastic inverse heat conduction using a spectral approach, Internat. J. Numer. Methods Engng 60, 1569–1593.CrossRefGoogle Scholar
Balas, E. (2005), Projection, lifting and extended formulation in integer and combinatorial optimization, Ann . Oper. Res. 140, 125.CrossRefGoogle Scholar
Banks, H. T. and Kunisch, K. (1989), Estimation Techniques for Distributed Parameter Systems, Systems & Control: Foundations & Applications, Birkhäuser.CrossRefGoogle Scholar
Bardsley, J. M. (2018), Computational Uncertainty Quantification for Inverse Problems, Society for Industrial and Applied Mathematics (SIAM).Google Scholar
Bardsley, J. M., Cui, T., Marzouk, Y. M. and Wang, Z. (2020), Scalable optimization-based sampling on function space, SIAM J. Sci. Comput. 42, A1317–A1347.CrossRefGoogle Scholar
Bardsley, J., Solonen, A., Haario, H. and Laine, M. (2014), Randomize-then-optimize: A method for sampling from posterior distributions in nonlinear inverse problems, SIAM J. Sci. Comput. 36, A1895–A1910.CrossRefGoogle Scholar
Barker, A., Rees, T. and Stoll, M. (2016), A fast solver for an H 1 regularized PDE-constrained optimization problem, Commun . Comput. Phys. 19, 143–167.CrossRefGoogle Scholar
Barrault, M., Maday, Y., Nguyen, N. C. and Patera, A. T. (2004), An ‘empirical interpola-tion’ method: Application to efficient reduced-basis discretization of partial differential equations, C. R. Math. Acad. Sci. Paris I, 339–667.Google Scholar
Bashir, O., Willcox, K., Ghattas, O., van Bloemen Waanders, B. and Hill, J. (2008), Hessian-based model reduction for large-scale systems with initial condition inputs, Internat. J. Numer. Methods Engng 73, 844–868.CrossRefGoogle Scholar
Battermann, A. and Heinkenschloss, M. (1998), Preconditioners for Karush–Kuhn–Tucker systems arising in the optimal control of distributed systems, in Optimal Control of Partial Differential Equations (Desch, W., Kappel, F. and Kunisch, K., eds), Vol. 126 of International Series of Numerical Mathematics, Birkhäuser, pp. 15–32.Google Scholar
Battermann, A. and Sachs, E. W. (2001), Block preconditioners for KKT systems in PDE-governed optimal control problems, in Fast Solution of Discretized Optimization Problems (Berlin, 2000) (Hoffmann, K.-H., Hoppe, R. H. W. and Schulz, V., eds), Vol. 138 of International Series of Numerical Mathematics, Birkhäuser, pp. 1–18.Google Scholar
Becker, R. and Vexler, B. (2007), Optimal control of the convection–diffusion equation using stabilized finite element methods, Numer. Math. 106, 349–367.CrossRefGoogle Scholar
Benner, P. and Breiten, T. (2015), Two-sided projection methods for nonlinear model order reduction, SIAM J. Sci. Comput. 37, B239–B260.CrossRefGoogle Scholar
Benner, P. and Stykel, T. (2017), Model order reduction for differential-algebraic equations: A survey, in Surveys in Differential-Algebraic Equations IV (Ilchmann, A. and Reis, T., eds), Springer, pp. 107–160.CrossRefGoogle Scholar
Benner, P., Goyal, P., Kramer, B., Peherstorfer, B. and Willcox, K. (2020), Operator inference for non-intrusive model reduction of systems with non-polynomial nonlinear terms, Comput. Methods Appl. Mech. Engng 372, 113433.CrossRefGoogle Scholar
Benner, P., Gugercin, S. and Willcox, K. (2015), A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev. 57, 483–531.CrossRefGoogle Scholar
Benner, P., Onwunta, A. and Stoll, M. (2016), Block-diagonal preconditioning for optimal control problems constrained by PDEs with uncertain inputs, SIAM J. Matrix Anal. Appl. 37, 491–518.CrossRefGoogle Scholar
Benning, M. and Burger, M. (2018), Modern regularization methods for inverse problems, in Acta Numerica, Vol. 27, Cambridge University Press, pp. 1–111.Google Scholar
Benzi, M., Haber, E. and Taralli, L. (2011), A preconditioning technique for a class of PDE-constrained optimization problems, Adv. Comput. Math. 35, 149–173.CrossRefGoogle Scholar
Berkooz, G., Holmes, P. and Lumley, J. L. (1993), The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech. 25, 539–575.CrossRefGoogle Scholar
Beskos, A., Girolami, M., Lan, S., Farrell, P. E. and Stuart, A. M. (2017), Geometric MCMC for infinite-dimensional inverse problems, J. Comput. Phys. 335, 327–351.CrossRefGoogle Scholar
Biegler, L. T., Ghattas, O., Heinkenschloss, M. and van Bloemen Waanders, B., eds (2003), Large-Scale PDE-Constrained Optimization, Vol. 30 of Lecture Notes in Computational Science and Engineering, Springer.CrossRefGoogle Scholar
Biegler, L. T., Ghattas, O., Heinkenschloss, M., Keyes, D. and van Bloemen Waanders, B., eds (2007), Real-Time PDE-Constrained Optimization, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Biros, G. and Doǧan, G. (2008), A multilevel algorithm for inverse problems with elliptic PDE constraints, Inverse Problems 24, 034010.CrossRefGoogle Scholar
Biros, G. and Ghattas, O. (1999), Parallel Newton–Krylov methods for PDE-constrained optimization, in SC ’99: Proceedings of the 1999 ACM/IEEE Conference on Supercomputing, IEEE, pp. 28–28.Google Scholar
Biros, G. and Ghattas, O. (2005a), Parallel Lagrange–Newton–Krylov–Schur methods for PDE-constrained optimization, I: The Krylov–Schur solver, SIAM J. Sci. Comput. 27, 687–713.CrossRefGoogle Scholar
Biros, G. and Ghattas, O. (2005b), Parallel Lagrange–Newton–Krylov–Schur methods for PDE-constrained optimization, II: The Lagrange–Newton solver and its application to optimal control of steady viscous flows, SIAM J. Sci. Comput. 27, 714–739.CrossRefGoogle Scholar
Borzì, A. (2003), Multigrid methods for parabolic distributed optimal control problems, J. Comput. Appl. Math. 157, 365–382.CrossRefGoogle Scholar
Borzì, A. and Griesse, R. (2005), Experiences with a space-time multigrid method for the optimal control of a chemical turbulence model, Internat. J. Numer. Methods Fluids 47, 879–885.CrossRefGoogle Scholar
Borzì, A. and Schulz, V. (2009), Multigrid methods for PDE optimization, SIAM Rev. 51, 361–395.CrossRefGoogle Scholar
Borzì, A. and Schulz, V. (2012), Computational Optimization of Systems Governed by Partial Differential Equations, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Braack, M. (2009), Optimal control in fluid mechanics by finite elements with symmetric stabilization, SIAM J. Control Optim. 48, 672–687.CrossRefGoogle Scholar
Bui-Thanh, T. and Ghattas, O. (2012a), Analysis of the Hessian for inverse scattering problems, I: Inverse shape scattering of acoustic waves, Inverse Problems 28, 055001.CrossRefGoogle Scholar
Bui-Thanh, T. and Ghattas, O. (2012b), Analysis of the Hessian for inverse scattering problems, II: Inverse medium scattering of acoustic waves, Inverse Problems 28, 055002.CrossRefGoogle Scholar
Bui-Thanh, T. and Ghattas, O. (2013), Analysis of the Hessian for inverse scattering problems, III: Inverse medium scattering of electromagnetic waves, Inverse Probl. Imaging 7, 1139–1155.CrossRefGoogle Scholar
Bui-Thanh, T. and Girolami, M. A. (2014), Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold Hamiltonian Monte Carlo, Inverse Problems 30, 114014.CrossRefGoogle Scholar
Bui-Thanh, T., Burstedde, C., Ghattas, O., Martin, J., Stadler, G. and Wilcox, L. C. (2012a), Extreme-scale UQ for Bayesian inverse problems governed by PDEs, in SC ’12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, IEEE, pp. 1–11.CrossRefGoogle Scholar
Bui-Thanh, T., Damodaran, M. and Willcox, K. (2004), Aerodynamic data reconstruction and inverse design using proper orthogonal decomposition, AIAA J. 42, 1505–1516.CrossRefGoogle Scholar
Bui-Thanh, T., Ghattas, O. and Higdon, D. (2012b), Adaptive Hessian-based nonstationary Gaussian process response surface method for probability density approximation with application to Bayesian solution of large-scale inverse problems, SIAM J. Sci. Comput. 34, A2837–A2871.CrossRefGoogle Scholar
Bui-Thanh, T., Ghattas, O., Martin, J. and Stadler, G. (2013), A computational framework for infinite-dimensional Bayesian inverse problems, I: The linearized case, with application to global seismic inversion, SIAM J. Sci. Comput. 35, A2494–A2523.CrossRefGoogle Scholar
Bui-Thanh, T., Willcox, K. and Ghattas, O. (2008a), Model reduction for large-scale systems with high-dimensional parametric input space, SIAM J. Sci. Comput. 30, 3270–3288.CrossRefGoogle Scholar
Bui-Thanh, T., Willcox, K. and Ghattas, O. (2008b), Parametric reduced-order models for probabilistic analysis of unsteady aerodynamic applications, AIAA J. 46, 2520–2529.CrossRefGoogle Scholar
Cai, X. C. and Keyes, D. E. (2002), Nonlinearly preconditioned inexact Newton algorithms, SIAM J. Sci. Comput. 24, 183–200.CrossRefGoogle Scholar
Campbell, S. L., Ipsen, I. C. F., Kelley, C. T. and Meyer, C. D. (1994), GMRES and the minimal polynomial, BIT 36, 664–675.CrossRefGoogle Scholar
Carlberg, K., Farhat, C., Cortial, J. and Amsallem, D. (2013), The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows, J. Comput. Phys. 242, 623–647.CrossRefGoogle Scholar
Chaillat, S. and Biros, G. (2012), FaIMS: A fast algorithm for the inverse medium problem with multiple frequencies and multiple sources for the scalar Helmholtz equation, J. Comput. Phys. 231, 4403–4421.CrossRefGoogle Scholar
Chaturantabut, S. and Sorensen, D. (2010), Nonlinear model reduction via discrete empirical interpolation, SIAM J. Sci. Comput. 32, 2737–2764.CrossRefGoogle Scholar
Chen, P. and Ghattas, O. (2019), Hessian-based sampling for high-dimensional model reduction, Internat. J. Uncertain. Quantif. 9, 103–121.CrossRefGoogle Scholar
Chen, P. and Ghattas, O. (2020a), Projected Stein variational gradient descent, in Advances in Neural Information Processing Systems 33 (NeurIPS 2020) (Larochelle, H. et al., eds), Vol. 33, Curran Associates, pp. 1947–1958. Available at https://proceedings.neurips.cc/paper/2020.Google Scholar
Chen, P. and Ghattas, O. (2020b), Stochastic optimization of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities. Preprint.CrossRefGoogle Scholar
Chen, P. and Ghattas, O. (2020c), Taylor approximation for chance constrained optimization problems governed by partial differential equations with high-dimensional random parameters. Available at arXiv:2011.09985.CrossRefGoogle Scholar
Chen, P. and Ghattas, O. (2021), Stein variational reduced basis Bayesian inversion, SIAM J. Sci. Comput. 43, A1163–A1193.CrossRefGoogle Scholar
Chen, P. and Schwab, C. (2015), Sparse-grid, reduced-basis Bayesian inversion, Comput . Methods Appl. Mech. Engng 297, 84–115.CrossRefGoogle Scholar
Chen, P. and Schwab, C. (2016a), Adaptive sparse grid model order reduction for fast Bayesian estimation and inversion, in Sparse Grids and Applications (Stuttgart 2014), Vol. 109 of Lecture Notes in Computational Science and Engineering, Springer, pp. 1–27.CrossRefGoogle Scholar
Chen, P. and Schwab, C. (2016b), Sparse-grid, reduced-basis Bayesian inversion: Nonaffine-parametric nonlinear equations, J. Comput. Phys. 316, 470–503.CrossRefGoogle Scholar
Chen, P., Haberman, M. and Ghattas, O. (2021), Optimal design of acoustic metamaterial cloaks under uncertainty, J. Comput. Phys. 431, 110114.CrossRefGoogle Scholar
Chen, P., Villa, U. and Ghattas, O. (2017), Hessian-based adaptive sparse quadrature for infinite-dimensional Bayesian inverse problems, Comput . Methods Appl. Mech. Engng 327, 147–172.CrossRefGoogle Scholar
Chen, P., Villa, U. and Ghattas, O. (2019a), Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty, J. Comput. Phys. 385, 163–186.CrossRefGoogle Scholar
Chen, P., Wu, K., Chen, J., O’Leary-Roseberry, T. and Ghattas, O. (2019b), Projected Stein variational Newton: A fast and scalable Bayesian inference method in high dimensions, in Advances in Neural Information Processing Systems 32 (NeurIPS 2019) (Wallach, H. et al., eds), Curran Associates. Available at https://proceedings.neurips.cc/paper/2019.Google Scholar
Collis, S. S. and Heinkenschloss, M. (2002), Analysis of the streamline upwind/Petrov Galerkin method applied to the solution of optimal control problems. Report TR02–01, Department of Computational and Applied Mathematics, Rice University, Houston, TX.Google Scholar
Colton, D. and Kress, R. (2019), Inverse Acoustic and Electromagnetic Scattering Theory, Vol. 93 of Applied Mathematical Sciences, fourth edition, Springer.CrossRefGoogle Scholar
Crestel, B., Stadler, G. and Ghattas, O. (2018), A comparative study of regularizations for joint inverse problems, Inverse Problems 35, 024003.CrossRefGoogle Scholar
Cui, T., Law, K. J. H. and Marzouk, Y. M. (2016), Dimension-independent likelihood-informed MCMC, J. Comput. Phys. 304, 109–137.CrossRefGoogle Scholar
Cui, T., Martin, J., Marzouk, Y. M., Solonen, A. and Spantini, A. (2014), Likelihood-informed dimension reduction for nonlinear inverse problems, Inverse Problems 30, 114015.CrossRefGoogle Scholar
Cui, T., Marzouk, Y. M. and Willcox, K. E. (2015), Data-driven model reduction for the Bayesian solution of inverse problems, Internat. J. Numer. Methods Engng 102, 966– 990.CrossRefGoogle Scholar
Daon, Y. and Stadler, G. (2018), Mitigating the influence of the boundary on PDE-based covariance operators, Inverse Probl. Imaging 12, 1083–1102.CrossRefGoogle Scholar
Dashti, M. and Stuart, A. M. (2017), The Bayesian approach to inverse problems, in Handbook of Uncertainty Quantification (Ghanem, R., Higdon, D. and Owhadi, H., eds), Springer, pp. 311–428.CrossRefGoogle Scholar
De los Reyes, J. C. (2015), Numerical PDE-Constrained Optimization, Springer.CrossRefGoogle Scholar
Degroote, J., Vierendeels, J. and Willcox, K. (2010), Interpolation among reduced-order matrices to obtain parameterized models for design, optimization and probabilistic analysis, Internat. J. Numer. Methods Fluids 63, 207–230.Google Scholar
Demanet, L., Létourneau, P.-D., Boumal, N., Calandra, H., Chiu, J. and Snelson, S. (2012), Matrix probing: A randomized preconditioner for the wave-equation Hessian, Appl. Comput. Harmon. Anal. 32, 155–168.CrossRefGoogle Scholar
Dembo, R. S., Eisenstat, S. C. and Steihaug, T. (1982), Inexact Newton methods, SIAM J. Numer. Anal. 19, 400–408.CrossRefGoogle Scholar
Detommaso, G., Cui, T., Marzouk, Y. M., Spantini, A. and Scheichl, R. (2018), A Stein variational Newton method, in Advances in Neural Information Processing Systems 31 (NeurIPS 2018) (Bengio, S. et al., eds), Curran Associates, pp. 9187–9197.Google Scholar
Dreyer, T., Maar, B. and Schulz, V. (2000), Multigrid optimization in applications, J. Comput. Appl. Math. 120, 67–84.CrossRefGoogle Scholar
Drmac, Z. and Gugercin, S. (2016), A new selection operator for the discrete empirical interpolation method: Improved a priori error bound and extensions, SIAM J. Sci. Comput. 38, A631–A648.CrossRefGoogle Scholar
Drohmann, M., Haasdonk, B. and Ohlberger, M. (2012), Reduced basis approximation for nonlinear parametrized evolution equations based on empirical operator interpolation, SIAM J. Sci. Comput. 34, 937–969.CrossRefGoogle Scholar
Drăgănescu, A. and Dupont, T. (2008), Optimal order multilevel preconditioners for regularized ill-posed problems, Math. Comp. 77, 2001–2038.CrossRefGoogle Scholar
Dunlop, M. M. (2019), Multiplicative noise in Bayesian inverse problems: Well-posedness and consistency of MAP estimators. Available at arXiv:1910.14632.Google Scholar
Eftang, J. L. and Stamm, B. (2012), Parameter multi-domain ‘hp’ empirical interpolation, Internat. J. Numer. Methods Engng 90, 412–428.CrossRefGoogle Scholar
Eisenstat, S. C. and Walker, H. F. (1996), Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput. 17, 16–32.CrossRefGoogle Scholar
Eldred, M., Giunta, A. and Collis, S. (2004), Second-order corrections for surrogate-based optimization with model hierarchies, in 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. AIAA paper 2004-4457.Google Scholar
Engl, H. W., Hanke, M. and Neubauer, A. (1996), Regularization of Inverse Problems, Vol. 375 of Mathematics and its Applications, Springer.CrossRefGoogle Scholar
Epanomeritakis, I., Akçelik, V., Ghattas, O. and Bielak, J. (2008), A Newton-CG method for large-scale three-dimensional elastic full-waveform seismic inversion, Inverse Problems 24, 034015.CrossRefGoogle Scholar
Everson, R. and Sirovich, L. (1995), The Karhunen–Loève procedure for gappy data, J. Optical Soc. Amer. 12, 1657–1664.CrossRefGoogle Scholar
Feng, C. and Marzouk, Y. M. (2019), A layered multiple importance sampling scheme for focused optimal Bayesian experimental design. Available at arXiv:1903.11187.Google Scholar
Flath, P. H. (2013), Hessian-based response surface approximations for uncertainty quantification in large-scale statistical inverse problems, with applications to groundwater flow. PhD thesis, The University of Texas at Austin.Google Scholar
Flath, P. H., Wilcox, L. C., Akçelik, V., Hill, J., Bloemen Waanders, B. van and Ghattas, O. (2011), Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations, SIAM J. Sci. Comput. 33, 407–432.CrossRefGoogle Scholar
Flores, J. D. (1993), The conjugate gradient method for solving Fredholm integral equations of the second kind, Int. J. Comput. Math. 48, 77–94.CrossRefGoogle Scholar
Fortuna, Z. (1979), Some convergence properties of the conjugate gradient method in Hilbert space, SIAM J. Numer. Anal. 16, 380–384.CrossRefGoogle Scholar
Galbally, D., Fidkowski, K., Willcox, K. and Ghattas, O. (2010), Nonlinear model reduction for uncertainty quantification in large-scale inverse problems, Internat. J. Numer. Methods Engng 81, 1581–1608.Google Scholar
Gerner, A.-L. and Veroy, K. (2012), Certified reduced basis methods for parametrized saddle point problems, SIAM J. Sci. Comput. 34, A2812–A2836.CrossRefGoogle Scholar
Ghanem, R. G. and Doostan, A. (2006), On the construction and analysis of stochastic models: Characterization and propagation of the errors associated with limited data, J. Comput. Phys. 217, 63–81.CrossRefGoogle Scholar
Ghanem, R. G. and Spanos, P. D. (1991), Stochastic Finite Elements: A Spectral Approach, Springer.CrossRefGoogle Scholar
Ghavamian, F., Tiso, P. and Simone, A. (2017), POD–DEIM model order reduction for strain-softening viscoplasticity, Comput . Methods Appl. Mech. Engng 317, 458–479.CrossRefGoogle Scholar
Gholami, A., Mang, A. and Biros, G. (2016), An inverse problem formulation for parameter estimation of a reaction–diffusion model of low grade gliomas, J. Math. Biol. 72, 409– 433.CrossRefGoogle ScholarPubMed
Girolami, M. and Calderhead, B. (2011), Riemann manifold Langevin and Hamiltonian Monte Carlo methods, J. R. Stat. Soc. Ser. B. Stat. Methodol. 73, 123–214.CrossRefGoogle Scholar
Giunta, A. A., Balabanov, V., Kaufman, M., Burgee, S., Grossman, B., Haftka, R. T., Mason, W. H. and Watson, L. T. (1997), Variable-complexity response surface design of an HSCT configuration, in Multidisciplinary Design Optimization: State of the Art (Alexandrov, N. M. and Hussaini, M. Y., eds), Society for Industrial and Applied Mathematics (SIAM).Google Scholar
Graham, W. R., Peraire, J. and Tang, K. Y. (1999), Optimal control of vortex shedding using low-order models, I: Open-loop model development, Internat. J. Numer. Methods Engng 44, 945–972.3.0.CO;2-F>CrossRefGoogle Scholar
Grepl, M. and Patera, A. (2005), A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations, ESAIM Math. Model. Numer. Anal. 39, 157–181.CrossRefGoogle Scholar
Griewank, A. (1992), Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation, Optim. Methods Software 1, 35–54.CrossRefGoogle Scholar
Griewank, A. (2003), A mathematical view of automatic differentiation, in Acta Numerica, Vol. 12, Cambridge University Press, pp. 321–398.Google Scholar
Griewank, A. and Walther, A. (2008), Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, second edition, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Gu, C. (2011), QLMOR: A projection-based nonlinear model order reduction approach using quadratic-linear representation of nonlinear systems, IEEE Trans. Computer-Aided Des. Integr. Circuits Syst. 30, 1307–1320.CrossRefGoogle Scholar
Gugercin, S., Antoulas, A. C. and Beattie, C. A. (2008), <mono>H</mono>2 model reduction for large-scale linear dynamical systems, SIAM J. Matrix Anal. Appl. 30, 609–638.CrossRefGoogle Scholar
Gunzburger, M. D. (2003), Perspectives in Flow Control and Optimization, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Gunzburger, M. D., Heinkenschloss, M. and Lee, H. K. (2000), Solution of elliptic partial differential equations by an optimization based domain decomposition method, Appl. Math. Comput. 113, 111–139.Google Scholar
Haasdonk, B. and Ohlberger, M. (2008), Reduced basis method for finite volume approximations of parametrized linear evolution equations, ESAIM Math. Model. Numer. Anal. 42, 277–302.CrossRefGoogle Scholar
Haasdonk, B. and Ohlberger, M. (2011), Efficient reduced models and a posteriori error estimation for parametrized dynamical systems by offline/online decomposition, Math. Comput. Model. Dyn. Syst. 17, 145–161.CrossRefGoogle Scholar
Haasdonk, B., Dihlmann, M. and Ohlberger, M. (2011), A training set and multiple bases generation approach for parameterized model reduction based on adaptive grids in parameter space, Math. Comput. Model. Dyn. Syst. 17, 423–442.CrossRefGoogle Scholar
Haber, E. and Ascher, U. (2001), Preconditioned all-at-once methods for large, sparse parameter estimation problems, Inverse Problems 17, 1847–1864.CrossRefGoogle Scholar
Hadamard, J. (1923), Lectures on the Cauchy Problem in Linear Partial Differential Equations, Yale University Press.Google Scholar
Hager, W. W. (2000), Runge–Kutta methods in optimal control and the transformed adjoint system, Numer . Math. 87, 247–282.Google Scholar
Halko, N., Martinsson, P. G. and Tropp, J. A. (2011), Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions, SIAM Rev. 53, 217–288.CrossRefGoogle Scholar
Hanke, M. (1995), Conjugate Gradient Type Methods for Ill-Posed Problems, Vol. 327 of Pitman Research Notes in Mathematics, Longman Scientific & Technical.Google Scholar
Hanke, M. (2017), A Taste of Inverse Problems: Basic Theory and Examples, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Hansen, P. C. (1998), Rank Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Harvazinski, M. E., Huang, C., Sankaran, V., Feldman, T. W., Anderson, W. E., Merkle, C. L. and Talley, D. G. (2015), Coupling between hydrodynamics, acoustics, and heat release in a self-excited unstable combustor, Phys . Fluids 27, 045102.CrossRefGoogle Scholar
Hay, A., Borggaard, J. T. and Pelletier, D. (2009), Local improvements to reduced-order models using sensitivity analysis of the proper orthogonal decomposition, J. Fluid Mech. 629, 41–72.CrossRefGoogle Scholar
Heimbach, P., Hill, C. and Giering, R. (2002), Automatic generation of efficient adjoint code for a parallel Navier–Stokes solver, in Computational Science: ICCS 2002 (Sloot, P. M. A. et al., eds), Vol. 2330 of Lecture Notes in Computer Science, Springer, pp. 1019– 1028.Google Scholar
Heinemann, R. F. and Poore, A. B. (1981), Multiplicity, stability, and oscillatory dynamics of the tubular reactor, Chem. Engng Sci. 36, 1411–1419.CrossRefGoogle Scholar
Heinkenschloss, M. (1993), Mesh independence for nonlinear least squares problems with norm constraints, SIAM J. Optim. 3, 81–117.CrossRefGoogle Scholar
Heinkenschloss, M. (2005), Time-domain decomposition iterative methods for the solution of distributed linear quadratic optimal control problems, J. Comput. Appl. Math. 173, 169–198.CrossRefGoogle Scholar
Heinkenschloss, M. and Herty, M. (2007), A spatial domain decomposition method for parabolic optimal control problems, J. Comput. Appl. Math. 201, 88–111.CrossRefGoogle Scholar
Heinkenschloss, M. and Leykekhman, D. (2010), Local error estimates for SUPG solutions of advection-dominated elliptic linear-quadratic optimal control problems, SIAM J. Numer. Anal. 47, 4607–4638.CrossRefGoogle Scholar
Heinkenschloss, M. and Nguyen, H. (2006), Neumann–Neumann domain decomposition preconditioners for linear-quadratic elliptic optimal control problems, SIAM J. Sci. Comput. 28, 1001–1028.CrossRefGoogle Scholar
Helin, T. and Kretschmann, R. (2020), Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems. Available at arXiv:2012.06603.Google Scholar
Herman, E., Alexanderian, A. and Saibaba, A. K. (2020), Randomization and reweighted <mono>ℓ</mono>1 -minimization for A-optimal design of linear inverse problems, SIAM J. Sci. Comput. 42, A1714–A1740.CrossRefGoogle Scholar
Herrmann, F. J., Moghaddam, P. and Stolk, C. C. (2008), Sparsity- and continuity-promoting seismic image recovery with curvelet frames, Appl. Comput. Harmon. Anal. 24, 150– 173.CrossRefGoogle Scholar
Herzog, R. and Sachs, E. (2010), Preconditioned conjugate gradient method for optimal control problems with control and state constraints, SIAM J. Matrix Anal. Appl. 31, 2291–2317.CrossRefGoogle Scholar
Herzog, R. and Sachs, E. (2015), Superlinear convergence of Krylov subspace methods for self-adjoint problems in Hilbert space, SIAM J. Numer. Anal. 53, 1304–1324.CrossRefGoogle Scholar
Hesse, M. and Stadler, G. (2014), Joint inversion in coupled quasistatic poroelasticity, J. Geophys. Research: Solid Earth 119, 1425–1445.CrossRefGoogle Scholar
Hinze, M. and Volkwein, S. (2008), Error estimates for abstract linear-quadratic optimal control problems using proper orthogonal decomposition, Comput . Optim. Appl. 39, 319–345.CrossRefGoogle Scholar
Hinze, M., Pinnau, R., Ulbrich, M. and Ulbrich, S. (2009), Optimization with PDE Constraints, Vol. 23 of Mathematical Modelling: Theory and Applications, Springer.Google Scholar
Hotelling, H. (1933), Analysis of a complex of statistical variables with principal components, J. Educ. Psychol. 24, 417–441, 498–520.CrossRefGoogle Scholar
Huan, X. and Marzouk, Y. M. (2013), Simulation-based optimal Bayesian experimental design for nonlinear systems, J. Comput. Phys. 232, 288–317.CrossRefGoogle Scholar
Huan, X. and Marzouk, Y. M. (2014), Gradient-based stochastic optimization methods in Bayesian experimental design, Internat. J. Uncertain. Quantif. 4, 479–510.CrossRefGoogle Scholar
Huang, C., Duraisamy, K. and Merkle, C. L. (2019), Investigations and improvement of robustness of reduced-order models of reacting flow, AIAA J. 57, 5377–5389.CrossRefGoogle Scholar
Huang, C., Xu, J., Duraisamy, K. and Merkle, C. (2018), Exploration of reduced-order models for rocket combustion applications, in AIAA Aerospace Sciences Meeting. AIAA paper 2018-1183.Google Scholar
Hughes, T. J. R., Franca, L. P. and Mallet, M. (1986), A new finite element formulation for computational fluid dynamics, I: Symmetric forms of the compressible Euler and Navier–Stokes equations and the second law of thermodynamics, Comput. Methods Appl. Mech. Engng 54, 223–234.CrossRefGoogle Scholar
Hutter, K. (1983), Theoretical Glaciology, Mathematical Approaches to Geophysics, Reidel.CrossRefGoogle Scholar
Iliescu, T. and Wang, Z. (2014), Are the snapshot difference quotients needed in the proper orthogonal decomposition?, SIAM J. Sci. Comput. 36, A1221–A1250.CrossRefGoogle Scholar
Isaac, T., Petra, N., Stadler, G. and Ghattas, O. (2015), Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet, J. Comput. Phys. 296, 348–368.CrossRefGoogle Scholar
Ito, K. and Kunisch, K. (2008), Lagrange Multiplier Approach to Variational Problems and Applications, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Jagalur-Mohan, J. and Marzouk, Y. M. (2020), Batch greedy maximization of non-submodular functions: Guarantees and applications to experimental design. Available at arXiv:2006.04554.Google Scholar
Jolliffe, I. (2005), Principal component analysis, in Encyclopedia of Statistics in Behavioral Science, Wiley.Google Scholar
Joughin, I., Alley, R. B. and Holland, D. M. (2012), Ice-sheet response to oceanic forcing, Science 338 (6111), 1172–1176.CrossRefGoogle ScholarPubMed
Kaercher, M., Boyaval, S., Grepl, M. A. and Veroy, K. (2018), Reduced basis approximation and a posteriori error bounds for 4D-Var data assimilation, Optim. Engng 19, 663–695.CrossRefGoogle Scholar
Kaipio, J. and Somersalo, E. (2005), Statistical and Computational Inverse Problems, Vol. 160 of Applied Mathematical Sciences, Springer.Google Scholar
Kalmikov, A. G. and Heimbach, P. (2014), A Hessian-based method for uncertainty quantification in global ocean state estimation, SIAM J. Sci. Comput. 36, S267–S295.CrossRefGoogle Scholar
Kaufman, M., Balabanov, V., Giunta, A. A., Grossman, B., Mason, W. H., Burgee, S. L., Haftka, R. T. and Watson, L. T. (1996), Variable-complexity response surface approximations for wing structural weight in HSCT design, Comput . Mech. 18, 112–126.Google Scholar
Kelley, C. T. (1999), Iterative Methods for Optimization, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Kelley, C. T. and Sachs, E. W. (1991), Mesh independence of Newton-like methods for infinite dimensional problems, J. Integral Equations Appl. 3, 549–573.CrossRefGoogle Scholar
Kennedy, M. C. and O’Hagan, A. (2001), Bayesian calibration of computer models, J. R. Stat. Soc. Ser. B. Stat. Methodol. 63, 425–464.CrossRefGoogle Scholar
Kerner, E. H. (1981), Universal formats for nonlinear ordinary differential systems, J. Math. Phys. 22, 1366–1371.CrossRefGoogle Scholar
Khodabakhshi, P. and Willcox, K. (2021), Non-intrusive data-driven model reduction for differential algebraic equations derived from lifting transformations. Oden Institute Report 21-08, University of Texas at Austin.CrossRefGoogle Scholar
Kirsch, A. (2011), An Introduction to the Mathematical Theory of Inverse Problems, second edition, Springer.CrossRefGoogle Scholar
Kolda, T. G. and Bader, B. W. (2009), Tensor decompositions and applications, SIAM Rev. 51, 455–500.CrossRefGoogle Scholar
Korda, M. and Mezić, I. (2018), Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control, Automatica 93, 149–160.CrossRefGoogle Scholar
Kosambi, D. D. (1943), Statistics in function space, J. Indian Math. Soc. 7, 76–88.Google Scholar
Koval, K., Alexanderian, A. and Stadler, G. (2020), Optimal experimental design under irreducible uncertainty for linear inverse problems governed by PDEs, Inverse Problems 36, 075007.CrossRefGoogle Scholar
Kramer, B. (2020), Stability domains for quadratic-bilinear reduced-order models. Available at arXiv:2009.02769.Google Scholar
Kramer, B. and Willcox, K. (2019), Nonlinear model order reduction via lifting transformations and proper orthogonal decomposition, AIAA J. 57, 2297–2307.CrossRefGoogle Scholar
Kramer, B. and Willcox, K. (2021), Balanced truncation model reduction for lifted nonlinear systems, in Realization and Model Reduction of Dynamical Systems: A Festschrift in Honor of the 70th Birthday of Thanos Antoulas (Beattie, C. et al., eds), Springer. To appear.Google Scholar
Kunisch, K. and Volkwein, S. (2001), Galerkin proper orthogonal decomposition methods for parabolic problems, Numer . Math. 90, 117–148.Google Scholar
Kunisch, K. and Volkwein, S. (2010), Optimal snapshot location for computing POD basis functions, ESAIM Math. Model. Numer. Anal. 44, 509–529.CrossRefGoogle Scholar
Kutz, J. N., Brunton, S. L., Brunton, B. W. and Proctor, J. L. (2016), Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Lall, S., Marsden, J. E. and Glavaski, S. (2002), A subspace approach to balanced truncation for model reduction of nonlinear control systems, Internat. J. Robust Nonlinear Control 12, 519–535.CrossRefGoogle Scholar
Leugering, G., Engell, S., Griewank, A., Hinze, M., Rannacher, R., Schulz, V., Ulbrich, M. and Ulbrich, S., eds (2012), Constrained Optimization and Optimal Control for Partial Differential Equations, Birkhäuser.CrossRefGoogle Scholar
Leykekhman, D. (2012), Investigation of commutative properties of discontinuous Galerkin methods in PDE constrained optimal control problems, J. Sci. Comput. 53, 483–511.CrossRefGoogle Scholar
Lieberman, C., Willcox, K. and Ghattas, O. (2010), Parameter and state model reduction for large-scale statistical inverse problems, SIAM J. Sci. Comput. 32, 2523–2542.CrossRefGoogle Scholar
Lindgren, F., Rue, H. and Lindström, J. (2011), An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach, J. R. Stat. Soc. Ser. B. Stat. Methodol. 73, 423–498.CrossRefGoogle Scholar
Liu, J. and Wang, Z. (2019), Non-commutative discretize-then-optimize algorithms for elliptic PDE-constrained optimal control problems, J. Comput. Appl. Math. 362, 596– 613.CrossRefGoogle Scholar
Liu, Q. and Wang, D. (2016), Stein variational gradient descent: A general purpose Bayesian inference algorithm, in Advances in Neural Information Processing Systems 29 (NIPS 2016) (Lee, D. D. et al., eds), Curran Associates, pp. 2378–2386.Google Scholar
Loève, M. (1955), Probability Theory, Van Nostrand.Google Scholar
Lohmann, B. and Eid, R. (2009), Efficient order reduction of parametric and nonlinear models by superposition of locally reduced models, in Methoden und Anwendungen der Regelungstechnik: Erlangen-Münchener Workshops 2007 und 2008 (Roppencker, G. and Lohmann, B., eds), Shaker, pp. 27–36.Google Scholar
Lumley, J. L. (1967), The structure of inhomogeneous turbulent flow, in Atmospheric Turbulence and Radio Wave Propagation (Yaglom, A. M. and Tartarsky, V. I., eds), Nauka, pp. 166–178.Google Scholar
Ly, H. V. and Tran, H. T. (2001), Modeling and control of physical processes using proper orthogonal decomposition, J. Math. Comput. Model. 33, 223–236.CrossRefGoogle Scholar
Mainini, L. and Willcox, K. (2015), Surrogate modeling approach to support real-time structural assessment and decision making, AIAA J. 53, 1612–1626.CrossRefGoogle Scholar
Mardal, K.-A., Nielsen, B. and Nordaas, M. (2017), Robust preconditioners for PDE-constrained optimization with limited observations, BIT 57, 405–431.CrossRefGoogle Scholar
Martin, J., Wilcox, L. C., Burstedde, C. and Ghattas, O. (2012), A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion, SIAM J. Sci. Comput. 34, A1460–A1487.CrossRefGoogle Scholar
Marzouk, Y. M. and Najm, H. N. (2009), Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems, J. Comput. Phys. 228, 1862– 1902.CrossRefGoogle Scholar
Marzouk, Y. M., Moselhy, T., Parno, M. and Spantini, A. (2016), Sampling via measure transport: An introduction, in Handbook of Uncertainty Quantification (Ghanem, R. et al., eds), Springer, pp. 1–41.Google Scholar
Marzouk, Y. M., Najm, H. N. and Rahn, L. A. (2007), Stochastic spectral methods for efficient Bayesian solution of inverse problems, J. Comput. Phys. 224, 560–586.CrossRefGoogle Scholar
McCormick, G. P. (1976), Computability of global solutions to factorable nonconvex programs, I: Convex underestimating problems, Math. Program. 10, 147–175.CrossRefGoogle Scholar
McQuarrie, S. A., Huang, C. and Willcox, K. (2021), Data-driven reduced-order models via regularised operator inference for a single-injector combustion process, J. Royal Soc. New Zealand 51, 194–211.CrossRefGoogle Scholar
Meehl, G. A., Stocker, T. F., Collins, W. D., Friedlingstein, A. T., Gaye, A. T., Gregory, J. M., Kitoh, A., Knutti, R., Murphy, J. M., Noda, A., Raper, S. C. B., Watterson, I. G., Weaver, A. J. and Zhao, Z.-C. (2007), Global climate projections, in Climate Change 2007: The Physical Science Basis (Solomon, S. D. et al., eds), Cambridge University Press, pp. 747–845.Google Scholar
Mezić, I. (2013), Analysis of fluid flows via spectral properties of the Koopman operator, Annu . Rev. Fluid Mech. 45, 357–378.CrossRefGoogle Scholar
Moselhy, T. A. E. and Marzouk, Y. M. (2012), Bayesian inference with optimal maps, J. Comput. Phys. 231, 7815–7850.CrossRefGoogle Scholar
Mueller, J. L. and Siltanen, S. (2012), Linear and Nonlinear Inverse Problems with Practical Applications, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Naumann, U. (2012), The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Nielsen, B. and Mardal, K.-A. (2010), Efficient preconditioners for optimality systems arising in connection with inverse problems, SIAM J. Control Optim. 48, 5143–5177.CrossRefGoogle Scholar
Nielsen, B. and Mardal, K.-A. (2012), Analysis of the minimum residual method applied to ill posed optimality systems, SIAM J. Sci. Comput. 35, A785–A814.CrossRefGoogle Scholar
Nocedal, J. and Wright, S. J. (2006), Numerical Optimization, second edition, Springer.Google Scholar
North, G. R., Bell, T. L., Cahalan, R. F. and Moeng, F. J. (1982), Sampling errors in the estimation of empirical orthogonal functions, Mon . Weather Rev. 110, 699–706.2.0.CO;2>CrossRefGoogle Scholar
Oden, J. T., Babuška, I. and Faghihi, D. (2017), Predictive computational science: Computer predictions in the presence of uncertainty, in Encyclopedia of Computational Mechanics, second edition, Wiley Online Library, pp. 1–26.Google Scholar
O’Leary-Roseberry, T., Villa, U., Chen, P. and Ghattas, O. (2020), Derivative-informed projected neural networks for high-dimensional parametric maps governed by PDEs. Available at arXiv:2011.15110.Google Scholar
Oliver, D. S. (2017), Metropolized randomized maximum likelihood for improved sampling from multimodal distributions, SIAM/ASA J. Uncertain. Quantif. 5, 259–277.CrossRefGoogle Scholar
Oliver, D. S., He, H. and Reynolds, A. C. (1996), Conditioning permeability fields to pressure data, in 5th European Conference on the Mathematics of Oil Recovery (ECMOR V), European Association of Geoscientists & Engineers, pp. 1–11.Google Scholar
Panzer, H., Mohring, J., Eid, R. and Lohmann, B. (2010), Parametric model order reduction by matrix interpolation, Automatisierungstechnik 58, 475–484.CrossRefGoogle Scholar
Patera, A. T. and Rozza, G. (2006), Reduced basis approximation and a posteriori error estimation for parametrized partial differential equations. Version 1.0, Copyright MIT.CrossRefGoogle Scholar
Paterson, W. S. B. and Budd, W. F. (1982), Flow parameters for ice sheet modeling, Cold Reg . Sci. Technol. 6, 175–177.Google Scholar
Pearson, J. W. and Wathen, A. (2012), A new approximation of the Schur complement in preconditioners for PDE-constrained optimization, Numer . Linear Algebra Appl. 19, 816–829.CrossRefGoogle Scholar
Pearson, J. W., Stoll, M. and Wathen, A. (2012), Regularization-robust preconditioners for time-dependent PDE-constrained optimization problems, SIAM J. Matrix Anal. Appl. 33, 1126–1152.CrossRefGoogle Scholar
Pearson, J. W., Stoll, M. and Wathen, A. J. (2014), Preconditioners for state constrained optimal control problems with Moreau–Yosida penalty function, Numer . Linear Algebra Appl. 21, 81–97.CrossRefGoogle Scholar
Peherstorfer, B. (2020), Sampling low-dimensional Markovian dynamics for preasymptotic-ally recovering reduced models from data with operator inference, SIAM J. Sci. Comput. 42, A3489–A3515.CrossRefGoogle Scholar
Peherstorfer, B. and Willcox, K. (2016), Data-driven operator inference for nonintrusive projection-based model reduction, Comput . Methods Appl. Mech. Engng 306, 196–215.CrossRefGoogle Scholar
Peherstorfer, B., Butnaru, D., Willcox, K. and Bungartz, H. J. (2014), Localized discrete empirical interpolation method, SIAM J. Sci. Comput. 36, A168–A192.CrossRefGoogle Scholar
Peherstorfer, B., Drmac, Z. and Gugercin, S. (2020), Stability of discrete empirical interpolation and gappy proper orthogonal decomposition with randomized and deterministic sampling points, SIAM J. Sci. Comput. 42, A2837–A2864.CrossRefGoogle Scholar
Petra, N., Martin, J., Stadler, G. and Ghattas, O. (2014), A computational framework for infinite-dimensional Bayesian inverse problems, II: Stochastic Newton MCMC with application to ice sheet flow inverse problems, SIAM J. Sci. Comput. 36, A1525–A1555.CrossRefGoogle Scholar
Petra, N., Zhu, H., Stadler, G., Hughes, T. J. R. and Ghattas, O. (2012), An inexact Gauss– Newton method for inversion of basal sliding and rheology parameters in a nonlinear Stokes ice sheet model, J. Glaciology 58, 889–903.CrossRefGoogle Scholar
Prud’homme, C., Rovas, D., Veroy, K., Maday, Y., Patera, A. T. and Turinici, G. (2002), Reliable real-time solution of parameterized partial differential equations: Reduced-basis output bound methods, J. Fluids Engng 124, 70–80.CrossRefGoogle Scholar
Qian, E. (2021), A scientific machine learning approach to learning reduced models for nonlinear partial differential equations. PhD thesis, Massachusetts Institute of Technology.Google Scholar
Qian, E., Grepl, M., Veroy, K. and Willcox, K. (2017), A certified trust region reduced basis approach to PDE-constrained optimization, SIAM J. Sci. Comput. 39, S434–S460.CrossRefGoogle Scholar
Qian, E., Kramer, B., Peherstorfer, B. and Willcox, K. (2020), Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems, Phys. D 406, 132401.Google Scholar
Rasmussen, C. E. and Williams, C. K. (2006), Gaussian Processes for Machine Learning, MIT Press.Google Scholar
Rees, T. and Wathen, A. (2011), Preconditioning iterative methods for the optimal control of the Stokes equations, SIAM J. Sci. Comput. 33, 2903–2926.CrossRefGoogle Scholar
Rees, T., Dollar, S. H. and Wathen, A. J. (2010a), Optimal solvers for PDE-constrained optimization, SIAM J. Sci. Comput. 32, 271–298.CrossRefGoogle Scholar
Rees, T., Stoll, M. and Wathen, A. (2010b), All-at-once preconditioning in PDE-constrained optimization, Kybernetika 46, 341–360.Google Scholar
Rewienski, M. and J. White (2003), A trajectory piecewise-linear approach to model order reduction and fast simulation of nonlinear circuits and micromachined devices, IEEE Trans. Computer-Aided Des. Integr. Circuits Syst. 22, 155–170.CrossRefGoogle Scholar
Rignot, E., Mouginot, J. and Scheuchl, B. (2011), Ice flow of the Antarctic ice sheet, Science 333 (6048), 1427–1430.CrossRefGoogle ScholarPubMed
Rozza, G., Huynh, D. B. P. and Patera, A. T. (2008), Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations: Application to transport and continuum mechanics, Arch. Comput. Methods Engng 15, 229–275.CrossRefGoogle Scholar
Savageau, M. A. and Voit, E. O. (1987), Recasting nonlinear differential equations as S-systems: A canonical nonlinear form, Math. Biosci. 87, 83–115.CrossRefGoogle Scholar
Schiela, A. and Ulbrich, S. (2014), Operator preconditioning for a class of inequality constrained optimal control problems, SIAM J. Optim. 24, 435–466.CrossRefGoogle Scholar
Schillings, C. and Schwab, C. (2013), Sparse, adaptive Smolyak quadratures for Bayesian inverse problems, Inverse Problems 29, 065011.CrossRefGoogle Scholar
Schillings, C., Sprungk, B. and Wacker, P. (2020), On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems, Numer . Math. 145, 915–971.Google Scholar
Schmid, P. J. (2010), Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech. 656, 5–28.CrossRefGoogle Scholar
Schöberl, J. and Zulehner, W. (2007), Symmetric indefinite preconditioners for saddle point problems with applications to PDE-constrained optimization problems, SIAM J. Matrix Anal. Appl. 29, 752–773.CrossRefGoogle Scholar
Shewchuk, J. (1994), An introduction to the conjugate gradient method without the agonizing pain. Report, Carnegie Mellon University.Google Scholar
Simpson, T. W., Mauery, T. M., Korte, J. J. and Mistree, F. (2001), Kriging models for global approximation in simulation-based multidisciplinary design optimization, AIAA J. 39, 2233–2241.CrossRefGoogle Scholar
Sirovich, L. (1987), Turbulence and the dynamics of coherent structures, 1: Coherent structures, Quart. Appl. Math. 45, 561–571.CrossRefGoogle Scholar
Smith, R. C. (2013), Uncertainty Quantification: Theory, Implementation, and Applications, Society for Industrial and Applied Mathematics (SIAM).Google Scholar
Ştefănescu, R., Sandu, A. and Navon, I. M. (2015), POD/DEIM reduced-order strategies for efficient four dimensional variational data assimilation, J. Comput. Phys. 295, 569–595.CrossRefGoogle Scholar
Stuart, A. M. (2010), Inverse problems: A Bayesian perspective, in Acta Numerica, Vol. 19, Cambridge University Press, pp. 451–559.Google Scholar
Subramanian, S., Scheufele, K., Mehl, M. and Biros, G. (2020), Where did the tumor start? An inverse solver with sparse localization for tumor growth models, Inverse Problems 36, 045006.CrossRefGoogle Scholar
Sullivan, T. J. (2015), Introduction to Uncertainty Quantification, Springer.CrossRefGoogle Scholar
Swischuk, R., Kramer, B., Huang, C. and Willcox, K. (2020), Learning physics-based reduced-order models for a single-injector combustion process, AIAA J. 58, 2658–2672.CrossRefGoogle Scholar
Swischuk, R., Mainini, L., Peherstorfer, B. and Willcox, K. (2019), Projection-based model reduction: Formulations for physics-based machine learning, Comput . Fluids 179, 704– 717.Google Scholar
Symes, W. W. (2009), The seismic reflection inverse problem, Inverse Problems 25, 123008.CrossRefGoogle Scholar
Takacs, S. and Zulehner, W. (2011), Convergence analysis of multigrid methods with collective point smoothers for optimal control problems, Comput. Vis. Sci. 14, 131–141.CrossRefGoogle Scholar
Tarantola, A. (2005), Inverse Problem Theory and Methods for Model Parameter Estimation, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Tenorio, L. (2017), An Introduction to Data Analysis and Uncertainty Quantification for Inverse Problems, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Tong, S., Vanden-Eijnden, E. and Stadler, G. (2020), Extreme event probability estimation using PDE-constrained optimization and large deviation theory, with application to tsunamis. Available at arXiv:2007.13930.Google Scholar
Tröltzsch, F. (2010), Optimal Control of Partial Differential Equations: Theory, Methods and Applications, Vol. 112 of Graduate Studies in Mathematics, American Mathematical Society.CrossRefGoogle Scholar
Ulbrich, M. (2011), Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces, Society for Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
Utke, J., Hascoët, L., Heimbach, P., Hill, C., Hovland, P. and Naumann, U. (2009), Toward adjoinable MPI, in IEEE International Symposium on Parallel & Distributed Processing (IPDPS 2009), IEEE, pp. 1–8.Google Scholar
Venter, G., Haftka, R. and Starnes, J. H. (1998), Construction of response surface approximations for design optimization, AIAA J. 36, 2242–2249.CrossRefGoogle Scholar
Veroy, K. and Patera, A. (2005), Certified real-time solution of the parametrized steady incompressible Navier–Stokes equations: Rigorous reduced-basis a posteriori error bounds, Internat. J. Numer. Methods Fluids 47, 773–788.CrossRefGoogle Scholar
Veroy, K., Prud’homme, C., Rovas, D. V. and Patera, A. T. (2003), A posteriori error bounds for reduced-basis approximation of parametrized noncoercive and nonlinear elliptic partial differential equations, in Proceedings of the 16th AIAA Computational Fluid Dynamics Conference. AIAA paper 2003-3847.Google Scholar
Villa, U., Petra, N. and Ghattas, O. (2021), hIPPYlib: An extensible software framework for large-scale inverse problems governed by PDEs, I: Deterministic inversion and linearized Bayesian inference, ACM Trans. Math. Software 47, 16.CrossRefGoogle Scholar
Vogel, C. R. (2002), Computational Methods for Inverse Problems, Frontiers in Applied Mathematics, Society for Industrial and Applied Mathematics (SIAM).Google Scholar
Wang, K., Bui-Thanh, T. and Ghattas, O. (2018), A randomized maximum a posteriori method for posterior sampling of high dimensional nonlinear Bayesian inverse problems, SIAM J. Sci. Comput. 40, A142–A171.Google Scholar
Wang, Q., Hesthaven, J. S. and Ray, D. (2019), Non-intrusive reduced order modeling of unsteady flows using artificial neural networks with application to a combustion problem, J. Comput. Phys. 384, 289–307.CrossRefGoogle Scholar
Wang, Z., Bardsley, J. M., Solonen, A., Cui, T. and Marzouk, Y. M. (2017), Bayesian inverse problems with l 1 priors: A randomize-then-optimize approach, SIAM J. Sci. Comput. 39, S140–S166.CrossRefGoogle Scholar
Wilcox, L. C., Stadler, G., Bui-Thanh, T. and Ghattas, O. (2015), Discretely exact derivatives for hyperbolic PDE-constrained optimization problems discretized by the discontinuous Galerkin method, J. Sci. Comput. 63, 138–162.CrossRefGoogle Scholar
Wild, S. M., Regis, R. G. and Shoemaker, C. A. (2008), ORBIT: Optimization by radial basis function interpolation in trust-regions, SIAM J. Sci. Comput. 30, 3197–3219.CrossRefGoogle Scholar
Willcox, K. and Peraire, J. (2002), Balanced model reduction via the proper orthogonal decomposition, AIAA J. 40, 2323–2330.Google Scholar
Worthen, J., Stadler, G., Petra, N., Gurnis, M. and Ghattas, O. (2014), Towards adjoint-based inversion for rheological parameters in nonlinear viscous mantle flow, Phys. Earth Planet. Inter. 234, 23–34.CrossRefGoogle Scholar
Wu, K., Chen, P. and Ghattas, O. (2020), A fast and scalable computational framework for large-scale and high-dimensional Bayesian optimal experimental design. Available at arXiv:2010.15196.Google Scholar
Wu, K., Chen, P. and Ghattas, O. (2021), A fast and scalable computational framework for goal-oriented linear Bayesian optimal experimental design: Application to optimal sensor placement. Available at arXiv:2102.06627.Google Scholar
Xiu, D. and Karniadakis, G. E. (2002), The Wiener–Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput. 24, 619–644.CrossRefGoogle Scholar
Yang, H. and Veneziani, A. (2017), Efficient estimation of cardiac conductivities via POD– DEIM model order reduction, Appl. Numer. Math. 115, 180–199.CrossRefGoogle Scholar
Yang, S., Stadler, G., Moser, R. and Ghattas, O. (2011), A shape Hessian-based boundary roughness analysis of Navier–Stokes flow, SIAM J. Appl. Math. 71, 333–355.CrossRefGoogle Scholar
Yücel, H., Heinkenschloss, M. and Karasözen, B. (2013), Distributed optimal control of diffusion–convection–reaction equations using discontinuous Galerkin methods, in Numerical Mathematics and Advanced Applications 2011 (Cangiani, A. et al., eds), Springer, pp. 389–397.Google Scholar
Zhang, Z., Bader, E. and Veroy, K. (2016), A slack approach to reduced-basis approximation and error estimation for variational inequalities, Comptes Rendus Mathématique 354, 283–289.CrossRefGoogle Scholar
Zhu, H., Li, S., Fomel, S., Stadler, G. and Ghattas, O. (2016a), A Bayesian approach to estimate uncertainty for full waveform inversion with a priori information from depth migration, Geophysics 81, R307–R323.CrossRefGoogle Scholar
Zhu, H., Petra, N., Stadler, G., Isaac, T., Hughes, T. J. R. and Ghattas, O. (2016b), Inversion of geothermal heat flux in a thermomechanically coupled nonlinear Stokes ice sheet model, Cryosphere 10, 1477–1494.CrossRefGoogle Scholar