Hostname: page-component-77f85d65b8-pkds5 Total loading time: 0 Render date: 2026-04-19T02:49:17.537Z Has data issue: false hasContentIssue false
  • English
  • Français

Inverse problems: A Bayesian perspective

Published online by Cambridge University Press:  10 May 2010

A. M. Stuart
A. M. Stuart
Affiliation:
Mathematics Institute, University of Warwick, Coventry CV4 7AL, UK, E-mail: a.m.stuart@warwick.ac.uk
Get access Rights & Permissions [Opens in a new window]

Extract

The subject of inverse problems in differential equations is of enormous practical importance, and has also generated substantial mathematical and computational innovation. Typically some form of regularization is required to ameliorate ill-posed behaviour. In this article we review the Bayesian approach to regularization, developing a function space viewpoint on the subject. This approach allows for a full characterization of all possible solutions, and their relative probabilities, whilst simultaneously forcing significant modelling issues to be addressed in a clear and precise fashion. Although expensive to implement, this approach is starting to lie within the range of the available computational resources in many application areas. It also allows for the quantification of uncertainty and risk, something which is increasingly demanded by these applications. Furthermore, the approach is conceptually important for the understanding of simpler, computationally expedient approaches to inverse problems.

Information

Type
Research Article
Information
Acta Numerica , Volume 19 , May 2010 , pp. 451 - 559
Copyright
Copyright © Cambridge University Press 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

REFERENCES

Adler, R. J. (1990), An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes, Vol. 12 of Institute of Mathematical Statistics Lecture Notes: Monograph Series, Institute of Mathematical Statistics, Hayward, CA.Google Scholar
Akella, S. and Navon, I. (2009), ‘Different approaches to model error formulation in 4D-Var: A study with high resolution advection schemes’, Tellus 61A, 112–128.CrossRefGoogle Scholar
Alekseev, A. and Navon, I. (2001), ‘The analysis of an ill-posed problem using multiscale resolution and second order adjoint techniques’, Comput. Meth. Appl. Mech. Engrg 190, 1937–1953.CrossRefGoogle Scholar
Antoulas, A., Soresen, D. and Gugerrin, S. (2001), A Survey of Model Reduction Methods for Large Scale Dynamical Systems, AMS.Google Scholar
Apte, A., Hairer, M., Stuart, A. and Voss, J. (2007), ‘Sampling the posterior: An approach to non-Gaussian data assimilation’, Physica D 230, 50–64.CrossRefGoogle Scholar
Apte, A., Jones, C. and Stuart, A. (2008 a), ‘A Bayesian approach to Lagrangian data assimilation’, Tellus 60, 336–347.CrossRefGoogle Scholar
Apte, A., Jones, C., Stuart, A. and Voss, J. (2008 b), ‘Data assimilation: Mathematical and statistical perspectives’, Internat. J. Numer. Methods Fluids 56, 1033–1046.CrossRefGoogle Scholar
Archambeau, C., Cornford, D., Opper, M. and Shawe, J.-Taylor (2007), Gaussian process approximations of stochastic differential equations. In JMLR Workshop and Conference Proceedings 1: Gaussian Processes in Practice (Lawrence, N., ed.), The MIT Press, pp. 1–16.Google Scholar
Archambeau, C., Opper, M., Shen, Y., Cornford, D. and Shawe-Taylor, J. (2008), Variational inference for diffusion processes. In Advances in Neural Information Processing Systems 20 (Platt, J., Koller, D., Singer, Y. and Roweis, S., eds), The MIT Press, Cambridge, MA, pp. 17–24.Google Scholar
Backus, G. (1970 a), ‘Inference from inadequate and inaccurate data I’, Proc. Nat. Acad. Sci. 65, 1–7.CrossRefGoogle ScholarPubMed
Backus, G. (1970 b), ‘Inference from inadequate and inaccurate data II’, Proc. Nat. Acad. Sci. 65, 281–287.CrossRefGoogle ScholarPubMed
Backus, G. (1970 c), ‘Inference from inadequate and inaccurate data III’, Proc. Nat. Acad. Sci. 67, 282–289.CrossRefGoogle ScholarPubMed
Bain, A. and Crisan, D. (2009), Fundamentals of Stochastic Filtering, Springer.CrossRefGoogle Scholar
Bannister, R., Katz, D., Cullen, M., Lawless, A. and Nichols, N. (2008), ‘Modelling of forecast errors in geophysical fluid flows’, Internat. J. Numer. Methods Fluids 56, 1147–1153.CrossRefGoogle Scholar
Beck, J., Blackwell, B. and Clair, C. (2005), Inverse Heat Conduction: Ill-Posed Problems, Wiley.Google Scholar
Bell, M., Martin, M. and Nichols, N. (2004), ‘Assimilation of data into an ocean model with systematic errors near the equator’, Quart. J. Royal Met. Soc. 130, 873–894.CrossRefGoogle Scholar
Bengtsson, T., Bickel, P. and Li, B. (2008), ‘Curse of dimensionality revisited: The collapse of importance sampling in very large scale systems’, IMS Collections: Probability and Statistics: Essays in Honor of David Freedman 2, 316–334.CrossRefGoogle Scholar
Bengtsson, T., Snyder, C. and Nychka, D. (2003), ‘Toward a nonlinear ensemble filter for high-dimensional systems’, J. Geophys. Res. 108, 8775.Google Scholar
Bennett, A. (2002), Inverse Modeling of the Ocean and Atmosphere, Cambridge University Press.CrossRefGoogle Scholar
Bennett, A. and Budgell, W. (1987), ‘Ocean data assimilation and the Kalman filter: Spatial regularity’, J. Phys. Oceanography 17, 1583–1601.2.0.CO;2>CrossRefGoogle Scholar
Bennett, A. and Chua, B. (1994), ‘Open ocean modelling as an inverse problem’, Monthly Weather Review 122, 1326–1336.2.0.CO;2>CrossRefGoogle Scholar
Bennett, A. and Miller, R. (1990), ‘Weighting initial conditions in variational assimilation schemes’, Monthly Weather Review 119, 1098–1102.2.0.CO;2>CrossRefGoogle Scholar
Bergemann, K. and Reich, S. (2010), ‘A localization technique for ensemble transform Kalman filters’, Quart. J. Royal Met. Soc. To appear.CrossRefGoogle Scholar
Berliner, L. (2001), ‘Monte Carlo based ensemble forecasting’, Statist. Comput. 11, 269–275.CrossRefGoogle Scholar
Bernardo, J. and Smith, A. (1994), Bayesian Theory, Wiley.CrossRefGoogle Scholar
Beskos, A. and Stuart, A. (2009), MCMC methods for sampling function space. In Invited Lectures: Sixth International Congress on Industrial and Applied Mathematics, ICIAM07 (Jeltsch, R. and Wanner, G., eds), European Mathematical Society, pp. 337–364.Google Scholar
Beskos, A. and Stuart, A. M. (2010), Computational complexity of Metropolis Hastings methods in high dimensions. In Monte Carlo and Quasi-Monte Carlo Methods 2008 (L'Ecuyer, P. and Owen, A. B., eds), Springer, pp. 61–72.Google Scholar
Beskos, A., Roberts, G. O. and Stuart, A. M. (2009), ‘Optimal scalings for local Metropolis-Hastings chains on non-product targets in high dimensions’, Ann. Appl. Probab. 19, 863–898.CrossRefGoogle Scholar
Beskos, A., Roberts, G. O., Stuart, A. M. and Voss, J. (2008), ‘MCMC methods for diffusion bridges’, Stochastic Dynamics 8, 319–350.CrossRefGoogle Scholar
Bickel, P. and Doksum, K. (2001), Mathematical Statistics, Prentice-Hall.Google Scholar
Bickel, P., Li, B. and Bengtsson, T. (2008), ‘Sharp failure rates for the bootstrap particle filter in high dimensions’, IMS Collections: Pushing the Limits of Contemporary Statistics 3, 318–329.Google Scholar
Bogachev, V. (1998), Gaussian Measures, AMS.CrossRefGoogle Scholar
Bolhuis, P., Chandler, D., Dellago, D. and Geissler, P. (2002), ‘Transition path sampling: Throwing ropes over rough mountain passes’, Ann. Rev. Phys. Chem. 53, 291–318.CrossRefGoogle ScholarPubMed
Borcea, L. (2002), ‘Electrical impedence tomography’, Inverse Problems 18, R99–R136.CrossRefGoogle Scholar
Brasseur, P., Bahurel, P., Bertino, L., Birol, F., Brankart, J.-M., Ferry, N., Losa, S., Remy, E., Schroeter, J., Skachko, S., Testut, C.-E., Tranchat, B., Van Leeuwen, P. and Verron, J. (2005), ‘Data assimilation for marine monitoring and prediction: The Mercator operational assimilation systems and the Mersea developments’, Quart. J. Royal Met. Soc. 131, 3561–3582.CrossRefGoogle Scholar
Breiman, L. (1992), Probability, Vol. 7 of Classics in Applied Mathematics, SIAM, Philadelphia, PA. Corrected reprint of the 1968 original.Google Scholar
Burgers, G., Van Leeuwen, P. and Evensen, G. (1998), ‘On the analysis scheme in the ensemble Kalman filter’, Monthly Weather Review 126, 1719–1724.2.0.CO;2>CrossRefGoogle Scholar
Calvetti, D. (2007), ‘Preconditioned iterative methods for linear discrete ill-posed problems from a Bayesian inversion perspective’, J. Comput. Appl. Math. 198, 378–395.CrossRefGoogle Scholar
Calvetti, D. and Somersalo, E. (2005 a), ‘Priorconditioners for linear systems’, Inverse Problems 21, 1397–1418.CrossRefGoogle Scholar
Calvetti, D. and Somersalo, E. (2005 b), ‘Statistical elimination of boundary artefacts in image deblurring’, Inverse Problems 21, 1697–1714.CrossRefGoogle Scholar
Calvetti, D. and Somersalo, E. (2006), ‘Large-scale statistical parameter estimation in complex systems with an application to metabolic models’, Multiscale Modeling and Simulation 5, 1333–1366.CrossRefGoogle Scholar
Calvetti, D. and Somersalo, E. (2007 a), ‘Gaussian hypermodel to recover blocky objects’, Inverse Problems 23, 733–754.CrossRefGoogle Scholar
Calvetti, D. and Somersalo, E. (2007 b), Introduction to Bayesian Scientific Computing, Vol. 2 of Surveys and Tutorials in the Applied Mathematical Sciences, Springer.Google Scholar
Calvetti, D. and Somersalo, E. (2008), ‘Hypermodels in the Bayesian imaging framework’, Inverse Problems 24, #034013.CrossRefGoogle Scholar
Calvetti, D., Hakula, H., Pursiainen, S. and Somersalo, E. (2009), ‘Conditionally Gaussian hypermodels for cerebral source location’, SIAM J. Imag. Sci. 2, 879–909.CrossRefGoogle Scholar
Calvetti, D., Kuceyeski, A. and Somersalo, E. (2008), ‘Sampling based analysis of a spatially distributed model for liver metabolism at steady state’, Multiscale Modeling and Simulation 7, 407–431.CrossRefGoogle Scholar
Candès, E. and Wakin, M. (2008), ‘An introduction to compressive sampling’, IEEE Signal Processing Magazine, March 2008, 21–30.CrossRefGoogle Scholar
Chemin, J.-Y. and Lerner, N. (1995), ‘Flot de champs de veceurs non lipschitziens et équations de Navier-Stokes’, J. Diff. Equations 121, 314–328.CrossRefGoogle Scholar
Chorin, A. and Hald, O. (2006), Stochastic Tools in Mathematics and Science, Vol. 1 of Surveys and Tutorials in the Applied Mathematical Sciences, Springer, New York.Google Scholar
Chorin, A. and Krause, P. (2004), ‘Dimensional reduction for a Bayesian filter’, Proc. Nat. Acad. Sci. 101, 15013–15017.CrossRefGoogle ScholarPubMed
Chorin, A. and Tu, X. (2009), ‘Implicit sampling for particle filters’, Proc. Nat. Acad. Sci. 106, 17249–17254.CrossRefGoogle ScholarPubMed
Chorin, A. and Tu, X. (2010), ‘Interpolation and iteration for nonlinear filters’, Math. Model. Numer. Anal. To appear.Google Scholar
Christie, M. (2010), Solution error modelling and inverse problems. In Simplicity, Complexity and Modelling, Wiley, New York, to appear.Google Scholar
Christie, M., Pickup, G., O'Sullivan, A. and Demyanov, V. (2008), Use of solution error models in history matching. In Proc. European Conference on the Mathematics of Oil Recovery XI, European Association of Geoscientists and Engineers.Google Scholar
Chua, B. and Bennett, A. (2001), ‘An inverse ocean modelling system’, Ocean. Meteor. 3, 137–165.Google Scholar
Cohn, S. (1997), ‘An introduction to estimation theory’, J. Met. Soc. Japan 75, 257–288.CrossRefGoogle Scholar
Cotter, S., Dashti, M., Robinson, J. and Stuart, A. (2009), ‘Bayesian inverse problems for functions and applications to fluid mechanics’, Inverse Problems 25, #115008.CrossRefGoogle Scholar
Cotter, S., Dashti, M. and Stuart, A. (2010 a), ‘Approximation of Bayesian inverse problems’, SIAM J. Numer. Anal. To appear.CrossRefGoogle Scholar
Cotter, S., Dashti, M., Robinson, J. and Stuart, A. (2010 b). In preparation.Google Scholar
Courtier, P. (1997), ‘Dual formulation of variational assimilation’, Quart. J. Royal Met. Soc. 123, 2449–2461.CrossRefGoogle Scholar
Courtier, P. and Talagrand, O. (1987), ‘Variational assimilation of meteorological observations with the adjoint vorticity equation II: Numerical results’, Quart. J. Royal Met. Soc. 113, 1329–1347.CrossRefGoogle Scholar
Courtier, P., Anderson, E., Heckley, W., Pailleux, J., Vasiljevic, D., Hamrud, M., Hollingworth, A., Rabier, F. and Fisher, M. (1998), ‘The ECMWF implementation of three-dimensional variational assimilation (3D-Var)’, Quart. J. Royal Met. Soc. 124, 1783–1808.Google Scholar
Cressie, N. (1993), Statistics for Spatial Data, Wiley.CrossRefGoogle Scholar
Cui, T., Fox, C., Nicholls, G. and O'Sullivan, M. (2010), ‘Using MCMC sampling to calibrate a computer model of a geothermal field’. Submitted.Google Scholar
Da, G. Prato and Zabczyk, J. (1992), Stochastic Equations in Infinite Dimensions, Vol. 44 of Encyclopedia of Mathematics and its Applications, Cambridge University Press.Google Scholar
Dacarogna, B. (1989), Direct Methods in the Calculus of Variations, Springer, New York.CrossRefGoogle Scholar
Dashti, M. and Robinson, J. (2009), ‘Uniqueness of the particle trajectories of the weak solutions of the two-dimensional Navier-Stokes equations’, Nonlinearity 22, 735–746.CrossRefGoogle Scholar
Dashti, M., Harris, S. and Stuart, A. M. (2010 a), Bayesian approach to an elliptic inverse problem. In preparation.Google Scholar
Dashti, M., Pillai, N. and Stuart, A. (2010 b), Bayesian Inverse Problems in Differential Equations. Lecture notes, available from: http://www.warwick.ac.uk/~masdr/inverse.html.Google Scholar
Derber, J. (1989), ‘A variational continuous assimilation technique’, Monthly Weather Review 117, 2437–2446.2.0.CO;2>CrossRefGoogle Scholar
Deuflhard, P. (2004), Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms, Springer.Google Scholar
DeVolder, B., Glimm, J., Grove, J., Kang, Y., Lee, Y., Pao, K., Sharp, D. and Ye, K. (2002), ‘Uncertainty quantification for multiscale simulations’, J. Fluids Engrg 124, 29–42.CrossRefGoogle Scholar
Donoho, D. (2006), ‘Compressed sensing’, IEEE Trans. Inform. Theory 52, 1289– 1306.CrossRefGoogle Scholar
Dostert, P., Efendiev, Y., Hou, T. and Luo, W. (2006), ‘Coarse-grain Langevin algorithms for dynamic data integration’, J. Comput. Phys. 217, 123–142.CrossRefGoogle Scholar
Doucet, N., de Frietas, A. and Gordon, N. (2001), Sequential Monte Carlo in Practice, Springer.CrossRefGoogle Scholar
Dudley, R. (2002), Real Analysis and Probability, Cambridge University Press, Cambridge.CrossRefGoogle Scholar
Dürr, D. and Bach, A. (1978), ‘The Onsager-Machlup function as Lagrangian for the most probable path of a diffusion process’, Comm. Math. Phys. 160, 153–170.CrossRefGoogle Scholar
Efendiev, Y., Datta-Gupta, A., Ma, X. and Mallick, B. (2009), ‘Efficient sampling techniques for uncertainty quantification in history matching using nonlinear error models and ensemble level upscaling techniques’, Water Resources Res. 45, #W11414.CrossRefGoogle Scholar
Eknes, M. and Evensen, G. (1997), ‘Parameter estimation solving a weak constraint variational formulation for an Ekman model’, J. Geophys. Res. 12, 479–491.Google Scholar
Ellerbroek, B. and Vogel, C. (2009), ‘Inverse problems in astronomical adaptive optics’, Inverse Problems 25, #063001.CrossRefGoogle Scholar
Engl, H., Hanke, M. and Neubauer, A. (1996), Regularization of Inverse Problems, Kluwer.CrossRefGoogle Scholar
Engl, H., Hofinger, A. and Kindermann, S. (2005), ‘Convergence rates in the Prokhorov metric for assessing uncertainty in ill-posed problems’, Inverse Problems 21, 399–412.CrossRefGoogle Scholar
Evensen, G. (2006), Data Assimilation: The Ensemble Kalman Filter, Springer.Google Scholar
Evensen, G. and Van Leeuwen, P. (2000), ‘An ensemble Kalman smoother for nonlinear dynamics’, Monthly Weather Review 128, 1852–1867.2.0.CO;2>CrossRefGoogle Scholar
Fang, F., Pain, C., Navon, I., Piggott, M., Gorman, G., Allison, P. and Goddard, A. (2009 a), ‘Reduced order modelling of an adaptive mesh ocean model’, Internat. J. Numer. Methods Fluids 59, 827–851.CrossRefGoogle Scholar
Fang, F., Pain, C., Navon, I., Piggott, M., Gorman, G., Farrell, P., Allison, P. and Goddard, A. (2009 b), ‘A POD reduced-order 4D-Var adaptive mesh ocean modelling approach’, Internat. J. Numer. Methods Fluids 60, 709–732.CrossRefGoogle Scholar
Farmer, C. (2005), Geological modelling and reservoir simulation. In Mathematical Methods and Modeling in Hydrocarbon Exploration and Production (Iske, A. and Randen, T., eds), Springer, Heidelberg, pp. 119–212.CrossRefGoogle Scholar
Farmer, C. (2007), Bayesian field theory applied to scattered data interpolation and inverse problems. In Algorithms for Approximation (Iske, A. and Levesley, J., eds), Springer, pp. 147–166.CrossRefGoogle Scholar
Fitzpatrick, B. (1991), ‘Bayesian analysis in inverse problems’, Inverse Problems 7, 675–702.CrossRefGoogle Scholar
Franklin, J. (1970), ‘Well-posed stochastic extensions of ill-posed linear problems’, J. Math. Anal. Appl. 31, 682–716.CrossRefGoogle Scholar
Freidlin, M. and Wentzell, A. (1984), Random Perturbations of Dynamical Systems, Springer, New York.CrossRefGoogle Scholar
Gelfand, A. and Smith, A. (1990), ‘Sampling-based approaches to calculating marginal densities’, J. Amer. Statist. Soc. 85, 398–409.CrossRefGoogle Scholar
Gibbs, A. and Su, F. (2002), ‘On choosing and bounding probability metrics’, Internat. Statist. Review 70, 419–435.CrossRefGoogle Scholar
Gittelson, C. and Schwab, C. (2011), Sparse tensor discretizations of high-dimen-sional PDEs. To appear in Acta Numerica, Vol. 20.Google Scholar
Glimm, J., Hou, S., Lee, Y., Sharp, D. and Ye, K. (2003), ‘Solution error models for uncertainty quantification’, Contemporary Mathematics 327, 115–140.CrossRefGoogle Scholar
Gratton, S., Lawless, A. and Nichols, N. (2007), ‘Approximate Gauss—Newton methods for nonlinear least squares problems’, SIAM J. Optimization 18, 106–132.CrossRefGoogle Scholar
Griffith, A. and Nichols, N. (1998), Adjoint methods for treating model error in data assimilation. In Numerical Methods for Fluid Dynamics VI, ICFD, Oxford, pp. 335–344.Google Scholar
Griffith, A. and Nichols, N. (2000), ‘Adjoint techniques in data assimilation for treating systematic model error’, J. Flow, Turbulence and Combustion 65, 469–488.Google Scholar
Grimmett, G. and Stirzaker, D. (2001), Probability and Random Processes, Oxford University Press, New York.CrossRefGoogle Scholar
Gu, C. (2002), Smoothing Spline ANOVA Models, Springer.CrossRefGoogle Scholar
Gu, C. (2008), ‘Smoothing noisy data via regularization’, Inverse Problems 24, #034002.CrossRefGoogle Scholar
Hagelberg, C., Bennett, A. and Jones, D. (1996), ‘Local existence results for the generalized inverse of the vorticity equation in the plane’, Inverse Problems 12, 437–454.CrossRefGoogle Scholar
Hairer, E. and Wanner, G. (1996), Solving Ordinary Differential Equations II, Vol. 14 of Springer Series in Computational Mathematics, Springer, Berlin.Google Scholar
Hairer, E., Nørsett, S. P. and Wanner, G. (1993), Solving Ordinary Differential Equations I, Vol. 8 of Springer Series in Computational Mathematics, Springer, Berlin.Google Scholar
Hairer, M. (2009), Introduction to Stochastic PDEs. Lecture notes.Google Scholar
Hairer, M., Stuart, A. M. and Voss, J. (2007), ‘Analysis of SPDEs arising in path sampling II: The nonlinear case’, Ann. Appl. Probab. 17, 1657–1706.CrossRefGoogle Scholar
Hairer, M., Stuart, A. M. and Voss, J. (2009), Sampling conditioned diffusions. In Trends in Stochastic Analysis, Vol. 353 of London Mathematical Society Lecture Notes, Cambridge University Press, pp. 159–186.CrossRefGoogle Scholar
Hairer, M., Stuart, A. and Voss, J. (2010 a), ‘Sampling conditioned hypoelliptic diffusions’. Submitted.CrossRefGoogle Scholar
Hairer, M., Stuart, A. and Voss, J. (2010 b), Signal processing problems on function space: Bayesian formulation, stochastic PDEs and effective MCMC methods. In Oxford Handbook of Nonlinear Filtering (Crisan, D. and Rozovsky, B., eds), Oxford University Press, to appear.Google Scholar
Hairer, M., Stuart, A., Voss, J. and Wiberg, P. (2005), ‘Analysis of SPDEs arising in path sampling I: The Gaussian case’, Comm. Math. Sci. 3, 587–603.CrossRefGoogle Scholar
Hastings, W. K. (1970), ‘Monte Carlo sampling methods using Markov chains and their applications’, Biometrika 57, 97–109.CrossRefGoogle Scholar
Hein, T. (2009), ‘On Tikhonov regularization in Banach spaces: Optimal convergence rate results’, Applicable Analysis 88, 653–667.CrossRefGoogle Scholar
Heino, J., Tunyan, K., Calvetti, D. and Somersalo, E. (2007), ‘Bayesian flux balance analysis applied to a skeletal muscle metabolic model’, J. Theor. Biol. 248, 91–110.CrossRefGoogle ScholarPubMed
Herbei, R. and McKeague, I. (2009), ‘Geometric ergodicity of hybrid samplers for ill-posed inverse problems’, Scand. J. Statist. 36, 839–853.CrossRefGoogle Scholar
Herbei, R., McKeague, I. and Speer, K. (2008), ‘Gyres and jets: Inversion of tracer data for ocean circulation structure’, J. Phys. Oceanography 38, 1180–1202.CrossRefGoogle Scholar
Hofinger, A. and Pikkarainen, H. (2007), ‘Convergence rates for the Bayesian approach to linear inverse problems’, Inverse Problems 23, 2469–2484.CrossRefGoogle Scholar
Hofinger, A. and Pikkarainen, H. (2009), ‘Convergence rates for linear inverse problems in the presence of an additive normal noise’, Stoch. Anal. Appl. 27, 240–257.CrossRefGoogle Scholar
Huddleston, M., Bell, M., Martin, M. and Nichols, N. (2004), ‘Assessment of wind stress errors using bias corrected ocean data assimilation’, Quart. J. Royal Met. Soc. 130, 853–872.CrossRefGoogle Scholar
Hurzeler, M. and Kunsch, H. (2001), Approximating and maximizing the likelihood for a general state space model. In Sequential Monte Carlo Methods in Practice (Doucet, A., de Freitas, N. and Gordon, N., eds), Springer, pp. 159–175.CrossRefGoogle Scholar
Huttunen, J. and Pikkarainen, H. (2007), ‘Discretization error in dynamical inverse problems: One-dimensional model case’, J. Inverse and Ill-posed Problems 15, 365–386.CrossRefGoogle Scholar
Ide, K. and Jones, C. (2007), ‘Data assimilation’, Physica D 230, vii–viii.CrossRefGoogle Scholar
Ide, K., Kuznetsov, L. and Jones, C. (2002), ‘Lagrangian data assimilation for pointvortex system’, J. Turbulence 3, 53.CrossRefGoogle Scholar
Ikeda, N. and Watanabe, S. (1989), Stochastic Differential Equations and Diffusion Processes, second edn, North-Holland, Amsterdam.Google Scholar
Jardak, M., Navon, I. and Zupanski, M. (2010), ‘Comparison of sequential data assimilation methods for the Kuramoto—Sivashinsky equation’, Internat. J. Numer. Methods Fluids 62, 374–402.CrossRefGoogle Scholar
Johnson, C., Hoskins, B. and Nichols, N. (2005), ‘A singular vector perspective of 4DVAR: Filtering and interpolation’, Quart. J. Royal Met. Soc. 131, 1–20.CrossRefGoogle Scholar
Johnson, C., Hoskins, B., Nichols, N. and Ballard, S. (2006), ‘A singular vector perspective of 4DVAR: The spatial structure and evolution of baroclinic weather systems’, Monthly Weather Review 134, 3436–3455.CrossRefGoogle Scholar
Kaipio, J. and Somersalo, E. (2000), ‘Statistical inversion and Monte Carlo sampling methods in electrical impedance tomography’, Inverse Problems 16, 1487–1522.CrossRefGoogle Scholar
Kaipio, J. and Somersalo, E. (2005), Statistical and Computational Inverse problems, Vol. 160 of Applied Mathematical Sciences, Springer.Google Scholar
Kaipio, J. and Somersalo, E. (2007 a), ‘Approximation errors in nonstationary inverse problems’, Inverse Problems and Imaging 1, 77–93.Google Scholar
Kaipio, J. and Somersalo, E. (2007 b), ‘Statistical inverse problems: Discretization, model reduction and inverse crimes’, J. Comput. Appl. Math. 198, 493–504.CrossRefGoogle Scholar
Kalnay, E. (2003), Atmospheric Modeling, Data Assimilation and Predictability, Cambridge University Press.Google Scholar
Kalnay, E., Li, H., Miyoshi, S., Yang, S. and Ballabrera-Poy, J. (2007), ‘4D-Var or ensemble Kalman filter?’, Tellus 59, 758–773.CrossRefGoogle Scholar
Kaltenbacher, B., Schöpfer, F. and Schuster, T. (2009), ‘Iterative methods for non-linear ill-posed problems in Banach spaces: Convergence and applications to parameter identification problems’, Inverse Problems 25, #065003.Google Scholar
Kennedy, M. and O'Hagan, A. (2001), ‘Bayesian calibration of computer models’, J. Royal Statist. Soc. 63B, 425–464.CrossRefGoogle Scholar
Kinderlehrer, D. and Stampacchia, G. (1980), An Introduction to Variational In-equalities and their Applications, SIAM.Google Scholar
Kolda, T. and Bader, B. (2009), ‘Tensor decompositions and applications’, SIAM Review 51, 455–500.CrossRefGoogle Scholar
Kuznetsov, L., Ide, K. and Jones, C. (2003), ‘A method for assimilation of Lagrangian data’, Monthly Weather Review 131, 2247–2260.2.0.CO;2>CrossRefGoogle Scholar
Lassas, M. and Siltanen, S. (2004), ‘Can one use total variation prior for edge-preserving Bayesian inversion?’, Inverse Problems 20, 1537–1563.CrossRefGoogle Scholar
Lassas, M., Saksman, E. and Siltanen, S. (2009), ‘Discretization-invariant Bayesian inversion and Besov space priors’, Inverse Problems and Imaging 3, 87–122.CrossRefGoogle Scholar
Lawless, A. and Nichols, N. (2006), ‘Inner loop stopping criteria for incremental four-dimensional variational data assimilation’, Monthly Weather Review 134, 3425–3435.CrossRefGoogle Scholar
Lawless, A., Gratton, S. and Nichols, N. (2005 a), ‘Approximate iterative methods for variational data assimilation’, Internat. J. Numer. Methods Fluids 47, 1129–1135.CrossRefGoogle Scholar
Lawless, A., Gratton, S. and Nichols, N. (2005 b), ‘An investigation of incremental 4D-Var using non-tangent linear models’, Quart. J. Royal Met. Soc. 131, 459–476.CrossRefGoogle Scholar
Lawless, A., Nichols, N., Boess, C. and Bunse-Gerstner, A. (2008 a), ‘Approximate Gauss—Newton methods for optimal state estimation using reduced order models’, Internat. J. Numer. Methods Fluids 56, 1367–1373.CrossRefGoogle Scholar
Lawless, A., Nichols, N., Boess, C. and Bunse-Gerstner, A. (2008 b), ‘Using model reduction methods within incremental four-dimensional variational data assimilation’, Monthly Weather Review 136, 1511–1522.CrossRefGoogle Scholar
Lehtinen, M., Paivarinta, L. and Somersalo, E. (1989), ‘Linear inverse problems for generalized random variables’, Inverse Problems 5, 599–612.CrossRefGoogle Scholar
Lifshits, M. (1995), Gaussian Random Functions, Vol. 322 of Mathematics and its Applications, Kluwer, Dordrecht.CrossRefGoogle Scholar
Livings, D., Dance, S. and Nichols, N. (2008), ‘Unbiased ensemble square root filters’, Physica D: Nonlinear Phenomena 237, 1021–1028.CrossRefGoogle Scholar
Lo, M.éve (1977), Probability Theory I, fourth edn, Vol. 45 of Graduate Texts in Mathematics, Springer, New York.Google Scholar
Loéve, M. (1978), Probability Theory II, fourth edn, Vol. 46 of Graduate Texts in Mathematics, Springer, New York.CrossRefGoogle Scholar
Lorenc, A. (1986), ‘Analysis methods for numerical weather prediction’, Quart. J. Royal Met. Soc. 112, 1177–1194.CrossRefGoogle Scholar
Lubich, C. (2008), From Quantum to Classical Molecular Dynamics: Reduced Models and Numerical Analysis, European Mathematical Society.CrossRefGoogle Scholar
Ma, X., Al-Harbi, M., Datta-Gupta, A. and Efendiev, Y. (2008), ‘Multistage sampling approach to quantifying uncertainty during history matching geological models’, Soc. Petr. Engrg J. 13, 77–87.Google Scholar
Majda, A. and Gershgorin, B. (2008), ‘A nonlinear test model for filtering slow-fast systems’, Comm. Math. Sci. 6, 611–649.Google Scholar
Majda, A. and Grote, M. (2007), ‘Explicit off-line criteria for stable accurate filtering of strongly unstable spatially extended systems’, Proc. Nat. Acad. Sci. 104, 1124–1129.CrossRefGoogle ScholarPubMed
Majda, A. and Harlim, J. (2010), ‘Catastrophic filter divergence in filtering nonlinear dissipative systems’, Comm. Math. Sci. 8, 27–43.Google Scholar
Majda, A., Harlim, J. and Gershgorin, B. (2010), ‘Mathematical strategies for filtering turbulent dynamical systems’, Disc. Cont. Dyn. Sys. To appear.Google Scholar
Mandelbaum, A. (1984), ‘Linear estimators and measurable linear transformations on a Hilbert space’, Probab. Theory Rel. Fields 65, 385–397.Google Scholar
Martin, M., Bell, M. and Nichols, N. (2002), ‘Estimation of systematic error in an equatorial ocean model using data assimilation’, Internat. J. Numer. Methods Fluids 40, 435–444.CrossRefGoogle Scholar
McKeague, I., Nicholls, G., Speer, K. and Herbei, R. (2005), ‘Statistical inversion of south Atlantic circulation in an abyssal neutral density layer’, J. Marine Res. 63, 683–704.CrossRefGoogle Scholar
McLaughlin, D. and Townley, L. (1996), ‘A reassessment of the groundwater inverse problem’, Water Resources Res. 32, 1131–1161.CrossRefGoogle Scholar
Metropolis, N., Rosenbluth, R., Teller, M. and Teller, E. (1953), ‘Equations of state calculations by fast computing machines’, J. Chem. Phys. 21, 1087–1092.CrossRefGoogle Scholar
Meyn, S. P. and Tweedie, R. L. (1993), Markov Chains and Stochastic Stability, Communications and Control Engineering Series, Springer, London.CrossRefGoogle Scholar
Michalak, A. and Kitanidis, P. (2003), ‘A method for enforcing parameter nonnegativity in Bayesian inverse problems with an application to contaminant source identification’, Water Resources Res. 39, 1033.CrossRefGoogle Scholar
Mitchell, T., Buchanan, B., DeJong, G., Dietterich, T., Rosenbloom, P. and Waibel, A. (1990), ‘Machine learning’, Annual Review of Computer Science 4, 417–433.CrossRefGoogle Scholar
Mohamed, L., Christie, M. and Demyanov, V. (2010), ‘Comparison of stochastic sampling algorithms for uncertainty quantification’, Soc. Petr. Engrg J. To appear. http://dx.doi.org/10.2118/119139-PACrossRefGoogle Scholar
Mosegaard, K. and Tarantola, A. (1995), ‘Monte Carlo sampling of solutions to inverse problems’, J. Geophys. Research 100, 431–447.Google Scholar
Neubauer, A. (2009), ‘On enhanced convergence rates for Tikhonov regularization of nonlinear ill-posed problems in Banach spaces’, Inverse Problems 25, #065009.CrossRefGoogle Scholar
Neubauer, A. and Pikkarainen, H. (2008), ‘Convergence results for the Bayesian inversion theory’, J. Inverse and Ill-Posed Problems 16, 601–613.CrossRefGoogle Scholar
Nichols, N. (2003 a), Data assimilation: Aims and basic concepts. In Data Assimilation for the Earth System (Swinbank, R., Shutyaev, V. and Lahoz, W. A., eds), Kluwer Academic, pp. 9–20.CrossRefGoogle Scholar
Nichols, N. (2003 b), Treating model error in 3-D and 4-D data assimilation. In Data Assimilation for the Earth System (Swinbank, R., Shutyaev, V. and Lahoz, W. A., eds), Kluwer Academic, pp. 127–135.CrossRefGoogle Scholar
Nodet, M. (2005), Mathematical modeling and assimilation of Lagrangian data in oceanography. PhD thesis, University of Nice.Google Scholar
Nodet, M. (2006), ‘Variational assimilation of Lagrangian data in oceanography’, Inverse Problems 22, 245–263.CrossRefGoogle Scholar
Oksendal, B. (2003), Stochastic Differential Equations: An Introduction with Applications, sixth edn, Universitext, Springer.CrossRefGoogle Scholar
Orrell, D., Smith, L., Barkmeijer, J. and Palmer, T. (2001), ‘Model error in weather forecasting’, Non. Proc. in Geo. 8, 357–371.CrossRefGoogle Scholar
O'Sullivan, A. and Christie, M. (2006 a), ‘Error models for reducing history match bias’, Comput. Geosci. 10, 405–405.CrossRefGoogle Scholar
O'Sullivan, A. and Christie, M. (2006 b), ‘Simulation error models for improved reservoir prediction’, Reliability Engineering and System Safety 91, 1382–1389.CrossRefGoogle Scholar
Ott, E., Hunt, B., Szunyogh, I., Zimin, A., Kostelich, E., Corazza, M., Kalnay, E., Patil, D. and Yorke, J. (2004), ‘A local ensemble Kalman filter for atmospheric data assimilation’, Tellus A 56, 273–277.CrossRefGoogle Scholar
Palmer, T., Doblas-Reyes, F., Weisheimer, A., Shutts, G., Berner, J. and Murphy, J. (2009), ‘Towards the probabilistic earth-system model’, J. Climate 70, 419–435.Google Scholar
Pikkarainen, H. (2006), ‘State estimation approach to nonstationary inverse problems: Discretization error and filtering problem’, Inverse Problems 22, 365–379.CrossRefGoogle Scholar
Pimentel, S., Haines, K. and Nichols, N. (2008 a), ‘The assimilation of satellite derived sea surface temperatures into a diurnal cycle model’, J. Geophys. Research: Oceans 113, #C09013.Google Scholar
Pimentel, S., Haines, K. and Nichols, N. (2008 b), ‘Modelling the diurnal variability of sea surface temperatures’, J. Geophys. Research: Oceans 113, #C11004.Google Scholar
Ramsay, J. and Silverman, B. (2005), Functional Data Analysis, Springer.CrossRefGoogle Scholar
Reznikoff, M. and Vanden Eijnden, E. (2005), ‘Invariant measures of SPDEs and conditioned diffusions’, CR Acad. Sci. Paris 340, 305–308.CrossRefGoogle Scholar
Richtmyer, D. and Morton, K. (1967), Difference Methods for Initial Value Problems, Wiley.Google Scholar
Roberts, G. and Rosenthal, J. (1998), ‘Optimal scaling of discrete approximations to Langevin diffusions’, J. Royal Statist. Soc. B 60, 255–268.CrossRefGoogle Scholar
Roberts, G. and Rosenthal, J. (2001), ‘Optimal scaling for various Metropolis—Hastings algorithms’, Statistical Science 16, 351–367.CrossRefGoogle Scholar
Roberts, G. and Tweedie, R. (1996), ‘Exponential convergence of Langevin distributions and their discrete approximations’, Bernoulli 2, 341–363.CrossRefGoogle Scholar
Roberts, G., Gelman, A. and Gilks, W. (1997), ‘Weak convergence and optimal scaling of random walk Metropolis algorithms’, Ann. Appl. Probab. 7, 110–120.Google Scholar
Rudin, L., Osher, S. and Fatemi, E. (1992), ‘Nonlinear total variation based noise removal algorithms’, Physica D 60, 259–268.CrossRefGoogle Scholar
Rue, H. and Held, L. (2005), Gaussian Markov Random Fields: Theory and Applications, Chapman & Hall.CrossRefGoogle Scholar
Salman, H., Ide, K. and Jones, C. (2008), ‘Using flow geometry for drifter deployment in Lagrangian data assimilation’, Tellus 60, 321–335.CrossRefGoogle Scholar
Salman, H., Kuznetsov, L., Jones, C. and Ide, K. (2006), ‘A method for assimilating Lagrangian data into a shallow-water equation ocean model’, Monthly Weather Review 134, 1081–1101.CrossRefGoogle Scholar
Sanz-Serna, J. M. and Palencia, C. (1985), ‘A general equivalence theorem in the theory of discretization methods’, Math. Comp. 45, 143–152.CrossRefGoogle Scholar
Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M. and Lenzen, F. (2009), Variational Methods in Imaging, Springer.Google Scholar
Schwab, C. and Todor, R. (2006), ‘Karhunen-Loeve approximation of random fields in domains by generalized fast multipole methods’, J. Comput. Phys. 217, 100–122.CrossRefGoogle Scholar
Shen, Y., Archambeau, C., Cornford, D. and Opper, M. (2008 a), Variational Markov chain Monte Carlo for inference in partially observed nonlinear diffusions. In Proceedings of the Workshop on Inference and Estimation in Probabilistic Time-Series Models (Barber, D., Cemgil, A. T. and Chiappa, S., eds), Isaac Newton Institute for Mathematical Sciences, Cambridge, pp. 67–78.Google Scholar
Shen, Y., Archambeau, C., Cornford, D., Opper, M., Shawe-Taylor, J. and Barillec, R. (2008 b), ‘A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems’, J. Signal Processing Systems. In press (published online).CrossRefGoogle Scholar
Shen, Y., Cornford, D., Archambeau, C. and Opper, M. (2010), ‘Variational Markov chain Monte Carlo for Bayesian inference in partially observed non-linear diffusions’, Comput. Statist. Submitted.CrossRefGoogle Scholar
Smith, A. and Roberts, G. (1993), ‘Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods’, J. Royal Statist. Soc. B 55, 3–23.Google Scholar
Snyder, T., Bengtsson, T., Bickel, P. and Anderson, J. (2008), ‘Obstacles to high-dimensional particle filtering’, Monthly Weather Review 136, 4629–4640.CrossRefGoogle Scholar
Spanos, P. and Ghanem, R. (1989), ‘Stochastic finite element expansion for random media’, J. Engrg Mech. 115, 1035–1053.CrossRefGoogle Scholar
Spanos, P. and Ghanem, R. (2003), Stochastic Finite Elements: A Spectral Approach, Dover.Google Scholar
Spiller, E., Budhiraja, A., Ide, K. and Jones, C. (2008), ‘Modified particle filter methods for assimilating Lagrangian data into a point-vortex model’, Physica D 237, 1498–1506.CrossRefGoogle Scholar
Stanton, L., Lawless, A., Nichols, N. and Roulstone, I. (2005), ‘Variational data assimilation for Hamiltonian problems’, Internat. J. Numer. Methods Fluids 47, 1361–1367.Google Scholar
Stuart, A., Voss, J. and Wiberg, P. (2004), ‘Conditional path sampling of SDEs and the Langevin MCMC method’, Comm. Math. Sci 2, 685–697.CrossRefGoogle Scholar
Talagrand, P. and Courtier, O. (1987), ‘Variational assimilation of meteorological observations with the adjoint vorticity equation I: Theory’, Quart. J. Royal Met. Soc. 113, 1311–1328.CrossRefGoogle Scholar
Tarantola, A. (2005), Inverse Problem Theory, SIAM.Google Scholar
Tierney, L. (1998), ‘A note on Metropolis—Hastings kernels for general state spaces’, Ann. Appl. Probab. 8, 1–9.CrossRefGoogle Scholar
Todor, R. and Schwab, C. (2007), ‘Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients’, IMA J. Numer. Anal. 27, 232–261.CrossRefGoogle Scholar
Uhlmann, G. (2009), Visibility and invisibility. In Invited Lectures, Sixth International Congress on Industrial and Applied Mathematics, ICIAM07 (Jeltsch, R. and Wanner, G., eds), European Mathematical Society, pp. 381–408.Google Scholar
Van Leeuwen, P. (2001), ‘An ensemble smoother with error estimates’, Monthly Weather Review 129, 709–728.2.0.CO;2>CrossRefGoogle Scholar
Van Leeuwen, P. (2003), ‘A variance minimizing filter for large-scale applications’, Monthly Weather Review 131, 2071–2084.2.0.CO;2>CrossRefGoogle Scholar
Van Leeuwen, P. (2009), ‘Particle filtering in geophysical systems’, Monthly Weather Review 137, 4089–4114.CrossRefGoogle Scholar
Vernieres, G., Ide, K. and Jones, C. (2010), ‘Lagrangian data assimilation, an application to the Gulf of Mexico’, Physica D. Submitted.Google Scholar
Vogel, C. (2002), Computational Methods for Inverse Problems, SIAM.CrossRefGoogle Scholar
Vossepoel, F. and Van Leeuwen, P. (2007), ‘Parameter estimation using a particle method: Inferring mixing coefficients from sea-level observations’, Monthly Weather Review 135, 1006–1020.CrossRefGoogle Scholar
Vrettas, M., Cornford, D. and Shen, Y. (2009), A variational basis function approximation for diffusion processes. In Proceedings of the 17th European Symposium on Artificial Neural Networks, D-side publications, Evere, Belgium, pp. 497–502.Google Scholar
Wahba, G. (1990), Spline Models for Observational Data, SIAM.CrossRefGoogle Scholar
Watkinson, L., Lawless, A., Nichols, N. and Roulstone, I. (2007), ‘Weak constraints in four dimensional variational data assimilation’, Meteorologische Zeitschrift 16, 767–776.CrossRefGoogle Scholar
White, L. (1993), ‘A study of uniqueness for the initialization problem for Burgers' equation’, J. Math. Anal. Appl. 172, 412–431.CrossRefGoogle Scholar
Williams, D. (1991), Probability with Martingales, Cambridge University Press, Cambridge.CrossRefGoogle Scholar
Wlasak, M. and Nichols, N. (1998), Application of variational data assimilation to the Lorenz equations using the adjoint method. In Numerical Methods for Fluid Dynamics VI, ICFD, Oxford, pp. 555–562.Google Scholar
Wlasak, M., Nichols, N. and Roulstone, I. (2006), ‘Use of potential vorticity for incremental data assimilation’, Quart. J. Royal Met. Soc. 132, 2867–2886.CrossRefGoogle Scholar
Yu, L. and O'Brien, J. (1991), ‘Variational estimation of the wind stress drag coefficient and the oceanic eddy viscosity profile’, J. Phys. Ocean. 21, 1361–1364.2.0.CO;2>CrossRefGoogle Scholar
Zeitouni, O. and Dembo, A. (1987), ‘A maximum a posteriori estimator for trajectories of diffusion processes’, Stochastics 20, 221–246.CrossRefGoogle Scholar
Zimmerman, D., de Marsily, G., Gotway, C., Marietta, M., Axness, C., Beauheim, R., Bras, R., Carrera, J., Dagan, G., Davies, P., Gallegos, D., Galli, A., Gomez-Hernandez, J., Grindrod, P., Gutjahr, A., Kitanidis, P., Lavenue, A., McLaughlin, D., Neuman, S., RamaRao, B., Ravenne, C. and Rubin, Y. (1998), ‘A comparison of seven geostatistically based inverse approaches to estimate transmissivities for modeling advective transport by groundwater flow’, Water Resources Res. 6, 1373–1413.CrossRefGoogle Scholar
Zuazua, E. (2005), ‘Propagation, observation, control and numerical approximation of waves approximated by finite difference method’, SIAM Review 47, 197–243.CrossRefGoogle Scholar
Zupanski, D. (1997), ‘A general weak constraint applicable to operational 4DVAR data assimilation systems’, Monthly Weather Review 125, 2274–2292.2.0.CO;2>CrossRefGoogle Scholar
Zupanski, M., Navon, I. and Zupanski, D. (2008), ‘The maximum likelihood ensemble filter as a non-differentiable minimization algorithm’, Quart. J. Royal Met. Soc. 134, 1039–1050.CrossRefGoogle Scholar