NWP is an initial/boundary value problem: given an estimate of the present state of the atmosphere (initial conditions) and appropriate boundary conditions, the model simulates (forecasts) the atmospheric evolution. More accurate estimates of initial conditions lead to better forecasts. Currently, operational NWP centers produce initial conditions through a statistical combination of observations and short-range forecasts that account for the uncertainty associated with each source of information. This approach has become known as “data assimilation.” In this chapter, we review early attempts at data assimilation and then introduce the statistical estimation methods that provide a solid foundation for data assimilation. Examples using toy models are provided to illustrate the principles of data assimilation. We then discuss in detail all state-of-the-art data assimilation methods adopted in operational centers, including optimal interpolation, 3D-Var, 4D-Var, ensemble Kalman filter, and hybrid methods. Specifically, we discuss several improvements for the ensemble Kalman filter that make it competitive with 4D-Var. We also discuss Ensemble Forecast Sensitivity to Observations (EFSO), a powerful tool that can estimate the impact of any observations on short-range forecasts, and then we discuss the proactive quality control (PQC) built upon EFSO. We also briefly introduce the non-Gaussian assimilation method particle filters.

Abstract: This chapter provides a broad introduction to Bayesian data assimilation that will be useful to practitioners in interpreting algorithms and results, and for theoretical studies developing novel schemes with an understanding of the rich history of geophysical data assimilation and its current directions. The simple case of data assimilation in a ‘perfect’ model is primarily discussed for pedagogical purposes. Some mathematical results are derived at a high level in order to illustrate key ideas about different estimators. However, the focus of this chapter is on the intuition behind these methods, where more formal and detailed treatments of the data assimilation problem can be found in the various references. In surveying a variety of widely used data assimilation schemes, the key message of this chapter is how the Bayesian analysis provides a consistent framework for the estimation problem and how this allows one to formulate its solution in a variety of ways to exploit the operational challenges in the geosciences.

Three areas where machine learning (ML) and physics have been merging: (a) Physical models can have computationally expensive components replaced by inexpensive ML models, giving rise to hybrid models. (b) In physics-informed machine learning, ML models can be solved satisfying the laws of physics (e.g. conservation of energy, mass, etc.) either approximately or exactly. (c) In forecasting, ML models can be combined with numerical/dynamical models under data assimilation.