In certain data assimilation and optimization problems, gradient information is essentially required. For this purpose, the adjoint model (ADJM) is often employed. The ADJM is a transpose of the tangent linear model (TLM); thus, both are based on the tangent linear approximation of the corresponding nonlinear model. Derivations, formulations, and correctness checks of the TLM and ADJM are described in detail along with the construction of the practical codes of the TLM/ADJM. Practical methods of deriving the ADJM are introduced, using the adjoint operator, Lagrangian multipliers, and chain rules. Uncertainty and validity of the TLM/ADJM are also discussed in terms of nonlinearity and discontinuous physical processes in numerical models. An example of deriving the ADJM is given for Burgers’ equation by comparing the adjoint operator method and the Lagrangian multipliers methods.

Fundamental backgrounds on linear algebra and functional analysis, required for data assimilation, are described in detail.

The theoretical background on sensitivity analysis, especially on the deterministic approach, is described along with definitions on the forward sensitivity coefficient, adjoint sensitivity coefficient, and relative sensitivity coefficient along with examples of their practical applications. Concept, strategies, and applications of adaptive (targeted) observations are discussed, using adjoint sensitivity analysis, singular vectors, the ensemble transform Kalman filter, and conditional nonlinear optimal perturbations. Forecast sensitivity of observations is also discussed as a tool for assessing the impact of observations. In addition, various targeting field programs are introduced.

Overview of discretizing partial differential equations is provided for selected finite difference methods, which is required for construction of discrete tangent linear and adjoint models.

In this chapter the data assimilation problem is introduced as a control theory problem for partial differential equations, with initial conditions, model error, and empirical model parameters as optional control variables. An alternative interpretation of data assimilation as a processing of information in a dynamic-stochastic system is also introduced. Both approaches will be addressed in more detail throughout this book. The historical development of data assimilation has been documented, starting from the early nineteenth-century works by Legendre, Gauss, and Laplace, to optimal interpolation and Kalman filtering, to modern data assimilation based on variational and ensemble methods, and finally to future methods such as particle filters. This suggests that data assimilation is not a very new concept, given that it has been of scientific and practical interest for a long time. Part of the chapter focuses on introducing the common terminologies and notation used in data assimilation, with special emphasis on observation equation, observation errors, and observation operators. Finally, a basic linear estimation problem based on least squares is presented.

Coupled data assimilation is presented in detail. Starting from a coupled modeling system, a classification of coupled data assimilation based on coupling strength is defined. This includes uncoupled, weakly coupled, and strongly coupled data assimilation, and the coupling strength is quantified using mutual information. The most interesting aspects of coupled data assimilation can be related to a strongly coupled system in which the information exchange is maximized. The challenges of strongly coupled data assimilation include the account of the complex control variable and error covariance. The mentioned challenges can considerably increase in realistic high-dimensional applications. Additional issues that can hamper strongly coupled data assimilation include non-Gaussian errors and potentially different spatiotemporal scales of coupled system components. To improve understanding of strongly coupled data assimilation, a simple two-component system is introduced and analyzed. The theoretical assessment is followed by real-world examples of strongly coupled forecast error covariance. Finally, the coupled covariance localization is analyzed and a practical method to address it is described.

Variational data assimilation (VAR) is described in its various forms and their mathematical formulations are explained, including three-dimensional/four-dimensional VAR, first guess at appropriate time (FGAT), Physical-space Statistical Analysis System (PSAS), and incremental approaches. A historical overview of and differences in the calculus of variations and optimal control theory, the root theories on VAR, are also discussed, which are represented by the Euler–Lagrange equations and Pontryagin’s maximum (minimum) principle, respectively. Furthermore, major elements of VAR are reviewed with an emphasis on various formalisms of cost function, including Tikhonov regularization, strong- versus weak-constraint and incremental formulation, and on specification and diagnosis of error covariances, including observation error covariance, background error covariance, and model error covariance. Issues on minimization of the VAR cost function, including gradient, preconditioning, and assimilation period, are also addressed.

Mathematical background and formulation of numerical minimization process are described in terms of gradient-based methods, whose ingredients include gradient, Hessian, directional derivatives, optimality conditions for minimization, Hessian eigensystem, conjugate number of Hessian, and conjugate vectors. Various minimization algorithms, such as the steepest descent method, Newton’s method, conjugate gradient method, and quasi-Newton’s method, are introduced along with practical examples.