This chapter sets out a simplified mathematical framework that allows us to discuss the concept of forecasting and, more generally, prediction. Two key ingredients of prediction are: (i) we have a computational model which we use to simulate the future evolution of the physical process of interest given its current state; and (ii) we have some measurement procedure providing partially observed data on the current and past states of the system. These two ingredients include three different types of error which we need to take into account when making predictions: (i) precision errors in our knowledge of the current state of the physical system; (ii) differences between the evolution of the computational model and the physical system, known as model errors; and (iii) measurement errors in the data that must occur since all measurement procedures are imperfect. Precision and model errors will both lead to a growing divergence between the predicted state and the system state over time, which we attempt to correct with data which have been polluted with measurement errors. This leads to the key question of data assimilation: how can we best combine the data with the model to minimise the impact of these errors, and obtain predictions (and quantify errors in our predictions) of the past, present and future state of the system?

Physical processes and observations

In this book we shall introduce data assimilation algorithms, and we shall want to discuss and evaluate their accuracy and performance. We shall illustrate this by choosing examples where the physical dynamical system can be represented mathematically. This places us in a somewhat artificial situation where we must generate data from some mathematical model and then pretend that we have only observed part of it. However, this will allow us to assess the performance of data assimilation algorithms by comparing our forecasts with the “true evolution” of the system.

We have come to the end of our introduction to ensemble-based forecasting and data assimilation algorithms. The field is rapidly developing and key challenges come from imperfect model scenarios and the high dimensionality of models. While certain particle filters are asymptotically optimal within a perfect model scenario, this picture is much less clear for “real world” data assimilation problems. Here the ensemble Kalman filter has been shown to be both robust and sufficiently accurate for many practical problems. However, it should be kept in mind that most successful applications of the ensemble Kalman filter have arisen against a background of relatively advanced mechanistic models and for problems for which the forecast uncertainties can be approximated by Gaussian distributions. Further challenges arise when, in the absence of accepted mechanistic design principles, only crude empirical models are available and model parameters as well as model states need to be estimated. Ideally, in such cases one might wish to convert empirical models into mechanistic design principles by extracting causalities from data using data assimilation algorithms. These topics are beyond our introductory textbook and we end by inviting the reader to become an active participant in the exciting enterprise called probabilistic forecasting and data assimilation.

Recall from our discussion in the Preface that Laplace's demon possessed (i) a perfect mathematical model of the physical process under consideration, (ii) a snapshot of the state of that process at an arbitrary point in the past or the present, and (iii) infinite computational resources to unravel explicit solutions of the mathematical model. In Chapter 1 we discussed these aspects in a very simplified mathematical setting where physical processes were reduced to one set of mathematical equations (the surrogate physical process) and the mathematical model was represented by a system of difference equations. We also discussed partial and noisy observations of state space as presentations of our knowledge about the surrogate physical process, and briefly touched upon the issue of numerical approximation errors, which arise from putting a mathematical model into algorithmic form amenable to computer implementations. However, contrary to these general considerations made in Chapter 1, we have mostly limited the discussion of data assimilation algorithms in Chapters 6 to 8 to an even more simplified setting where the mathematical model is assumed to be a perfect replica of the surrogate physical process. In other words, the same model has been used both for generating the surrogate physical process and for making predictions about this process. We also generally assumed that the mathematical models come in algorithmic form and discretisation errors were discarded. This setting is called an ideal twin experiment. Within a perfect model setting, uncertainty only arises from incomplete knowledge of the model's initial state. A slight generalisation of this perfect model scenario arises when the surrogate physical process is a particular realisation of the stochastic difference equation which is used for producing forecasts. In that case, forecast uncertainties are caused by the unknown distribution of initial conditions and the unknown realisations of the stochastic contributions to the evolution equation.

In this chapter, we will discuss how to deal with imperfect models and parameter dependent families of imperfect models from a Bayesian perspective. Again our situation will be simplified by the assumption that the underlying reference solution is generated by a known computational model.

Classical mechanics is built upon the concept of determinism. Determinism means that knowledge of the current state of a mechanical system completely determines its future (as well as its past). During the nineteenth century, determinism became a guiding principle for advancing our understanding of natural phenomena, from empirical evidence to first principles and natural laws. In order to formalise the concept of determinism, the French mathematician Pierre Simon Laplace postulated an intellect now referred to as Laplace's demon:

Laplace's demon has three properties: (i) exact knowledge of the laws of nature; (ii) complete knowledge of the state of the universe at a particular point in time (of course, Laplace was writing in the days before knowledge of quantum mechanics and relativity); and (iii) the ability to solve any form of mathematical equation exactly. Except for extremely rare cases, none of these three conditions is met in practice. First, mathematical models generally provide a much simplified representation of nature. In the words of the statistician George Box: “All models are wrong, some are useful”. Second, reality can only be assessed through measurements which are prone to measurement errors and which can only provide a very limited representation of the current state of nature. Third, most mathematical models cannot be solved analytically; we need to approximate them and then implement their solution on a computer, leading to further errors.

So far we have investigated the behaviour of data assimilation algorithms for models with state space dimension Nz ≤ 3. Furthermore, we have investigated the behaviour of ensemble-based data assimilation algorithms for ensemble sizes M ≫ Nz. Classical theoretical results about particle filters discuss convergence to the optimal estimates for M → ∞ within the perfect model scenario and with fixed dimension of state space Nz. While we do not cover these theoretical results in this book, in the previous two chapters we found that the particle filters did indeed converge numerically in terms of their time-averaged RMSEs as the ensemble size M was increased. In fact, the same observation also applies to the ensemble Kalman filters. However, an ensemble Kalman filter does not generally converge to the optimal estimates because of a systematic bias due to the Gaussian assumption in the Bayesian inference step. Nevertheless, the ensemble Kalman filter remains a very popular method in the geosciences since it has much better control over the variance error for small ensemble sizes.

In this chapter, we will apply ensemble-based data assimilation algorithms to models which arise from spatio-temporal processes. More specifically, the models from this chapter can be viewed as spatial and temporal discretisations of partial differential equations (PDEs). For simplicity, we will only consider evolutionary PDEs in one spatial dimension (denoted by x). The dimension of state space of the arising models is inversely proportional to the spatial discretisation parameter Δx and the limit Δx → 0 leads to Nz → ∞. While a rigorous analysis of data assimilation algorithms for M fixed and Nz → ∞ is beyond the scope of this book, we will demonstrate some of the practical problems that arise from such a scenario, which is often referred to as the curse of dimensionality. We will also introduce ensemble inflation and localisation as two practical techniques for making ensemble-based data assimilation techniques applicable to spatio-temporal processes.

In this chapter, we define Bayesian inference, explain what it is used for and introduce some mathematical tools for applying it.

We are required to make inferences whenever we need to make a decision in light of incomplete information. Sometimes the information is incomplete because of partial measurements. For example, it can be difficult determine from a photograph whether the ball crossed the goal line in a football match because the information is incomplete: three-dimensional reality has been projected into a two-dimensional image, and in addition, the picture is only a snapshot in time and we cannot determine the speed and direction in which the ball is moving. The information is partial because we only see the situation from one angle, and at one moment in time. Also, sometimes the information is incomplete because of inaccurate measurements. In our example, this would occur if the photographic image was fuzzy, so we could not even determine exactly where the ball and the goal line are in the photograph. Incomplete information results in uncertainties which make decision-making difficult. However, often we have to make a decision anyway, despite the presence of uncertainty. In this situation, we have to combine the incomplete information with preconceived assumptions. We do this all the time in our daily lives, without even thinking about it; it is called “intuition”. Intuition can be surprisingly successful in some situations, for example we are somehow able to control cars at high speeds on motorways with relatively few accidents. However, in some other situations, intuition can fail miserably. To increase skill in decision making, in predicting the spread of tuberculosis in cattle for example, it is necessary to adopt a more rigorous approach.

When making an inference we have to decide how much to trust the incomplete information and how much to trust prior assumptions (assumptions based on previous experience before taking the measurement). If we take an uncertain measurement that does not match our previous experience, do we assume that the situation is changing and take notice of the measurement, or do we neglect the measurement, assuming that an error was made?