Model error in weather and climate forecasting

doi:10.1017/CBO9780511617652.016

Chapter 15 - Model error in weather and climate forecasting

Published online by Cambridge University Press: 03 December 2009

Myles Allen ,

David Frame ,

Jamie Kettleborough and

David Stainforth

Edited by

Tim Palmer and

Renate Hagedorn

Show author details

Myles Allen: Affiliation:
Department of Physics, University of Oxford
David Frame: Affiliation:
Department of Physics, University of Oxford
Jamie Kettleborough: Affiliation:
Space Science and Technology Department, Rutherford Appleton Laboratory, Didcot
David Stainforth: Affiliation:
Department of Physics, University of Oxford
Tim Palmer: Affiliation:
European Centre for Medium-Range Weather Forecasts
Renate Hagedorn: Affiliation:
European Centre for Medium-Range Weather Forecasts

Book contents

Get access

Summary

As if someone were to buy several copies of the morning newspaper to assure himself that what it said was true.

Ludwig Wittgenstein

Introduction

The phrase ‘model error’ means different things to different people, frequently arousing surprisingly passionate emotions. Everyone accepts that all models are wrong, but to some this is simply an annoying caveat on otherwise robust (albeit model-dependent) conclusions, while to others it means that no inference based on ‘electronic storytelling’ can be taken seriously at all. This chapter will focus on how to quantify and minimise the cumulative effect of model ‘imperfections’ (errors by any other name, but we are trying to avoid inflammatory language) that either have not been eliminated because of incomplete observations/understanding or cannot be eliminated because they are intrinsic to the model's structure. We will not provide a recipe for eliminating these imperfections, but rather some ideas on how to live with them. Live with them we must, because no matter how clever model developers, or how fast supercomputers, become, these imperfections will always be with us and represent the hardest source of uncertainty to quantify in a weather or climate forecast (Smith, this volume). This is not meant to underestimate the importance of identifying and improving representations of dynamics (see Hoskins, this volume) or parametrisations (see Palmer, this volume) or existing (and planned) ensemble-based forecast systems (Anderson, Buizza, this volume), merely to draw attention to the fact that our models will always be subject to error or inadequacy (Smith, this volume), and that this fact is especially chronic in those cases where we lack the ability to use conventional verification/falsification procedures (i.e. the climate forecasting problem).

Information

Type: Chapter
Information: Predictability of Weather and Climate , pp. 391 - 427

DOI: https://doi.org/10.1017/CBO9780511617652.016 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Allen, M. R. (1999). Do-it-yourself climate prediction. Nature, 401, 642CrossRef Google Scholar

Allen, M. R. and Ingram, W. J. (2002). Constraints on future climate change and the hydrological cycle. Nature, 419, 224–32CrossRef Google Scholar

Allen, M. R. and Stainforth, D. A. (2002). Towards objective probabilistic climate forecasting. Nature, 419, 228CrossRef Google Scholar

Allen, M. R., Stott, P. A., Mitchell, J. F. B., Schnur, R. and Delworth, T. (2000). Quantifying the uncertainty in forecasts of anthropogenic climate change. Nature, 407, 617–20CrossRef Google Scholar PubMed

Andronova, N. G. and Schlesinger, M. E. (2000). Causes of global temperature changes during the 19th and 20th centuries. Geophys. Res. Lett., 27, 2137–3140CrossRef Google Scholar

Bertrand, J. (1889). Calcul des Probabilities. Gauthier-Villars, ParisGoogle Scholar

Buizza, R., Miller, M. and Palmer, T. N. (1999). Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Quart. J. Roy. Meteor. Soc., 125B, 2887–908CrossRef Google Scholar

Covey, C., AchutaRao, K. M., Lambert, S. J. and , K. E. Taylor (2000). Intercomparison of Present and Future Climates Simulated by Coupled Ocean-Atmosphere GCMs. Technical Report 66. Program for Climate Model Diagnosis and Intercomposition (PCMDI)CrossRef Google Scholar

Craig, P. S., Goldstein, M., Rougier, J. C. and Seheult, A. H. (2001). Bayesian forecasting for complex systems using computer simulators. J. Am. Stat. Assoc., 96, 717–29CrossRef Google Scholar

ECMWF (2002). Talagrand diagram. Online: www.ecmwf.int/products/forecasts/guide/Talagrand_diagram.html

Forest, C. E., Allen, M. R., Stone, P. H. and Sokolov, A. P. (2000). Constraining uncertainties in climate models using climate change detection techniques. Geophys. Res. Lett., 27, 569–72CrossRef Google Scholar

Forest, C. E., Stone, P. H., Sokolov, A. P., Allen, M. R. and Webster, M. D. (2002). Quantifying uncertainties in climate system properties with the use of recent climate observations. Science, 295, 113–17CrossRef Google Scholar PubMed

Fraassen, B. C. (1989). Laws and Symmetry. Clarendon PressCrossRef Google Scholar

Frame, D. J., Booth, B. B. B., Kettleborough, J. A., et al. (2005). Constraining climate forecasts: the role of prior assumptions. Geophys. Res. Lett., 32, doi: 10.1029/2004GL022241CrossRef Google Scholar

Giorgi, F.and Mearns, L. O. (2002). Calculation of average, uncertainty range, and reliability of regional climate changes from AOGCM simulations via the reliability ensemble averaging (REA) method. J. Climate, 15, 1141–582.0.CO;2>CrossRef Google Scholar

Goldstein, M.and Rougier, J. (2004). Probabilistic formulations for transferring inferences from mathematical models to physical systems. SIAM J. Sci. Comput. 26, 467–87CrossRef Google Scholar

Goldstein, M. and Rougier, J. (2005). Probabilistic formulations for transferring inferences from mathematical models to physical systems. SIAM J. Sci. Comput. 26, 467–87CrossRef Google Scholar

Gregory, J. M., Stouffer, R., Raper, S., Rayner, N. and Stott, P. A. (2002). An observationally-based estimate of the climate sensitivity. J. Climate, 15, 3117–212.0.CO;2>CrossRef Google Scholar

Hansen, J., Russell, G., Lacis, A., et al. (1985). Climate response times: dependence on climate sensitivity and ocean mixing. Science, 229, 857–9CrossRef Google Scholar PubMed

Hansen, J. A.and Smith, L. A. (2001). Probabilistic noise reduction. Tellus, 5, 585–98CrossRef Google Scholar

Heidbreder, G. R. (1996).Maximum Entropy and Bayesian Methods. Kluwer Academic PublishersCrossRef Google Scholar

Houghton, J. T., Ding, Y., Griggs, D. J., et al. (eds.) (2001). Climate Change 2001: The Science of Climate Change. Cambridge University PressGoogle Scholar

Howson, C. and Urbach, P. (1993). Scientific Reasoning: The Bayesian Approach. Open CourtGoogle Scholar

Jaynes, E. T.(2003). Probability Theory: The Logic of Science. Cambridge University PressCrossRef Google Scholar

Jewson, S. P.and Caballero, R. (2003). The use of weather forecasts in the pricing of weather derivatives. Meteorol. Appl. 10, 367–76CrossRef Google Scholar

Judd, K. (2003). Nonlinear state estimation, indistinguishable states and the extended Kalman filter. Physica D, 183, 273–81CrossRef Google Scholar

Kass, R. and Wassermann, L. (1996). The selection of prior distribution by formal rules. J. Am. Stat. Assoc., 91, 1343–70CrossRef Google Scholar

Kennedy, M. and O'Hagan, A. (2001). Bayesian calibration of computer models. J. Roy. Stat. Soc. B, 63, 425–64CrossRef Google Scholar

Keynes, J. M. (1921). A Treatise on Probability. Part I. MacmillanGoogle Scholar

Knutti, R., Stocker, T. F., Joos, F. and Plattner, G. K. (2002). Constraints on radiative forcing and future climate change from observations and climate model ensembles. Nature, 416, 719–23CrossRef Google Scholar PubMed

Lea, D. J., Allen, M. R. and Haine, T. W. N. (2002). Sensitivity analysis of the climate of a chaotic system. Tellus, 52A, 523–32Google Scholar

Levitus, S., Antonov, J. I., Boyer, T. P. and Stephens, C. (2000). Warming of the world ocean. Science, 287, 2225–9CrossRef Google Scholar

Levitus, S., Antonov, J. I. and Boyer, T. P. (2005). Warming of the world ocean: 1955–2003. Geophys. Res. Lett., 32, L02604CrossRef Google Scholar

Lorenz, E. N. (1963). Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–412.0.CO;2>CrossRef Google Scholar

Molteni, F., Buizza, R., Palmer, T. N. and Petroliagis, T. (1996). The ECMWF ensemble prediction system: methodology and validation. Quart. J. Roy. Meteor. Soc., 122A, 73–119CrossRef Google Scholar

Moore, A. M. and Kleeman, R. (1999). Stochastic forcing of ENSO by the intraseasonal oscillation. J. Climate, 12, 1199–2202.0.CO;2>CrossRef Google Scholar

Morgan, M. G. and Keith, D. W. (1995). Subjective judgements by climate experts. Environ. Sci. Technol., 29, 468–76Google Scholar

Murphy, J., Sexton, D. M. H., Barnett, D. N., et al. (2004). Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature, 430, 768–72CrossRef Google Scholar

Mylne, K. R., Evans, R. E. and Clark, R. T. (2002). Multi-model multi-analysis ensembles in quasi-operational medium-range forecasting. Quart. J. Roy. Meteor. Soc., 128, 361–84CrossRef Google Scholar

Nolan, D. S. and Farrell, B. F. (1999). The intensification of two-dimensional swirling flows by stochastic asymmetric forcing. J. Atmos. Sci., 56, 3937–622.0.CO;2>CrossRef Google Scholar

Palmer, T. N. (2000). Predicting uncertainty in forecasts of weather and climate. Rep. Prog. Phys., 63, 71–116CrossRef Google Scholar

Palmer, T. N. and Raisanen, J. (2002). Quantifying the risk of extreme seasonal precipitation events in a changing climate. Nature, 415, 512–14CrossRef Google Scholar

Palmer, T. N., Buizza, R., Molteni, F., Chen, Y. Q. and Corti, S. (1994). Singular vectors and the predictability of weather and climate. Philos. Tr. Roy. Soc., S-A 348, 459–75CrossRef Google Scholar

Puri, K., Barkmeijer, J. and Palmer, T. N. (2001). Ensemble prediction of tropical cyclones using targeted diabatic singular vectors. Quart. J. Roy. Meteor. Soc., 127B, 709–31CrossRef Google Scholar

Rosenkrantz, R. D. (1977). Inference, Method and Decision: Towards a Bayesian Philosophy of Science. Reidel, D.CrossRef Google Scholar

Roulston, M. S. and Smith, L. A. (2002). Evaluating probabilistic forecasts using information theory. Mon. Weather Rev., 130, 1653–602.0.CO;2>CrossRef Google Scholar

Schneider, S. H. (2002). Can we estimate the likelihood of climatic changes at 2100?Climatic Change, 52, 441–51CrossRef Google Scholar

Smith, L. A. (1995). Accountability and error in nonlinear forecasting. In Proceedings of the 1995 Seminar on Predictability, pp. 351–68. ECMWFGoogle Scholar

Smith, L. A. (2000). Disentangling uncertainty and error: on the predictability of nonlinear systems. In Nonlinear Dynamics and Statistics, ed. A. I. Meas, pp. 31–64. BirkhauserGoogle Scholar

Smith, L. A. (2002).What might we learn from climate forecasts?Proc. Nat. Acad. Sci., 99, 2487–92CrossRef Google Scholar PubMed

Stainforth, D., Kettleborough, J., Allen, M., Collins, M., Heaps, A. and Murphy, J. (2002). Distributed computing for public-interest climate modeling research. Comput. Sci. Eng., 4, 82–9CrossRef Google Scholar

Stainforth, D. A., Aina, T., Christensen, C., et al. (2004). Evaluating uncertainty in the climate response to changing levels of greenhouse gases. Nature, 433, 403–6CrossRef Google Scholar

Stott, P. A. and Kettleborough, J. A. (2002). Origins and estimates of uncertainty in predictions of twenty-first century temperature rise. Nature, 416, 723–6CrossRef Google Scholar PubMed

Tebaldi, C., Smith, R. L., Nychka, D. and Mearns, L. O. (2005). Quantifying uncertainty in projections of regional climate change: a Bayesian approach to the analysis of multimodel ensembles. J. Climate, 18, 1524–40CrossRef Google Scholar

Toth, Z. and Kalnay, E. (1997). Ensemble forecasting at NCEP and the breeding method. Mon. Weather Rev., 125, 3297–3192.0.CO;2>CrossRef Google Scholar

Weatherford, R. (1982). Philosophical Foundations of Probability Theory. Routledge & Kegan PaulGoogle Scholar

Wigley, T. M. L. and Raper, S. C. B. (1993). Sea level changes due to thermal expansion of the oceans. In Climate and Sea Level Change: Observations, Projections and Implications, ed. Warrick, R. A., Barrow, E. M. and Wigley, T. M. L.. Cambridge University PressGoogle Scholar

Wigley, T. M. L. and Raper, S. C. B. (2001). Interpretation of high projections for global-mean warming. Science, 293, 451–4CrossRef Google Scholar PubMed

Wigley, T. M. L., , C. M. Ammann, Santer, B. D. and Raper, S. C. B. (2005). The effect of climate sensitivity on the response to volcanic forcing. J. Geophys. Res., 110, D09107CrossRef Google Scholar

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the PDF of this chapter is currently unknown and may be updated in the future.