State space methods and unobserved component models are important in a wide range of subjects, including economics, finance, environmental science, medicine and engineering. The conference ‘State space and unobserved component models’, part of the Academy Colloquium programme of the Royal Netherlands Academy of Arts and Sciences, held in Amsterdam from 29 August to 3 September, 2002 brought together researchers from many different areas, but all pursuing a common statistical theme. The papers selected for this volume will give people unfamiliar with state space models a flavour of the work being carried out as well as providing experts with valuable state-of-the-art summaries of different topics.

The conference on state space methods afforded an ideal opportunity to honour Jim Durbin. Jim has been an active researcher in statistics for over fifty years. His first paper, published in 1950, set out the theory of what was to become known as the Durbin–Watson test. He subsequently published in many other areas of statistics, including sampling theory and regression, but over the last fifteen years or so his attention has again been focussed on time series, and in particular on state space models. A steady stream of work has appeared, beginning with the study of the British seat belt law with Andrew Harvey and culminating in the book with Siem Jan Koopman.

It is entirely fitting that the first article in the volume should be by Jim. His clear and lucid style has been an inspiration to generations of students at the London School of Economics – including all three editors of this book – and his paper provides an ideal introduction to unobserved components models.

Abstract

This paper introduces particle filtering to an audience who are more familiar with the Kalman filtering methodology. An example is used to draw comparisons and discuss differences between the two approaches and to motivate some avenues of current research.

Introduction

This paper introduces particle filtering to an audience that is unfamiliar with the literature. It compliments other introductions (Doucet, de Freitas and Gordon 2001) and tutorials (Arulampalam, Maskell, Gordon and Clapp 2002) with a contribution that appeals to intuition and documents an understanding that demystifies what can appear to be a much more complex subject than it truly is.

Particle filtering is a new statistical technique for sequentially updating estimates about a time evolving system as measurements are received. The approach has been developed in parallel by a number of different researchers and so is also known as: the CONDENSATION algorithm (Blake and Isard 1998), the bootstrap filter (Gordon, Salmond and Smith 1993), interacting particle approximations (Dan, Moral and Lyons 1999) and survival of the fittest (Kitagawa 1996). The approach opens the door to the analysis of time series using nonlinear non-Gaussian state space models. While linear Gaussian models can cope with a large variety of systems (e.g. Harvey (1989) and West and Harrison 1997)), nonlinear non-Gaussian models offer an even richer vocabulary with which to describe the evolution of a system and observations of this system.

Particle filtering is based on the idea that uncertainty over the value of a continuous random variable x can be represented using a probability density function, a pdf, p(x).

Abstract

The article explores the extended and strengthened role of state structure when statistical analysis is coupled with the optimisation of decisions. The LQG theory is of course well developed, with its explicit algorithms and complete formal duality of estimation and control. However, the path-integral formulation which is so natural for the state-structured case leads to an elegant formalism which is less well known. The risk-sensitive models give a mild but significant generalisation of the LQG case, with a complete theory and a special simplification in the state-structured case. The application of large-deviation methods, when these are applicable, leads to a direct but radical generalisation of the LQG theory.

State structure in time series analysis

Durbin and Koopman (2001) (and references quoted therein) have eloquently demonstrated the importance of the concept of state in time series analysis. If the underlying model has state structure then this greatly eases inference, but a central (and well-recognised) thesis of this paper is that it also eases decision-making. This enhanced role also requires an enhancement of the concept of state.

The simplest state-structured model is the first-order scalar linear autoregression

where the residuals (‘noise variables’) ∈ t are supposed NID(0, v). From this one obtains an immediate and simple evaluation of the quantity certainly required for inference: the likelihood based on the sample (x1, x2, …, xn).

Abstract

We use the term ‘RegComponent’ model to refer to a regression model whose errors follow an ARIMA component time series model. This is a generalisation of ‘RegARIMA’ models for which the error term follows a standard ARIMA model. Specific forms of RegComponent models (e.g., ‘structural models’) have been used for some time in connection with modelling seasonal time series and for model-based seasonal adjustment, but this paper takes a broader view of RegComponent models. We discuss a general form for RegComponent models along with some theoretical considerations in their treatment regarding likelihood evaluation, likelihood maximisation, forecasting and signal extraction. We also illustrate the use of RegComponent models on some less familiar applications. These include: (i) modelling and seasonal adjustment of time series observed with sampling error; (ii) use of regression models with stochastically time-varying regression parameters and error terms that follow ARIMA or ARIMA component models; and (iii) using components present only occasionally to allow for seasonal or other types of heteroskedasticity. The examples presented illustrate both some of the general model capabilities and some of the capabilities of the US Census Bureau's REGCMPNT computer program, which handles general RegComponent models.

Introduction

This paper explores the use of RegComponent time series models. We use the term ‘RegComponent’ model to refer to a time series model with a regression mean function and an error term that follows an ARIMA component time series model. This is a generalisation of ‘RegARIMA’ models for which the error term follows a standard ARIMA (autoregressive-integrated-moving average) time series model (Box and Jenkins 1976). This paper provides an overview of RegComponent models and illustrates their use with three examples.

Abstract

Resampling the innovations sequence of state space models has proved to be a useful tool in many respects. For example, while under general conditions, the Gaussian MLEs of the parameters of a state space model are asymptotically normal, several researchers have found that samples must be fairly large before asymptotic results are applicable. Moreover, problems occur if any of the parameters are near the boundary of the parameter space. In such situations, the bootstrap applied to the innovation sequence can provide an accurate assessment of the sampling distributions of the parameter estimates. We have also found that a resampling procedure can provide insight into the validity of the model. In addition, the bootstrap can be used to evaluate conditional forecast errors of state space models. The key to this method is the derivation of a reverse-time innovations form of the state space model for generating conditional data sets. We will provide some theoretical insight into our procedures that shows why resampling works in these situations, and we provide simulations and data examples that demonstrate our claims.

Introduction

A very general model that seems to subsume a whole class of special cases of interest is the state space model or the dynamic linear model, which was introduced in Kalman (1960) and Kalman and Bucy (1961). Although the model was originally developed as a method primarily for use in aerospace-related research, it has been applied to modelling data from such diverse fields as economics (e.g. Harrison and Stevens (1976), Harvey and Pierse (1984), Harvey and Todd (1983), Kitagawa and Gersch (1984), Shumway and Stoffer (1982)), medicine (e.g. Jones (1984)) and molecular biology (e.g. Stultz, White and Smith (1993)).

Abstract

This article presents a model-based approach to the investigation of cyclical properties of a time series. It is shown how models for stochastic cycles, both stationary and nonstationary, may be set up and how deterministic cycles emerge as a special case. The Lagrange multiplier principle is used to formulate a test of the null of a deterministic cycle against the alternative of a stochastic cycle and a test of a nonstationary against a stationary cycle. Similar ideas are used to set up a test against the presence of any kind of cycle. All the test statistics have asymptotic distributions that belong to the Cramér-von Mises family under the null. A Wald test against a deterministic cycles is also described as is a test of the null hypothesis that the series contains a permanent cycle. The modelling framework may be extended to include other components, such as trends, and explanatory variables. Finally it is argued that cycles are best detected by fitting models rather than by examining the periodogram.

Introduction

The traditional paradigm of spectral analysis is one in which deterministic cycles are embedded in a stationary indeterministic process. By allowing the initial conditions to be random, these deterministic cycles can be set up so as to be stationary, as in the Wold decomposition. Deterministic cycles are often identified from large periodogram ordinates. Tests of significance of these ordinates may then be carried out, a famous early example being that of Fisher (1929). However, Priestley (1981, Chapter 8) cautioned against identifying cycles from the periodogram since this is normally computed at only T/2 discrete points and these points may not correspond to the frequencies in the series.

Abstract

The Normal linear regression model with natural conjugate prior offers an attractive framework for carrying out Bayesian analysis of non- or semiparametric regression models. The points on the unknown nonparametric regression line can be treated as unobserved components. Prior information on the degree of smoothness of the nonparametric regression line can be combined with the data to yield a proper posterior, despite the fact that the number of parameters in the model is greater than the number of data points. In this paper, we investigate how much prior information is required in order to allow for empirical Bayesian inference about the nonparametric regression line. To be precise, if η is the parameter controlling the degree of smoothness in the nonparametric regression line, we investigate what the minimal amount of nondata information is such that η can be estimated in an empirical Bayes fashion. We show how this problem relates to the issue of estimation of the error variance in the state equation for a simple state space model. Our theoretical results are illustrated empirically using artificial data.

Introduction

The Normal linear regression model with natural conjugate prior provides an attractive framework for conducting Bayesian inference in non- or semi-parametric regression models. Not only does this well-understood model provide simple analytical results, it can easily be used as a component of a more complicated model (e.g. a nonparametric probit or tobit model or nonparametric regression with non-Normal errors). Since Bayesian analysis of more complicated models often requires posterior simulation (e.g. Markov chain Monte Carlo, MCMC, algorithms), having analytical results for the basic nonparametric regression component of the MCMC algorithm offers great computational simplifications.

Abstract

The paper presents a broad general review of the state space approach to time series analysis. It begins with an introduction to the linear Gaussian state space model. Applications to problems in practical time series analysis are considered. The state space approach is briefly compared with the Box–Jenkins approach. The Kalman filter and smoother and the simulation smoother are described. Missing observations, forecasting and initialisation are considered. A representation of a multivariate series as a univariate series is displayed. The construction and maximisation of the likelihood function are discussed. An application to real data is presented. The treatment is extended to non-Gaussian and nonlinear state space models. A simulation technique based on importance sampling is described for analysing these models. The use of antithetic variables in the simulation is considered. Bayesian analysis of the models is developed based on an extension of the importance sampling technique. Classical and Bayesian methods are applied to a real time series.

Introduction to state space models

Basic ideas

The organisers have asked me to provide a broad, general introduction to state space time series analysis. In the pursuit of this objective I will try to make the exposition understandable for those who have relatively little prior knowledge of the subject, while at the same time including some results of recent research. My starting point is the claim that state space models provide an effective basis for practical time series analysis in a wide range of fields including statistics, econometrics and engineering.

Abstract

We use high frequency financial data to proxy, via the realised variance, each day's financial variability. Based on a semiparametric stochastic volatility process, a limit theory shows you can represent the proxy as a true underlying variability plus some measurement noise with known characteristics. Hence filtering, smoothing and forecasting ideas can be used to improve our estimates of variability by exploiting the time series structure of the realised variances. This can be carried out based on a model or without a model. A comparison is made between these two methods.

Introduction

Neil Shephard was fortunate to have Jim Durbin as his supervisor and time series teacher during his first year of graduate studies at the London School of Economics (LSE) in 1986–7. It was just before Jim retired. Jim was very interested in state space models, having recently written the Harvey and Durbin (1986) influential seat-belt case study on structural time series models.

Ole Barndorff-Nielsen's main contact with the research work of Jim Durbin has been with his pathbreaking paper Durbin (1980). Together with the papers by Cox (1980) and Hinkley (1980), this was of key import for the discovery of the general form of the p*-formula for the law of the maximum likelihood estimator and hence the development of the theory that has flown from that formula (see Barndorff-Nielsen and Cox (1994) and the survey paper by Skovgaard (2001)).

Jim's research has had a profound impact on statistics and econometrics. From modelling, estimating and testing time series models to instrumental variables and general estimating equations, through to modern distribution theory, his work has been characterised by energy and inventiveness. He has an original mind.

Abstract

This article surveys some common state space models used in macroeconomics and finance and shows how to specify and estimate these models using the SsfPack library of algorithms implemented in the S−PLUS module S+FinMetrics. Examples include recursive regression models, time varying parameter models, exact autoregressive moving average models and calculation of the Beveridge–Nelson decomposition, unobserved components models, stochastic volatility models, and term structure models.

Introduction

The first version of SsfPack appeared in 1998 and was developed further during the years that the last author was working with Jim Durbin on their 2001 textbook on state space methods. The fact that SsfPack functions are now a part of the S−PLUS software is partly due to Jim Durbin. He convinced Doug Martin that SsfPack would be very beneficial to S−PLUS. Indeed the persuasive arguments of Jim Durbin has initiated the development of SsfPack functions for S−PLUS as part of the S+FinMetrics module. It is therefore an honour for us, the developers of SsfPack for S+FinMetrics, to contribute to this volume with the presentation of various applications in economics and finance that require the use of SsfPack for S+FinMetrics in empirical research.

State space modelling in economics and finance has become widespread over the last decade. Textbook treatments of state space models are given in Harvey (1989, 1993), Hamilton (1994), West and Harrison (1997), Kim and Nelson (1999), Shumway and Stoffer (2000), Durbin and Koopman (2001) and Chan (2002). However, until recently there has not been much flexible software for the statistical analysis of general models in state space form.