Reverend Thomas Bayes (born circa 1702; died 1761) was the oldest son of Reverend Joshua Bayes, who was one of the first ordained nonconformist ministers in England. Relatively little is known about the personal life of Thomas Bayes. Although he was elected a Fellow of the Royal Society in 1742, his only known mathematical works are two articles published posthumously by his friend Richard Price in 1763. The first dealt with the divergence of the Stirling series, and the second, “An Essay Toward Solving a Problem in the Doctrine of Chances,” is the basis of the paradigm of statistics named for him. His ideas appear to have been independently developed by James Bernoulli in 1713, also published posthumously, and later popularized independently by Pierre Laplace in 1774. In their comprehensive treatise, Bernardo and Smith (1994, p. 4) offer the following summarization of Bayesian statistics:

The field of time series econometrics is an enormous one and it is difficult to do it justice in only a chapter or two. There is a large range of models used with time series data. For univariate time series we have models in the autoregressive moving average (ARMA) class, state space models, models of conditional volatility in the autoregressive conditional heteroscedastic (ARCH) class, and many more. For multivariate time series data, vector autoregressive (VAR) models are the most popular, but many others exist. For both univariate and multivariate specifications there is a growing interest in nonlinear alternatives involving thresholds, regime switching, or structural breaks. In addition, with time series data, new issues arise such as those relating to the presence of nonstationary data (i.e., unit roots and cointegration). In frequentist econometrics, the presence of unit roots and cointegration causes problems since the asymptotic theory underlying, for example, estimators and test statistics, becomes quite complicated. Bayesian econometrics, which conditions upon the observed data, does not run into these problems. However, subtle issues relating to prior elicitation and identification arise when unit root and cointegration issues are relevant. In this chapter and the next, we cannot hope to cover this wide range of models and issues; but rather, we offer a selection of questions to give the reader a flavor for the Bayesian approach to time series methods. In the present chapter, we focus on stationary time series and, thus, do not worry about the complications caused by the presence of nonstationary data.

The past two decades have seen econometrics grow into a vast discipline. Many different branches of the subject now happily coexist with one another. These branches interweave econometric theory and empirical applications and bring econometric method to bear on a myriad of economic issues. Against this background, a guided treatment of the modern subject of econometrics in volumes of worked econometric exercises seemed a natural and rather challenging idea.

The present series, Econometric Exercises, was conceived in 1995 with this challenge in mind. Now, almost a decade later it has become an exciting reality with the publication of the first installment of a series of volumes of worked econometric exercises. How can these volumes work as a tool of learning that adds value to the many existing textbooks of econometrics? What readers do we have in mind as benefiting from this series? What format best suits the objective of helping these readers learn, practice, and teach econometrics? These questions we now address, starting with our overall goals for the series.

Econometric Exercises is published as an organized set of volumes. Each volume in the series provides a coherent sequence of exercises in a specific field or subfield of econometrics. Solved exercises are assembled together in a structured and logical pedagogical framework that seeks to develop the subject matter of the field from its foundations through to its empirical applications and advanced reaches.

As seen in previous chapters, the Bayesian approach typically involves the evaluation of multidimensional integrals such as those appearing in posterior expectations, marginal likelihoods, and predictive densities. In nonconjugate situations, these integrals cannot be calculated analytically. In future chapters we present questions relating to various methods for evaluating such integrals using posterior simulation. However, posterior simulation can be computationally demanding and, hence, Bayesians sometimes use asymptotic approximations. Such approximations are the focus of this chapter.

In most situations the prior p.d.f. does not depend on sample size T, and so in large samples, the likelihood eventually dominates the prior density over their common support. This suggests that consideration of the behavior of the posterior density as T → ∞ may provide useful analytical approximations when T is in fact large. Such “Bayesian asymptotics” differ, however, from frequentist asymptotics in an important way. The thing being approximated in Bayesian asymptotics is conceptually well defined in finite samples, namely the posterior p.d.f. Nuisance parameters can be addressed in a straightforward manner by marginalizing them out of the posterior to obtain the exact marginal posterior p.d.f. for the parameters of interest. In general, there is no frequentist classical sampling distribution of estimators in finite samples that is free of nuisance parameters.

In the same way that sampling distributions of maximum likelihood estimators (MLEs) in regular situations are asymptotically normal, posterior densities are also approximately normal. This result is stated informally in the following theorem.

Many microeconometric applications (including binary, discrete choice, tobit, and generalized tobit analyses) involve the use of latent data. These latent data are unobserved by the econometrician, but the observed choices economic agents make typically impose some type of truncation or ordering among the latent variables.

In this chapter we show how the Gibbs sampler can be used to fit a variety of common microeconomic models involving the use of latent data. In particular, we review how data augmentation [see, e.g., Tanner and Wong (1987), Chib (1992), and Albert and Chib (1993)] can be used to simplify the computations in these models. Importantly, we recognize that many popular models in econometrics are essentially linear regression models on suitably defined latent data. Thus, conditioned on the latent data's values, we can apply all of the known techniques discussed in previous chapters (especially Chapter 10) for inference in the linear regression model. The idea behind data augmentation is to add (or augment) the joint posterior distribution with the latent data itself. In the Gibbs sampler, then, we can typically apply standard techniques to draw from the model parameters given the latent data, and then add an additional step to draw values of the latent data given the model parameters.

To review the idea behind data augmentation in general terms, suppose that we are primarily interested in characterizing the joint posterior p(θ|y) of a k-dimensional parameter vector θ.

The reader has undoubtedly noted that the majority of exercises in previous chapters have included normality assumptions. These assumptions should not be taken as necessarily “correct,” and indeed, such assumptions will not be appropriate in all situations (see, e.g., Exercise 11.13 for a question related to error-term diagnostic checking). In this chapter, we review a variety of computational strategies for extending analyses beyond the textbook normality assumption. The strategies we describe, perhaps ironically, begin with the normal model and proceed to augment it in some way. In the first section, we describe scale mixtures of normals models, which enable the researcher to generalize error distributions to the Student t and double exponential classes (among others). Exercises 15.5 and 15.6 then describe finite normal mixture models, which can accommodate, among other features, skewness and multimodality in error distributions. Importantly, these models are conditionally normal (given values of certain mixing or component indicator variables), and thus standard computational techniques for the normal linear regression model can be directly applied (see, e.g., Exercises 10.1 and 11.8). The techniques described here are thus computationally tractable, flexible, and generalizable. That is, the basic methods provided in this chapter can often be adapted in a straightforward manner to add flexibility to many of the models introduced in previous and future exercises. Some exercises related to mixture models beyond those involving conditional normal sampling distributions are also provided within this chapter.