Method of Moments

Let X1, … , X n be a sample from a distribution Po that depends on a parameter ranging over some set. The method of moments consists of estimating by the solution of a system of equations

for given functions Thus the parameter is chosen such that the sample moments (on the left side) match the theoretical moments. If the parameter is k-dimensional one usually tries to match k moments in this manner. The choices lead to the method of moments in its simplest form.

Moment estimators are not necessarily the best estimators, but under reasonable conditions they have convergence rate and are asymptotically normal. This is a consequence of the delta method. Write the given functions in the vector notation and let be the vector-valued expectation Then the moment estimator solves the system of equations

For existence of the moment estimator, it is necessary that the vectorbe in the range of the function If is one-to-one, then the moment estimator is uniquely determined as

If is asymptotically normal and is differentiable, then the right side is asymptotically normal by the delta method.

The derivative of at is the inverse of the derivative of eat Because the function is often not explicit, it is convenient to ascertain its differentiability from the differentiability of This is possible by the inverse function theorem. According to this theorem a map that is (continuously) differentiable throughout an open set with nonsingular derivatives is locally one-to-one, is of full rank, and has a differentiable inverse. Thus we obtain the following theorem.

One-Sample V-Statistics

Let X I, … , Xn be a random sample from an unknown distribution. Given a known function h, consider estimation of the “parameter“

In order to simplify the formulas, it is assumed throughout this section that the function h is permutation symmetric in its r arguments. (A given h could always be replaced by a symmetric one.) The statistic h(XI , •.. , Xr) is an unbiased estimator for (), but it is unnatural, as it uses only the first r observations. A U-statistic with kernel h remedies this; it is defined as

where the sum is taken over the set of all unordered subsets f3 of r different integers chosen from ﹛I, … , n﹜. Because the observations are i.i.d., U is an unbiased estimator for () also. Moreover, U is permutation symmetric in Xl, … Xn, and has smaller variance than h(Xb … , Xr). In fact, if X (1) , •.• , X(n) denote the values Xl, … , Xn stripped from their order (the order statistics in the case of real-valued variables), then

Because a conditional expectation is a projection, and projecting decreases second moments, the variance of the U -statistic U is smaller than the variance of the naive estimator h(XJ, … , Xr).

In this section it is shown that the sequence ./ii(U–0) is asymptotically nonnal under the condition that Eh2(X\, … , Xr ) < 00.

12.1 Example. A U-statistic of degree r = 1 is a mean n-I1::7=\h(Xi ). The asserted asymptotic nonnality is then just the central limit theorem.

Introduction

Suppose that we are interested in a parameter (or “functional“) attached to the distribution ofobservations, …,A popular method for finding an estimator is to maximize a criterion function of the type

Here are known functions. An estimator maximizing over is called an M -estimator. In this chapter we investigate the asymptotic behavior of sequences of M -estimators.

Often the maximizing value is sought by setting a derivative (or the set of partial derivatives in the multidimensional case) equal to zero. Therefore, the name M -estimator is also used for estimators satisfying systems of equations of the type

Here are known vector-valued maps. For instance, if is k-dimensional, thentypically has k coordinate functions and (5.2) is shorthand for the system of equations

Even though in many examples is the partial derivative of some function this is irrelevant for the following. Equations, such as (5.2), defining an estimator are called estimating equations and need not correspond to a maximization problem. In the latter case it is probably better to call the corresponding estimators Z-estimators (for zero), but the use of the name M -estimator is widespread.

Sometimes the maximum of the criterion function Mn is not taken or the estimating equation does not have an exact solution. Then it is natural to use as estimator a value that almost maximizes the criterion function or is a near zero. This yields approximate M-estimators or Z-estimators. Estimators that are sufficiently close to being a point of maximum or a zero often have the same asymptotic behavior.

An operator notation for taking expectations simplifies the formulas in this chapter. We write P for the marginal law of the observations which we assume to be identically distributed. Furthermore, we write for the expectation and abbreviate the average. Thus Pn is the empirical distribution: the (random) discrete distribution that puts mass at every of the observations.

Basic Theory

A random vector in is a vector X = (X1, … , Xk) of real random variables. t The distributionfunction of X is the map

A sequence of random vectors Xn is said to converge in distribution to a random vector X if

for every x at which the limit distribution function is continuous. Alternative names are weak convergence and convergence in law. As the last name suggests, the convergence only depends on the induced laws of the vectors and not on the probability spaces on which they are defined. Weak convergence is denoted by if X has distribution L, or a distribution with a standard code, such as N(O, 1), then also by

Let d (x, y) be a distance function on IRk that generates the usual topology. For instance, the Euclidean distance

A sequence of random variables Xn is said to converge in probability to X if for all

This is denoted by. In this notation convergence in probability is the same as

As we shall see, convergence in probability is stronger than convergence in distribution. An even stronger mode of convergence is almost-sure convergence. The sequence Xn is said to converge almost surely to X if

This is denoted by Xn X. Note that convergence in probability and convergence almost surely only make sense if each of Xn and X are defined on the same probability space. For convergence in distribution this is not necessary.

This book describes regression methods for count data, where the response variable is a nonnegative integer. The methods are relevant for analysis of counts that arise in both social and natural sciences.

Despite their relatively recent origin, count data regression methods build on an impressive body of statistical research on univariate discrete distributions. Many of these methods have now found their way into major statistical packages, which has encouraged their application in a variety of contexts. Such widespread use has itself thrown up numerous interesting research issues and themes, which we explore in this book.

The objective of the book is threefold. First, we wish to provide a synthesis and integrative survey of the literature on count data regressions, covering both the statistical and econometric strands. The former has emphasized the framework of generalized linear models, exponential families of distributions, and generalized estimating equations; the latter has emphasized nonlinear regression and generalized method of moment frameworks. Yet between them there are numerous points of contact that can be fruitfully exploited. Our second objective is to make sophisticated methods of data analysis more accessible to practitioners with different interests and backgrounds. To this end we consider models and methods suitable for cross-section, time series, and longitudinal data. Detailed analyses of several data sets as well as shorter illustrations, implemented from a variety of viewpoints, are scattered throughout the book to put empirical flesh on theoretical or methodological discussion. We draw on examples from, and give references to, works in many applied areas.

Introduction

In this chapter we examine methods for modeling count data that are more flexible than those presented in previous chapters. The focus is on the cross-section case, although some of the methods given here have potential extension to time series, multivariate or longitudinal count data, and treatment of sample selection.

One type of flexible modeling is to specify low-order conditional moments of the dependent variable, rather than the entire distribution. This moment-based approach has already been considered extensively in previous chapters. Here we extend it by considering higher-order moments. The emphasis is on the more difficult question of the most efficient use of the moments, with estimators derived using results on optimal GMM.

The core of the chapter considers two basic types of flexible model. First, we consider a sequence of progressively more flexible parametric models, where the underlying parameters in the sequence are tightly specified, for example, equal to a specified function of a linear combination of regressors and parameters.

Second, we consider models in which part of the distribution or general functional form for the moment is tightly specified, but the remainder is flexibly modeled. For example, the conditional mean may be the exponential of the sum of a linear combination of all but one regressor and a flexible function of the remaining regressor. A second example, in which the conditional mean function is specified but the conditional variance is flexible, has already been considered in earlier chapters but is covered in further depth here.

Introduction

The previous chapters have focused on models for cross-section regression on a single count dependent variable. We now turn to models for more general types of data – univariate time series data in this chapter, multivariate cross-section data in Chapter 8, and longitudinal or panel data in Chapter 9.

Count data introduce complications of discreteness and heteroskedasticity. For cross-section data, this leads to moving from the linear model to the Poisson regression model. This model is often too restrictive for real data, which are typically overdispersed. With cross-section data, overdispersion is most frequently handled by leaving the conditional mean unchanged and rescaling the conditional variance. The same adjustment is made regardless of whether the underlying cause of overdispersion is unobserved heterogeneity in a Poisson point process or true contagion leading to dependence in the process.

For time series count data, one can again begin with the Poisson regression model. In this case, however, it is not clear how to proceed if dependence is present. For example, developing even a pure time series count model in which the count in period t, yt, depends only on the count in the previous period, yt−1, is not straightforward, and there are many possible ways to proceed. Even restricting attention to a fully parametric approach, one can specify distributions for yt either conditional on yt−1 or unconditional on yt−1. For count data this leads to quite different models, whereas for continuous data the assumption of joint normality leads to both conditional and marginal distributions that are also normal.

Introduction

This chapter deals with departures from the Poisson regression. One reason for the failure of the Poisson regression is unobserved heterogeneity, which contributes additional randomness. Mixture models obtained by averaging with respect to unobserved heterogeneity generally are not Poisson. A second reason is the failure of the Poisson process assumption and its replacement by a more general stochastic process.

Section 4.2 deals with the negative binomial model. One characterization of this is as a Poisson-gamma mixture. In Section 4.3 we examine the relation between waiting times and counts introduced in Chapter 1. Section 4.4 considers flexible functional forms which are alternatives to the Poisson. Sections 4.5 and 4.6 consider the case in which the range of observed counts is further restricted by either truncation or censoring. Section 4.7 considers an empirically important class of hurdle models that give a special treatment to zero counts. This class combines elements both of truncation and mixtures. Section 4.8 provides a detailed treatment of the finite mixture latent class model that is empirically implemented in Chapter 6. Section 4.9 gives an introduction to estimation by simulation. In the remainder of this section we summarize the motivation underlying the models analyzed in this chapter.

The leading motivation for considering parametric distributions other than the Poisson is that they have the potential to accommodate features of data that are inconsistent with the Poisson assumption.

Introduction

This chapter is intended to provide a self-contained treatment of basic cross-section count data regression analysis. It is analogous to a chapter in a standard statistics text that covers both homoskedastic and heteroskedastic linear regression models.

The most commonly used count models are Poisson and negative binomial. For readers interested only in these models it is sufficient to read sections 3.1 through 3.5, along with preparatory material in sections 1.2 and 2.2 in previous chapters.

Additional regression models for cross-section count data are given in the remainder of Chapter 3, most notably the ordered probit and logit models. These additional models generally ignore the count nature of the data. Still further models, such as the hurdle model, which do explicitly treat the data as count data, are given in Chapter 4. Some model diagnostic methods are presented in Chapter 3, but most are deferred to Chapter 5.

As indicated in Chapter 2, the properties of an estimator vary with the assumptions made on the dgp. By correct specification of the conditional mean or variance or density, we mean that the functional form and explanatory variables in the specified conditional mean or variance or density are those of the dgp.

Introduction

The well-known bivariate linear errors-in-variables regression model with additive measurement errors in both variables provides one benchmark for nonlinear errors-in-variables models. The standard textbook treatment of the errors-invariables case emphasizes the attenuation result, which says that the estimated least squares estimate of the slope parameter is downward-biased if both variables are subject to measurement error. The essential problem lies in the correlation between the observed explanatory variable and the measurement error. This leads to distorted inferences about the role of the covariate. Although this result does not always extend to general cases, such as a linear model with two or more covariates measured with error, it is usually of interest to consider whether a similar attenuation bias exists generally in nonlinear models (Carroll et al., 1995).

There are important similarities and differences between measurement errors in nonlinear and linear models. First, in nonlinear models it may be more natural to allow measurement errors to enter multiplicatively rather than additively. Second, models in which the measurement errors are confined to the count variable, rather than covariates, are of considerable interest. Third, the direction of measurement errors in count models is sometimes strongly suspected from a priori analysis, which permits stronger conclusions.

Given these motivations, this chapter considers estimation and inference in the presence of measurement errors in exposure time, errors due to underreporting and misclassification of events. Such errors are shown to have important consequences for model identification, specification, estimation, and testing.

Introduction

In this chapter we consider regression models for an m-dimensional vector of jointly distributed, and in general correlated, random variables y = (y1, y2, …, ym), a subset of which are event counts. A special case is if m = 2, y1 is a count, and y2 is either discrete or continuous. Multivariate data appear in three contexts in this book. The first is basic cross-section, which is the main subject of this chapter. The second is longitudinal data with repeated measures over time on the same variable, leading to special correlation structure handled in Chapter 9. The third is the context of multivariate cross-section data with endogeneity or feedback from yj to yk, dealt with in Chapter 10. There are other forms of multivariate data, such as multivariate time series analogs of Gaussian vector autoregressions, that we do not cover.

Multivariate linear Gaussian models are widely used, but multivariate nonlinear, non-Gaussian models are less common. Fully parametric approaches based on the joint distribution of non-Gaussian vector y, given a set of covariates x, are difficult to apply because analytically and computationally tractable expressions for such joint distributions are available for special cases only. Consequently, it is more convenient to analyze models that are of interest in specific situations.

Multivariate cross-section count models arise in several different settings. The first is that in which several related events are measured as counts and the joint distribution of several counts is required. These models are analogous to the seemingly unrelated regressions model.

This book is concerned with models of event counts. An event count refers to the number of times an event occurs, for example the number of airline accidents or earthquakes. An event count is the realization of a nonnegative integer-valued random variable. A univariate statistical model of event counts usually specifies a probability distribution of the number of occurrences of the event known up to some parameters. Estimation and inference in such models are concerned with the unknown parameters, given the probability distribution and the count data. Such a specification involves no other variables and the number of events is assumed to be independently identically distributed (iid). Much early theoretical and applied work on event counts was carried out in the univariate framework. The main focus of this book, however, is regression analysis of event counts.

The statistical analysis of counts within the framework of discrete parametric distributions for univariate iid random variables has a long and rich history (Johnson, Kotz, and Kemp, 1992). The Poisson distribution was derived as a limiting case of the binomial by Poisson (1837). Early applications include the classic study of Bortkiewicz (1898) of the annual number of deaths from being kicked by mules in the Prussian army. A standard generalization of the Poisson is the negative binomial distribution. It was derived by Greenwood and Yule (1920), as a consequence of apparent contagion due to unobserved heterogeneity, and by Eggenberger and Polya (1923) as a result of true contagion.

Introduction

The general modeling approaches most often used in count data analysis – likelihood-based, generalized linear models, and moment-based – are presented in this chapter. Statistical inference for these nonlinear regression models is based on asymptotic theory, which is also summarized.

The models and results vary according to the strength of the distributional assumptions made. Likelihood-based models and the associated maximum likelihood estimator require complete specification of the distribution. Statistical inference is usually performed under the assumption that the distribution is correctly specified.

A less parametric analysis assumes that some aspects of the distribution of the dependent variable are correctly specified while others are not specified, or if specified are potentially misspecified. For count data models considerable emphasis has been placed on analysis based on the assumption of correct specification of the conditional mean, or on the assumption of correct specification of both the conditional mean and the conditional variance. This is a nonlinear generalization of the linear regression model, where consistency requires correct specification of the mean and efficient estimation requires correct specification of the mean and variance. It is a special case of the class of generalized linear models, widely used in the statistics literature. Estimators for generalized linear models coincide with maximum likelihood estimators if the specified density is in the linear exponential family. But even then the analytical distribution of the same estimator can differ across the two approaches if different second moment assumptions are made.