The aim of this book is to present a detailed, but thoroughly clear and understandable, analysis of the nature and scope of the varieties of negative binomial model that are currently available for use in research. Modeling count data using the standard negative binomial model, termed NB2, has recently become a foremost method of analyzing count response models, yet relatively few researchers or applied statisticians are familiar with the varieties of available negative binomial models, or how best to incorporate them into a research plan.

Note that the Poisson regression model, traditionally considered as the basic count model, is in fact an instance of NB2 – it is an NB2 with a heterogeneity parameter of value 0. We shall discuss the implications of this in the book, as well as other negative binomial models that differ from the NB2. Since Poisson is a variety of the NB2 negative binomial, we may regard the latter as more general and perhaps as even more representative of the majority of count models used in everyday research.

I began writing this second edition of the text in mid-2009, some two years after the first edition of the text was published. Most of the first edition was authored in 2006. In just this short time – from 2006 to 2009/2010 – a number of advancements have been made to the modeling of count data.

I previously indicated that extended Poisson and negative binomial models have generally been developed to solve either a distributional or variance problem arising in the base Poisson and NB2 models. We shall discover later that some extended negative binomial models rely on the NB1 parameterization rather than on the NB2, e.g. fixed-effects negative binomial. But, except for a few models, most negative binomial models used in actual research are based on the NB2 model.

Changes to the negative binomial variance function were considered in the last chapter. In this chapter, we address the difficulties arising when the zero counts in Poisson and NB2 models differ considerably from model distributional assumptions. In the first section we address models that cannot have zero counts.

Zero-truncated count models

Many times we are asked to model count data that structurally exclude zero counts. Hospital length-of-stay data are an example of this type of data. When a patient first enters the hospital, the count begins upon registration, with the length of stay given as 1. There can be no 0 days – unless we are describing patients who do not enter the hospital. This latter situation describes a different type of model where there may be two generating processes – one, for example, for patients who may or may not enter the hospital, and another for patients who are admitted. This type of model will be discussed later.

Risk and 2×2 tables

The notion of risk lays at the foundation of the modeling of counts. In this chapter we discuss the technical meaning of risk and risk ratio, and how to interpret the estimated incidence rate ratios that are displayed in the model output of Poisson and negative binomial regression. In the process, we also discuss the associated relationship of risk difference as well as odds and odds ratios, which are generally understood with respect to logistic regression models.

Risk is an exposure to the chance or probability of some outcome, typically thought of as a loss or injury. In epidemiology, risk refers to the probability of a person or group becoming diseased given some set of attributes or characteristics. In more general terms, the risk that an individual with a specified condition will experience a given outcome is the probability that the individual actually experiences the outcome. It is the proportion of individuals with the risk factor who experience the outcome. In epidemiological terms, risk is therefore a measure of the probability of the incidence of disease. The attribute or condition upon which the risk is measured is termed a risk factor, or exposure.

There are many times when certain data elements are lost, discarded, ignored, or are otherwise excluded from analysis. Truncated and censored models have been developed to deal with these types of data. Both models take two forms: truncation or censoring from below, and truncation or censoring from above. Count model forms take their basic logic from truncated and censored continuous response data, in particular, from Tobit (Amemiya, 1984) and censored normal regression (Goldberger, 1983) respectively.

The traditional parameterization used for truncated and censored count data can be called the econometric or cut-point parameterization. This is the form of model discussed in standard econometric texts and is the form found in current econometric software implementations. I distinguish this from what I term a survival parameterization, the form of which is derived from standard survival models. This parameterization only relates to censored Poisson and censored negative binomial models. I shall first address the more traditional econometric parameterization.

Censored and truncated models – econometric parameterization

Censored and truncated count models are related, with only a relatively minor algorithmic difference between the two. The essential difference relates to how response values beyond a user-defined cut-point are handled. Truncated models eliminate the values altogether; censored models revalue them to the value of the cut-point. In both cases the probability function and log-likelihood functions must be adjusted to account for the change in distribution of the response. We begin by considering truncation.

This chapter takes us into areas that have been primarily addressed in the econometric literature. However, the methods discussed are just as appropriate for research in biostatistics and medical health, and in social and political analysis. Recall that hurdle, zero-inflated, and selection models all partition the response distribution into two components. Zero-inflated models explain excessive zero counts using a binary generating process, which is mixed with a Poisson generating process, including zero. Hurdle models, on the other hand, are also partitioned into binary and count components, but the counts are generated by two separate processes with no overlapping mixture. Zero versus positive counts are formatted into a binary model and a zero-truncated count model generating the positive counts. We found that zero-inflated models employ traditional binary models such as logit and probit regression for understanding the zero counts, whereas the binary component of the hurdle model may be a right censored at 1 count model or a traditional binary model. The finite mixture models discussed in Section 11.1 extend hurdle models in such a manner that processes generating disparate types of counts may be identified.

We follow an examination of finite mixture models with an overview of the methods used to deal with the problem of endogenous predictors. Actually, endogeneity touches on finite mixture models, as well as on count models with unexplained heterogeneity in general, e.g. generalized Waring regression.

What is a negative binomial model?

The negative binomial regression model is a truly unusual statistical model. Typically, those in the statistical community refer to the negative binomial as a single model, as we would in referring to Poisson regression, logistic regression, or probit regression. However, there are in fact several distinct negative binomial models, each of which are referred to as being a negative binomial model. Boswell and Patil (1970) identified 13 separate types of derivations for the negative binomial distribution. Other statisticians have argued that there are even more derivations. Generally, those who are using the distribution as the basis for a statistical model of count data have no idea that the parameterization of the negative binomial they are employing may differ from the parameterization being used by another. Most of the time it makes little difference how the distribution is derived, but, as we shall discover, there are times when it does. Perhaps no other model has such a varied pedigree.

I will provide an outline here of the intertwining nature of the negative binomial. Unless you previously have a solid background in this area of statistics, my overview is not likely to be completely clear. But, as we progress through the book, its logic will become evident.

Bayesian versus frequentist methodology

Bayesian statistics has continually been growing in popularity since the 1980s. Currently there are societies devoted to Bayesian methodology, and the major statistical associations typically have committees devoted to the promotion of Bayesian techniques. In the discipline of astrostatistics, for example, the majority of articles related to the statistical analysis of astronomical data employ some type of Bayesian approach. Not all disciplines are so committed to Bayesian methodology of course, but it is the case that there are a growing number of researchers who are turning to Bayesian modeling. In this chapter we shall present a brief overview of this approach, and give an example of a Bayesian negative binomial model.

Bayesian statistics is named after Thomas Bayes (1702–1761), a British Presbyterian minister and amateur mathematician who was interested in the notion of inverse probability, now referred to as posterior probability. The notion of posterior probability can perhaps best be understood with a simple example.

Suppose that 90% of the students in a statistics course pass. Students who passed are given the symbol P; students who failed are given an F. Therefore, 10% of the students failed the course. Everyone who passed purchased the course text, while only half of the students who failed did so. Let's suppose we observe a student with the course text (assume they purchased it). What is the probability that this student failed? This is an example of posterior probability.

Negative binomial regression has traditionally been used to model otherwise overdispersed count or Poisson data. It is now considered to be the general catchall method used when Poisson data are found to be overdispersed, particularly when the source of overdispersion has not been identified. When we can identify that which gives rise to extra correlation, and hence overdispersion, the basic Poisson and negative binomial algorithms may themselves be further adjusted or enhanced to directly address the identified source of extra correlation. For example, when overdispersion results from an excess of zero counts in the response, an appropriate strategy is to model the data using either a zero-inflated Poisson (ZIP) or zero-inflated negative binomial (ZINB). Employing a hurdle model may also result in a better fit. On the other hand, if the response is structured such that zero counts are not possible, as in hospital length-of-stay data, a zero-truncated Poisson (ZTP) or zero-truncated negative binomial (ZTNB) model may be appropriate.

A variety of alternative models have been developed to address specific facts in the data that give rise to overdispersion. Models dealing with an excess or absence of zeros typically define a mixture that alters the distributional assumptions of the Poisson distribution. Other models are constructed to alter not the probability and log-likelihood distributions, but rather the Poisson and negative binomial variance functions. We discuss these types of models in this chapter.

Overview of count panel models

A basic assumption in the construction of likelihood-based models is that constituent observations are independent. This is a reasonable assumption for perhaps the majority of studies. However, for longitudinal studies this assumption is not feasible, nor does it hold when data are clustered. For example, observations from a study on student drop-outs can be clustered by the type of schools sampled. If the study is related to intervention strategies, schools in affluent suburban, middle-class suburban, middle-class urban, and below-poverty-level schools have more highly correlated strategies within the school type than between types or groups. Likewise, if we have study data taken on a group of individual patients over time (e.g. treatment results obtained once per month for a year), the data related to individuals in the various time periods are likely to be more highly correlated than are treatment results between patients. Any time the data can be grouped into clusters, or panels, of correlated groups, we must adjust the likelihood-based model (based on independent observations) to account for the extra correlation.

We have previously employed robust variance estimators and bootstrapped standard errors when faced with overdispersed count data. Overdispersed Poisson models were replaced by negative binomial models, by adjusting the variance function of the basic Poisson model, or by designing a new log-likelihood function to account for the specific source of the overdispersion.

In this chapter we describe how count response data can be modeled using the NB2 negative binomial regression. NB2 is the traditional parameterization of the negative binomial model, and is the one with which most statisticians are familiar. For this chapter, then, any reference to negative binomial regression will be to the NB2 model unless otherwise indicated.

Poisson versus negative binomial

We have earlier stated that, given the direct relationship in the negative binomial variance between α and the fitted value, μ, the model becomes Poisson as the value of α approaches zero. A negative binomial with α = 0 will not converge because of division by zero, but values close to zero allow convergence. Where α is close to zero, the model statistics displayed in Poisson output are nearly the same as those of a negative binomial.

The relationship can be observed using simulated data. The code below constructs a 50,000 observation synthetic Poisson model with an intercept value of 2 and parameter values of x1 = 0.75 and x2 = 1.25. Each predictor is generated as a synthetic random normal variate. The Poisson data are then modeled using a negative binomial where the value of α is estimated prior to the calculation of parameter estimates and associated statistics.

In this and subsequent chapters we shall discuss the nature and utility of some 25 varieties of negative binomial regression that are useful for modeling count response data. In addition, we examine certain models that are related to the negative binomial family of models. This chapter will primarily be devoted to an examination of the derivation of the negative binomial model and to the two foremost methods of its estimation. We also consider how the probabilities generated from a negative binomial model differ from the Poisson, as well as how they differ among various negative binomial models based on both mean and ancillary parameters.

Varieties of negative binomial

I mentioned that the basic negative binomial model can be enhanced to allow for the modeling of a wide range of count response situations. The Poisson can likewise be enhanced to adjust for data that violate its essential distributional assumptions. In fact, many of the same distributional problems face both Poisson and negative binomial models. We therefore find similar approaches to the handling of such data for both the Poisson and negative binomial. These include models such as zero-inflated Poisson (ZIP), which is directly related to the zero-inflated negative binomial (ZINB). All of the models allow for an estimated variance that exceeds the mean – the principal assumption of the Poisson regression model being that of equidispersion.

This chapter can be considered as a continuation of the previous one. Few real-life Poisson data sets are truly equidispersed. Overdispersion to some degree is inherent to the vast majority of Poisson data. Thus, the real question deals with the amount of overdispersion in a particular model – is it statistically sufficient to require a model other than Poisson? This is a foremost question we address in this chapter, together with how we differentiate between real and apparent overdispersion.

What is overdispersion?

Not all overdispersion is real; apparent overdispersion may sometimes be identified and the model amended to eliminate it. We first address the difference between real and apparent overdispersion, and what can be done about the latter.

1. 1 What is overdispersion?

2. Overdispersion in Poisson models occurs when the response variance is greater than the mean.

3. 2 What causes overdispersion?

4. Overdispersion is caused by positive correlation between responses or by an excess variation between response probabilities or counts. Overdispersion also arises when there are violations in the distributional assumptions of the data, such as when the data are clustered and thereby violate the likelihood independence of observations assumption.

5. 3 Why is overdispersion a problem?

6. Overdispersion may cause standard errors of the estimates to be deflated or underestimated, i.e. a variable may appear to be a significant predictor when it is in fact not significant.

7. […]

Interaction terms play an important role in statistical modeling. However, it is a topic that relates to regression models in general and not specifically to count models. The subject was, therefore, not addressed in the text. On the other hand, there are few sources that provide specifics on how interactions are to be constructed and interpreted for non-linear models, and in particular for count models. This appendix provides a brief overview of interactions and their construction.

Recall that there are a variety of types of predictors. The three most common are binary (1/0), categorical (1, 2,…, k), and continuous. Continuous predictors are often constrained to have positive real numbers as allowable values. In addition, continuous predictors that do not begin at zero or 1 are usually centered, as discussed in Section 6.3.1 the text.

Many researchers believe that the coefficients of interaction terms are interpreted in the same manner as other predictors in a model. However, this is not the case, and in particular it is not the case for interactions in discrete response regression models.

I shall provide a brief overview of how to construct and interpret Binary × Binary, Binary × Continuous, Categorical × Continuous, and Continuous × Continuous interactions. These appear to be the most commonly employed interactions. Understanding these allows the construction and interpretation of any of the other options.

Two general methods are used to estimate count response models: (1) an iteratively re-weighted least squares (IRLS) algorithm based on the method of Fisher scoring, and (2) a full maximum likelihood Newton–Raphson type algorithm. Although the maximum likelihood approach was first used with both the Poisson and negative binomial, we shall discuss it following our examination of IRLS. We do this for strictly pedagogical purposes, which will become evident as we progress.

It should be noted at the outset that IRLS is a type or subset of maximum likelihood which can be used for estimation of generalized linear models (GLM). Maximum likelihood methods in general estimate model parameters by solving the derivative of the model log-likelihood function, termed the gradient, when set to zero. The derivative of the gradient with respect to the parameters is called the Hessian matrix, upon which model standard errors are based. Owing to the unique distributional structure inherent to members of GLM, estimation of model parameters and standard errors can be achieved using IRLS, which in general is a computationally simplier method of maximum likelihood estimation. Both methods are derived, described, and related in this chapter.

Derivation of the IRLS algorithm

The traditional generalized linear models (GLM) algorithm, from the time it was implemented in GLIM (generalized linear interactive modeling) through its current implementations in Stata, R, SAS, SPSS, GenStat, and other statistical software, uses some version of an IRLS estimating algorithm.