Saddlepoint approximations for conditional densities and mass functions are presented that make use of two saddlepoint approximations, one for the joint density and another for the marginal. In addition, approximations for univariate conditional distributions are developed. These conditional probability approximations are particularly important because alternative methods of computation, perhaps based upon simulation, are likely to be either very difficult to implement or not practically feasible. For the roles of conditioning in statistical inference, see Reid (1995).

Conditional saddlepoint density and mass functions

Let (X, Y) be a random vector having a nondegenerate distribution in ℜm with dim(X) = mx, dim(Y) = my, and mx + my = m.With all components continuous, suppose there is a joint density f (x, y) with support (x, y) ε X ε ⊆ ℜm. For lattice components on Im, assume there is a joint mass function p (j, k) for (j, k) ε X ε ⊆ Im. All of the saddlepoint procedures discussed in this chapter allowboth X and Y to mix components of the continuous and lattice type.

Approximations are presented below in the continuous setting by using symbols f, x, and y. Their discrete analogues simply amount to replacing these symbols with p, j, and k. We shall henceforth concentrate on the continuous notation but also describe the methods as if they were to be used in both the continuous and lattice settings. Any discrepancies that arise for the lattice case are noted.

Sometimes nonparametric bootstrap inference can be performed by using a saddlepoint inversion to replace the resampling that is normally required. This chapter considers this general concept and pursues the extent to which the general procedure can implemented.

The main set of examples have been taken from Butler and Bronson (2002) and Bronson (2001). They involve bootstrap inference for first passage characteristics and time to event distributions in the semi-Markov processes discussed in chapter 13. Saddlepoint approximations of nonparametric bootstrap estimates are given for the mean and standard deviation of first passage and as well as for the density, survival, and hazard functions associated with the passage events. Thus saddlepoint approximations become estimates for these moments and functions and represent approximations to the estimates that would normally be provided were a large amount of resampling to be implemented with “single” bootstrap resampling. In performing these computations however, no resampling has been used to compute the estimates, only saddlepoint inversions. The saddlepoint estimates utilize data from the process through an empirical estimate T (s) of the unknown system transmittance matrix T (s). Empirical transmittance T (s) is used in place of T (s) when implementing the saddlepoint inversions.

In addition to bootstrap estimates, confidence intervals for the moments and confidence bands for the functions may be computed by using the double bootstrap. Often the resampling demands inherent in the double bootstrap place its implementation beyond the range of practical computing (Booth and Presnell, 1998).

Saddlepoint methods are applied to many of the commonly used test statistics MANOVA. The intent here is to highlight the usefulness of saddlepoint procedures in providing simple and accurate probability computations in classical multivariate normal theory models. Power curves and p-values are computed for tests in MANOVA and for tests of covariance. Convenient formulae are given which facilitate the practical implementation of the methods.

The null distributions of many multivariate tests are easily computed by using their Mellin transforms which admit saddlepoint approximations leading to very accurate pvalues. The first section concentrates on the four important tests of MANOVA. Very accurate p-value computations are suggested for (i) the Wilks likelihood ratio for MANOVA, (ii) the Bartlett–Nanda–Pillai trace statistic, (iii) Roy's largest eigenvalue test, and (iv) the Lawley-Hotelling trace statistic. Saddlepoint methods for Wilks' likelihood ratio test were introduced in Srivastava and Yau (1989) and Butler et al. (1992a); approximations for (ii) were introduced in Butler et al. (1992b) and were also discussed in section 10.4; p-values for Roy's test do not use saddlepoint methods and are based on the development in Butler and Paige (2007) who extend the results of Gupta and Richards (1985); and p-values for Lawley–Hotelling trace are based on numerical inversion of its Laplace transform in work to be published by Butler and Paige.

The second section considers tests for covariance patterns. Saddlepoint approximations for p-values are suggested for the following tests: the likelihood ratio tests for (ii) block independence, (iii) sphericity, (iv) an intraclass correlation structure; and (v) the Bartlett–Box test for equal covariances. These applications were developed in Butler et al. (1993), and Booth et al. (1995).

Exponential families provide the context for many practical applications of saddlepoint methods including those using generalized linear models with canonical links. More traditional exponential family applications are considered in this chapter and some generalized linear models are presented in chapter 6. All these settings have canonical sufficient statistics whose joint distribution is conveniently approximated using saddlepoint methods. Such approximate distributions become especially important in this context because statistical methods are almost always based on these sufficient statistics. Furthermore, these distributions are most often intractable by other means when considered to the degree of accuracy that may be achieved when using saddlepoint approximations. Thus, in terms of accuracy, both in this context and in the many others previously discussed, saddlepoint methods are largely unrivaled by all other analytical approximations.

The subclass of regular exponential families of distributions was introduced in section 2.4.5 by “tilting” a distribution whose MGF converges in the “regular” sense. Tilting combined with Edgeworth expansions provide the techniques for an alternative derivation of saddlepoint densities that is presented in this chapter. This alternative approach also gives further understanding as to why the saddlepoint density achieves its remarkable accuracy. Both unconditional and conditional saddlepoint approximations are considered for the canonical sufficient statistics in a regular exponential family. The computation of p-values for various likelihood ratio tests and the determination of some uniformly most powerful unbiased tests are some of the practical examples that are considered.

Regular exponential families

Definitions and examples

Let z be the data and denote Z as the random variable for the data before observation.

When the joint MGF of (X, Y) is available, the conditional MGF of Y given X = x can be approximated by using the sequential saddlepoint method. Apart from providing conditional moments, this approximate conditional MGF may also serve as a surrogate for the true conditional MGF with saddlepoint methods. In such a role, it can be the input into a singlesaddlepoint method, such as the Lugananni and Rice approximation, to give an approximate conditional distribution. Thus, the resulting sequential saddlepoint approximation to Pr(Y = y|X = x) provides an alternative to the double-saddlepoint methods of sections 4.1-4.2.

Computation of a p-value for the Bartlett–Nandi–Pillai trace statistic in MANOVA provides a context in which the sequential saddlepoint approximation succeeds with high accuracy but the competing double-saddlepoint CDF approximation fails. Among the latter methods, only numerical integration of the double-saddlepoint density successfully replicates the accuracy of the sequential saddlepoint CDF approximation; see Butler et al. (1992b).

Another highly successful application of sequential saddlepoint methods occurs when approximating the power function of Wilks' likelihood ratio test inMANOVA. This example is deferred to section 11.3.1.

Sequential saddlepoint approximation

Suppose (X, Y) is a m-dimensional random vector with known joint CGF K(s, t) where s and t are the respective components. The goal is to use the joint CGF to determine conditional probabilities and moments of Y given X = x.

The p* densitywas introduced in Barndorff-Nielsen (1980, 1983) and has been prominently featured as an approximation for the density of the maximum likelihood estimate (MLE). Its development from saddlepoint approximations and its role in likelihood inference for regular exponential families are discussed in section 7.1. Section 7.2 considers group transformation models, such as linear regression and location and scale models. In this setting the p* formula provides the conditional distributions ofMLEsgiven a maximal ancillary statistic and agrees with expressions developed by Fisher (1934) and Fraser (1979). In curved exponential families, the p* formula also approximates the conditional distribution of MLEs given certain approximate ancillaries. This development is explored in section 7.3 with a more detailed discussion given for the situation in which the ancillaries are affine.

The p* density in regular exponential families

In the context of a regular exponential family, the p* density is the normalized saddlepoint density for the MLE. In its unnormalized form in the continuous setting, p* is simply a Jacobian transformation removed from the saddlepoint density of the canonical sufficient statistic. Consider the regular exponential family in (5.1) with canonical parameter θ ε ε ⊆ ℜm and canonical sufficient statistic X. Statistic X is also the MLE û for the mean parameterization μ = c′(θ) as discussed in section 5.1.2.

Among the various tools that have been developed for use in statistics and probability over the years, perhaps the least understood and most remarkable tool is the saddlepoint approximation. It is remarkable because it usually provides probability approximations whose accuracy is much greater than the current supporting theory would suggest. It is least understood because of the difficulty of the subject itself and the difficulty of the research papers and books that have been written about it. Indeed this lack of accessibility has placed its understanding out of the reach of many researchers in both statistics and its related subjects.

The primary aim of this book is to provide an accessible account of the theory and application of saddlepoint methods that can be understood by the widest possible audience. To do this, the book has been written at graduated levels of difficulty with the first six chapters forming the easiest part and the core of the subject. These chapters use little mathematics beyond the difficulty of advanced calculus (no complex variables) and should provide relatively easy reading to first year graduate students in statistics, engineering, and other quantitative fields. These chapters would also be accessible to senior-level undergraduate mathematics and advanced engineering majors. With the accessibility issue in mind, the first six chapters have been purposefully written to address the issue and should assure that the widest audience is able to read and learn the subject.

The presentation throughout the book takes the point of view of users of saddlepoint approximations; theoretical aspects of the methods are also covered but are given less emphasis than they are in journal articles.

Statistical inference is considered in five practical settings. In each application, saddlepoint approximations offer an innovative approach for computing p-values, mid-p-values, power functions, and confidence intervals. As case studies, these five settings motivate additional considerations connected with both the saddlepoint methodology and theory that are used for making statistical inference. Each additional topic is addressed as it arises through its motivating example.

The first application concerns logistic regression, with an emphasis on the determination of LD50. p-value computation, test inversion to determine confidence intervals, and the notion of mid-p-values are considered. Saddlepoint treatment of prospective and retrospective analyses are also compared.

The second application deals with common odds ratio estimation. Both single- and double-saddlepoint methods are applicable and these two approaches are compared with several examples. Power function computations and their properties are discussed in connection with both the single- and double-saddlepoint approaches. Time series data may also be analyzed and an example dealing with an autoregression of Poisson counts is given.

The final two applications are concerned with modeling nonhomogeneous Poisson processes, and inference with data that has been truncated. The former example reconsiders the Lake Konstanz data analyzed in Barndorff-Nielsen and Cox (1979) and the latter deals with the truncation of binomial counts that occur with genetic data.

The general use of saddlepoint methods for testing purposes in exponential families was introduced in Davison (1988), however these applications had already been anticipated much earlier in the discussion by Daniels (1958) concerning the logistic setting as well as in Barndorff-Nielsen and Cox (1979). Test inversion to determine confidence intervals was first considered in Pierce and Peters (1992).