Approximations to continuous univariate CDFs of MLEs in curved exponential and transformation families have been derived in Barndorff-Nielsen (1986, 1990, 1991) and are often referred to as r * approximations. These approximations, along with their equivalent approximations of the Lugannani and Rice/Skovgaard form, are presented in the next two chapters. Section 8.2 considers the conditional CDF for the MLE of a scalar parameter given appropriate ancillaries. The more complex situation that encompasses a vector nuisance parameter is the subject of chapter 9.

Other approaches to this distribution theory, aimed more toward p-value computation, are also presented in section 8.5. Fraser and Reid (1993, 1995, 2001) and Fraser et al. (1999a) have suggested an approach based on geometrical considerations of the inference problem. In this approach, explicit ancillary expressions are not needed which helps to simplify the computational effort. Along these same lines, Skovgaard (1996) also offers methods forCDF approximation that are quite simple computationally. Specification of ancillaries is again not necessary and these methods are direct approximations to the procedures suggested by Barndorff-Nielsen above.

Expressions for these approximate CDFs involve partial derivatives of the likelihood with respect the parameter but also with respect to the MLE and other quantities holding the approximate ancillary fixed. The latter partial derivatives are called sample space derivatives and can be difficult to compute. An introduction to these derivatives is given in the next section and approximations to such derivatives, as suggested in Skovgaard (1996), are presented in appropriate sections.

The ratio R = U/V of two random variables U and V, perhaps dependent, admits to saddlepoint approximation through the joint MGF of (U, V). If V > 0 with probability one, then the Lugannani and Rice approximation may be easily applied to approximate the associated CDF. Saddlepoint density approximation based on the joint MGF uses the Geary (1944) representation for its density. This approach was first noted in Daniels (1954, –9) and is discussed in section 12.1 below.

The ratio R is the root of the estimating equation U − RV = 0 and the distribution theory for ratios can be generalized to consider distributions for roots of general estimating equations. The results of section 12.1 are subsumed into the more general discussion of section 12.2 that provides approximate distributions for roots of general estimating equations. Saddlepoint approximations for these roots began in the robustness literature where M-estimates are the roots of certain estimating equations and the interestwas in determining their distributions when sample sizes are small. Hampel (1973), Field and Hampel (1982), and Field (1982) were instrumental in developing this general approach.

Saddlepoint approximation for a vector of ratios, such as for example (R1, R2, R3) = {U1/V, U2/V, U3/V}, is presented in section 12.3 and generalizes the results of Geary (1944). An important class of such examples to be considered includes vector ratios of quadratic forms in normal variables. A particularly prominent example in times series which is treated in detail concerns approximation to the joint distribution for the sequence of lag correlations comprising the serial autocorrelation function.

In engineering reliability and multistate survival analysis, the machine or patient is viewed as a stochastic system or process which passes from one state to another over time. In many practical settings, the state space of this system is finite and the dynamics of the process are modelled as either a Markov process or alternatively as a semi-Markov process if aspects of the Markov assumption are unreasonable or too restrictive.

This chapter gives the CGFs connected with first passage times in general semi-Markov models with a finite number of states as developed in Butler (1997, 2000, 2001). Different formulae apply to different types of passage times, however all of these CGF formulae have one common feature: they are all represented as explicit matrix expressions that are ratios of matrix determinants. When inverting these CGFs using saddlepoint methods, the required first and second derivatives are also explicit so that the whole saddlepoint inversion becomes a simple explicit computation. These ingredients when used with the parametric plot routine in Maple, lead to explicit plots for the first passage density or CDF that completely avoid the burden of solving the saddlepoint equation.

Distributions for first passage or return times to a single absorbing state are considered in section 13.2 along with many examples derived from various stochastic models that arise in engineering reliability and queueing theory. Distributions for first passage to a subset of states require a different CGF formula which is developed in section 13.3. Passage times for birth and death processes can be approached from a slightly different perspective due to the movement restriction to neighboring states.

Up to now, all of the saddlepoint formulas have involved the univariate normal density function ø(z) and its CDF Φ(z). These expressions are called normal-based saddlepoint approximations and, for the most part, they serve the majority of needs in a wide range of applications. In some specialized settings however, greater accuracy may be achieved by using saddlepoint approximations developed around the idea of using a different distributional base than the standard normal.

This chapter presents saddlepoint approximations that are based on distributions other than the standard normal distribution. Suppose this base distribution has density function λ(z) and CDF Λ(z) and define the saddlepoint approximations that use this distribution as (λ, Λ)-based saddlepoint approximations. Derivations and properties of such approximations are presented in section 16.1 along with some simple examples. Most of the development below is based on Wood et al. (1993).

Most prominent among the base distributions is the inverse Gaussian distribution. The importance of this base is that it provides very accurate probability approximations for certain heavy-tailed distributions in settings for which the usual normal-based saddlepoint approximations are not accurate. These distributions include various first passage times in random walks and queues in which the system is either unstable, so the first passage distribution may be defective, or stable and close to the border of stability. Examples include the first return distribution to state 0 in a random walk that is null persistent or close to being so in both discrete and continuous time. A second example considers a predictive Bayesian analysis for a Markov queue with infinite buffer capacity.

Saddlepoint approximations for conditional densities and mass functions are presented that make use of two saddlepoint approximations, one for the joint density and another for the marginal. In addition, approximations for univariate conditional distributions are developed. These conditional probability approximations are particularly important because alternative methods of computation, perhaps based upon simulation, are likely to be either very difficult to implement or not practically feasible. For the roles of conditioning in statistical inference, see Reid (1995).

Conditional saddlepoint density and mass functions

Let (X, Y) be a random vector having a nondegenerate distribution in ℜm with dim(X) = mx, dim(Y) = my, and mx + my = m.With all components continuous, suppose there is a joint density f (x, y) with support (x, y) ε X ε ⊆ ℜm. For lattice components on Im, assume there is a joint mass function p (j, k) for (j, k) ε X ε ⊆ Im. All of the saddlepoint procedures discussed in this chapter allowboth X and Y to mix components of the continuous and lattice type.

Approximations are presented below in the continuous setting by using symbols f, x, and y. Their discrete analogues simply amount to replacing these symbols with p, j, and k. We shall henceforth concentrate on the continuous notation but also describe the methods as if they were to be used in both the continuous and lattice settings. Any discrepancies that arise for the lattice case are noted.

Sometimes nonparametric bootstrap inference can be performed by using a saddlepoint inversion to replace the resampling that is normally required. This chapter considers this general concept and pursues the extent to which the general procedure can implemented.

The main set of examples have been taken from Butler and Bronson (2002) and Bronson (2001). They involve bootstrap inference for first passage characteristics and time to event distributions in the semi-Markov processes discussed in chapter 13. Saddlepoint approximations of nonparametric bootstrap estimates are given for the mean and standard deviation of first passage and as well as for the density, survival, and hazard functions associated with the passage events. Thus saddlepoint approximations become estimates for these moments and functions and represent approximations to the estimates that would normally be provided were a large amount of resampling to be implemented with “single” bootstrap resampling. In performing these computations however, no resampling has been used to compute the estimates, only saddlepoint inversions. The saddlepoint estimates utilize data from the process through an empirical estimate T (s) of the unknown system transmittance matrix T (s). Empirical transmittance T (s) is used in place of T (s) when implementing the saddlepoint inversions.

In addition to bootstrap estimates, confidence intervals for the moments and confidence bands for the functions may be computed by using the double bootstrap. Often the resampling demands inherent in the double bootstrap place its implementation beyond the range of practical computing (Booth and Presnell, 1998).

Saddlepoint methods are applied to many of the commonly used test statistics MANOVA. The intent here is to highlight the usefulness of saddlepoint procedures in providing simple and accurate probability computations in classical multivariate normal theory models. Power curves and p-values are computed for tests in MANOVA and for tests of covariance. Convenient formulae are given which facilitate the practical implementation of the methods.

The null distributions of many multivariate tests are easily computed by using their Mellin transforms which admit saddlepoint approximations leading to very accurate pvalues. The first section concentrates on the four important tests of MANOVA. Very accurate p-value computations are suggested for (i) the Wilks likelihood ratio for MANOVA, (ii) the Bartlett–Nanda–Pillai trace statistic, (iii) Roy's largest eigenvalue test, and (iv) the Lawley-Hotelling trace statistic. Saddlepoint methods for Wilks' likelihood ratio test were introduced in Srivastava and Yau (1989) and Butler et al. (1992a); approximations for (ii) were introduced in Butler et al. (1992b) and were also discussed in section 10.4; p-values for Roy's test do not use saddlepoint methods and are based on the development in Butler and Paige (2007) who extend the results of Gupta and Richards (1985); and p-values for Lawley–Hotelling trace are based on numerical inversion of its Laplace transform in work to be published by Butler and Paige.

The second section considers tests for covariance patterns. Saddlepoint approximations for p-values are suggested for the following tests: the likelihood ratio tests for (ii) block independence, (iii) sphericity, (iv) an intraclass correlation structure; and (v) the Bartlett–Box test for equal covariances. These applications were developed in Butler et al. (1993), and Booth et al. (1995).

Exponential families provide the context for many practical applications of saddlepoint methods including those using generalized linear models with canonical links. More traditional exponential family applications are considered in this chapter and some generalized linear models are presented in chapter 6. All these settings have canonical sufficient statistics whose joint distribution is conveniently approximated using saddlepoint methods. Such approximate distributions become especially important in this context because statistical methods are almost always based on these sufficient statistics. Furthermore, these distributions are most often intractable by other means when considered to the degree of accuracy that may be achieved when using saddlepoint approximations. Thus, in terms of accuracy, both in this context and in the many others previously discussed, saddlepoint methods are largely unrivaled by all other analytical approximations.

The subclass of regular exponential families of distributions was introduced in section 2.4.5 by “tilting” a distribution whose MGF converges in the “regular” sense. Tilting combined with Edgeworth expansions provide the techniques for an alternative derivation of saddlepoint densities that is presented in this chapter. This alternative approach also gives further understanding as to why the saddlepoint density achieves its remarkable accuracy. Both unconditional and conditional saddlepoint approximations are considered for the canonical sufficient statistics in a regular exponential family. The computation of p-values for various likelihood ratio tests and the determination of some uniformly most powerful unbiased tests are some of the practical examples that are considered.

Regular exponential families

Definitions and examples

Let z be the data and denote Z as the random variable for the data before observation.

When the joint MGF of (X, Y) is available, the conditional MGF of Y given X = x can be approximated by using the sequential saddlepoint method. Apart from providing conditional moments, this approximate conditional MGF may also serve as a surrogate for the true conditional MGF with saddlepoint methods. In such a role, it can be the input into a singlesaddlepoint method, such as the Lugananni and Rice approximation, to give an approximate conditional distribution. Thus, the resulting sequential saddlepoint approximation to Pr(Y = y|X = x) provides an alternative to the double-saddlepoint methods of sections 4.1-4.2.

Computation of a p-value for the Bartlett–Nandi–Pillai trace statistic in MANOVA provides a context in which the sequential saddlepoint approximation succeeds with high accuracy but the competing double-saddlepoint CDF approximation fails. Among the latter methods, only numerical integration of the double-saddlepoint density successfully replicates the accuracy of the sequential saddlepoint CDF approximation; see Butler et al. (1992b).

Another highly successful application of sequential saddlepoint methods occurs when approximating the power function of Wilks' likelihood ratio test inMANOVA. This example is deferred to section 11.3.1.

Sequential saddlepoint approximation

Suppose (X, Y) is a m-dimensional random vector with known joint CGF K(s, t) where s and t are the respective components. The goal is to use the joint CGF to determine conditional probabilities and moments of Y given X = x.

The p* densitywas introduced in Barndorff-Nielsen (1980, 1983) and has been prominently featured as an approximation for the density of the maximum likelihood estimate (MLE). Its development from saddlepoint approximations and its role in likelihood inference for regular exponential families are discussed in section 7.1. Section 7.2 considers group transformation models, such as linear regression and location and scale models. In this setting the p* formula provides the conditional distributions ofMLEsgiven a maximal ancillary statistic and agrees with expressions developed by Fisher (1934) and Fraser (1979). In curved exponential families, the p* formula also approximates the conditional distribution of MLEs given certain approximate ancillaries. This development is explored in section 7.3 with a more detailed discussion given for the situation in which the ancillaries are affine.

The p* density in regular exponential families

In the context of a regular exponential family, the p* density is the normalized saddlepoint density for the MLE. In its unnormalized form in the continuous setting, p* is simply a Jacobian transformation removed from the saddlepoint density of the canonical sufficient statistic. Consider the regular exponential family in (5.1) with canonical parameter θ ε ε ⊆ ℜm and canonical sufficient statistic X. Statistic X is also the MLE û for the mean parameterization μ = c′(θ) as discussed in section 5.1.2.

Among the various tools that have been developed for use in statistics and probability over the years, perhaps the least understood and most remarkable tool is the saddlepoint approximation. It is remarkable because it usually provides probability approximations whose accuracy is much greater than the current supporting theory would suggest. It is least understood because of the difficulty of the subject itself and the difficulty of the research papers and books that have been written about it. Indeed this lack of accessibility has placed its understanding out of the reach of many researchers in both statistics and its related subjects.

The primary aim of this book is to provide an accessible account of the theory and application of saddlepoint methods that can be understood by the widest possible audience. To do this, the book has been written at graduated levels of difficulty with the first six chapters forming the easiest part and the core of the subject. These chapters use little mathematics beyond the difficulty of advanced calculus (no complex variables) and should provide relatively easy reading to first year graduate students in statistics, engineering, and other quantitative fields. These chapters would also be accessible to senior-level undergraduate mathematics and advanced engineering majors. With the accessibility issue in mind, the first six chapters have been purposefully written to address the issue and should assure that the widest audience is able to read and learn the subject.

The presentation throughout the book takes the point of view of users of saddlepoint approximations; theoretical aspects of the methods are also covered but are given less emphasis than they are in journal articles.

Statistical inference is considered in five practical settings. In each application, saddlepoint approximations offer an innovative approach for computing p-values, mid-p-values, power functions, and confidence intervals. As case studies, these five settings motivate additional considerations connected with both the saddlepoint methodology and theory that are used for making statistical inference. Each additional topic is addressed as it arises through its motivating example.

The first application concerns logistic regression, with an emphasis on the determination of LD50. p-value computation, test inversion to determine confidence intervals, and the notion of mid-p-values are considered. Saddlepoint treatment of prospective and retrospective analyses are also compared.

The second application deals with common odds ratio estimation. Both single- and double-saddlepoint methods are applicable and these two approaches are compared with several examples. Power function computations and their properties are discussed in connection with both the single- and double-saddlepoint approaches. Time series data may also be analyzed and an example dealing with an autoregression of Poisson counts is given.

The final two applications are concerned with modeling nonhomogeneous Poisson processes, and inference with data that has been truncated. The former example reconsiders the Lake Konstanz data analyzed in Barndorff-Nielsen and Cox (1979) and the latter deals with the truncation of binomial counts that occur with genetic data.

The general use of saddlepoint methods for testing purposes in exponential families was introduced in Davison (1988), however these applications had already been anticipated much earlier in the discussion by Daniels (1958) concerning the logistic setting as well as in Barndorff-Nielsen and Cox (1979). Test inversion to determine confidence intervals was first considered in Pierce and Peters (1992).