Kullback-Leibler Information and the Maximum Empirical Exponential Likelihood

George G. Judge; Ron C. Mittelhammer

doi:10.1017/CBO9781139033848.009

Six - Kullback-Leibler Information and the Maximum Empirical Exponential Likelihood

from Part II - Formulation and Solution of Stochastic Inverse Problems

Published online by Cambridge University Press: 05 June 2012

George G. Judge and

Ron C. Mittelhammer

Show author details

George G. Judge: Affiliation:
University of California, Berkeley
Ron C. Mittelhammer: Affiliation:
Washington State University

Book contents

Get access

Summary

Introduction

In this chapter, we continue to build on the extremum (E) estimation and estimating equations (EE) concepts introduced in Chapter 3 and the maximum empirical likelihood (MEL) approach demonstrated in Chapters 4 and 5. Our focus is on methods of estimation and inference involving stochastic inverse problems of the form (Y, X)⇒(η, σ2, ε). In this context, in this chapter we introduce the Kullback-Leibler Information Criterion (KLIC) and the maximum empirical exponential likelihood (MEEL) function and estimator.

Solutions to Systems of Estimating Equations and Kullback-Leibler Information

The procedures examined in the previous three chapters involved solutions relating to systems of estimating equations (EEs). In both the MEL and EE contexts, the EEs were empirical analogs to expectation relations of the type E[h(Y, X, θ)] = 0. Estimators for θ, defined by the solution to moment equations, subsume a large collection of familiar estimation procedures that include linear and nonlinear least squares. The idea of basing an estimator definition on the solution to EEs can be applied quite generally to extremum problems, as demonstrated in Chapter 3.

Type: Chapter
Information: An Information Theoretic Approach to Econometrics , pp. 104 - 132

DOI: https://doi.org/10.1017/CBO9781139033848.009 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Brown, L. D 1986 Fundamentals of Statistical Exponential FamiliesHayward, CAInstitute of Mathematical StatisticsGoogle Scholar

Csiszar, I 1991 Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse ProblemsAnnals of Statistics 19 2032CrossRef Google Scholar

Gokhale, D. VKullback, S 1978 The Information in Contingency TablesNew YorkMarcel DekkerGoogle Scholar

Golan, AJudge, G. GMiller, D 1996 Maximum Entropy Econometrics: Robust Estimation with Limited DataNew YorkJohn Wiley & SonsGoogle Scholar

Good, I. J 1963 Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency TablesAnnals of Mathematical Statistics 34 911CrossRef Google Scholar

Imbens, G 1997 One-Step Estimators for Over-Identified Generalized Method of Moment ModelsReview of Economic Studies 64 359CrossRef Google Scholar

Imbens, GSpady, RJohnson, P 1998 Information Theoretic Approaches to Inference in Moment Condition ModelsEconometrica 66 333CrossRef Google Scholar

Jaynes, E. T 1957 Information Theory and Statistical MechanicsPhysics Review 106 620CrossRef Google Scholar

Jaynes, E. T 1957 Information Theory and Statistical Mechanics IIPhysics Review 108 171CrossRef Google Scholar

Jaynes, E. T 1963 Information Theory and Statistical MechanicsFord, K. WStatistical Physics181New YorkW. A. BenjaminGoogle Scholar

Jaynes, E. T 1984 Prior Information and Ambiguity in Inverse ProblemsMcLaughlin, D. WInverse Problems151Providence, RIAmerican Mathematical SocietyGoogle Scholar

Judge, G. GBock, M. E 1978 The Statistical Implications of Pre-Test and Stein-Rule Estimators in EconometricsAmsterdamNorth-HollandGoogle Scholar

Judge, G. GMittelhammer, R 2004 A Semiparametric Like Basis for Combining Estimators under Quadratic LossJournal of American Statistical Association 99 479CrossRef Google Scholar

Judge, G. GMittelhammer, R 2007 Estimation and Inference in Case of Competing Sets of Estimating EquationsJournal of Econometrics 138 513CrossRef Google Scholar

Kitamura, YStutzer, M 1997 An Information-Theoretic Alternative to Generalized Method of Moments EstimationEconometrica 65 861CrossRef Google Scholar

Kullback, S 1987 Letter to the Editor: The Kullback-Leibler DistanceAmerican Statistician 41 340Google Scholar

Kullback, S 1959 Information Theory and StatisticsNew YorkJohn Wiley & SonsGoogle Scholar

Kullback, SLeibler, R. A 1951 On Information and SufficiencyAnnals of Mathematical Statistics 22 79CrossRef Google Scholar

Mittelhammer, RJudge, GMiller, D 2000 Econometric FoundationsNew YorkCambridge University PressGoogle Scholar

Owen, A 1990 Empirical Likelihood Ratio Confidence RegionsAnnals of Statistics 18 90CrossRef Google Scholar

Shannon, C. E 1948 A Mathematical Theory of CommunicationBell System Technical Journal 27 379CrossRef Google Scholar