Skip to main content
×
Home
    • Aa
    • Aa

CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?

  • Hannes Leeb (a1) and Benedikt M. Pötscher (a2)
Abstract

We consider the problem of estimating the unconditional distribution of a post-model-selection estimator. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model-selection criterion such as the Akaike information criterion [AIC] or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for the distribution; performance is here measured by the probability that the estimation error exceeds a given threshold. These lower bounds are shown to approach ½ or even 1 in large samples, depending on the situation considered. Similar impossibility results are also obtained for the distribution of linear functions (e.g., predictors) of the post-model-selection estimator.The research of the first author was supported by the Max Kade Foundation and by the Austrian National Science Foundation (FWF), Grant no. P13868-MAT. A preliminary draft of the material in this paper was written in 1999.

Copyright
Corresponding author
Address correspondence to Benedikt M. Pötscher, Department of Statistics, University of Vienna, Universitätsstrasse 5, A-1010 Vienna, Austria; e-mail: Benedikt.Poetscher@univie.ac.at
References
Hide All

REFERENCES

Ahmed, S.E. & A.K. Basu (2000) Least squares, preliminary test and Stein-type estimation in general vector AR(p) models. Statistica Neerlandica 54, 4766.
Bauer, P., B.M. Pötscher, & P. Hackl (1988) Model selection by multiple test procedures. Statistics 19, 3944.
Billingsley, P. (1995) Probability and Measure, 3rd ed. Wiley.
Brownstone, D. (1990) Bootstrapping improved estimators for linear regression models. Journal of Econometrics 44, 171187.
Danilov, D.L. & J.R. Magnus (2004) On the harm that ignoring pre-testing can cause. Journal of Econometrics 122, 2746.
Dijkstra, T.K. & J.H. Veldkamp (1988) Data-driven selection of regressors and the bootstrap. Lecture Notes in Economics and Mathematical Systems 307, 1738.
Dukić, V.M. & E.A. Peña (2005) Variance estimation in a model with Gaussian submodel. Journal of the American Statistical Association 100, 296309.
Freedman, D.A., W. Navidi, & S.C. Peters (1988) On the impact of variable selection in fitting regression equations. Lecture Notes in Economics and Mathematical Systems 307, 116.
Hansen, P.R. (2003) Regression Analysis with Many Specifications: A Bootstrap Method for Robust Inference. Working paper, Department of Economics, Brown University.
Hjort, N.L. & G. Claeskens (2003) Frequentist model average estimators. Journal of the American Statistical Association 98, 879899.
Kabaila, P. (1995) The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537549.
Kapetanios, G. (2001) Incorporating lag order selection uncertainty in parameter inference for AR models. Economics Letters 72, 137144.
Kilian, L. (1998) Accounting for lag order uncertainty in autoregressions: The endogenous lag order bootstrap algorithm. Journal of Time Series Analysis 19, 531548.
Knight, K. (1999) Epi-convergence in Distribution and Stochastic Equi-semicontinuity. Working paper, Department of Statistics, University of Toronto.
Kulperger, R.J. & S.E. Ahmed (1992) A bootstrap theorem for a preliminary test estimator. Communications in Statistics: Theory and Methods 21, 20712082.
Leeb, H. (2002) On a differential equation with advanced and retarded arguments. Communications on Applied Nonlinear Analysis 9, 7786.
Leeb, H. (2005) The distribution of a linear predictor after model selection: Conditional finite-sample distributions and asymptotic approximations. Journal of Statistical Planning and Inference 134, 6489.
Leeb, H. (2006) The distribution of a linear predictor after model selection: Unconditional finite-sample distributions and asymptotic approximations. IMS Lecture Notes-Monograph Series 49, 291311.
Leeb, H. & B.M. Pötscher (2003) The finite-sample distribution of post-model-selection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100142.
Leeb, H. & B.M. Pötscher (2005a) Model selection and inference: Facts and fiction. Econometric Theory 21, 2159.
Leeb, H. & B.M. Pötscher (2005b) Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators? Working paper, Department of Statistics, University of Vienna.
Leeb, H. & B.M. Pötscher (2006a) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk bound results. Econometric Theory 22, 6997. (Corrigendum, Econometric Theory, forthcoming.)
Leeb, H. & B.M. Pötscher (2006b) Can one estimate the conditional distribution of post-model-selection estimators? Annals of Statistics 34, 25542591.
Lehmann, E.L. & G. Casella (1998) Theory of Point Estimation, 2nd ed. Springer Texts in Statistics. Springer-Verlag.
Nickl, R. (2003) Asymptotic distribution theory of post-model-selection maximum likelihood estimators. Master's thesis, Department of Statistics, University of Vienna.
Pötscher, B.M. (1991) Effects of model selection on inference. Econometric Theory 7, 163185.
Pötscher, B.M. (1995) Comment on “The effect of model selection on confidence regions and prediction regions” by P. Kabaila. Econometric Theory 11, 550559.
Pötscher, B.M. & A.J. Novak (1998) The distribution of estimators after model selection: Large and small sample results. Journal of Statistical Computation and Simulation 60, 1956.
Rao, C.R. & Y. Wu (2001) On model selection. IMS Lecture Notes-Monograph Series 38, 157.
Robinson, G.K. (1979) Conditional properties of statistical procedures. Annals of Statistics 7, 742755.
Sen, P.K. (1979) Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 10191033.
Sen, P.K. & A.K.M.E. Saleh (1987) On preliminary test and shrinkage M-estimation in linear models. Annals of Statistics 15, 15801592.
van der Vaart, A.W. (1998) Asymptotic Statistics. Cambridge University Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Econometric Theory
  • ISSN: 0266-4666
  • EISSN: 1469-4360
  • URL: /core/journals/econometric-theory
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 1
Total number of PDF views: 35 *
Loading metrics...

Abstract views

Total abstract views: 194 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 21st October 2017. This data will be updated every 24 hours.