Skip to main content
×
Home
    • Aa
    • Aa

CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS?

  • Hannes Leeb (a1) and Benedikt M. Pötscher (a2)
Abstract

We consider the problem of estimating the unconditional distribution of a post-model-selection estimator. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model-selection criterion such as the Akaike information criterion [AIC] or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for the distribution; performance is here measured by the probability that the estimation error exceeds a given threshold. These lower bounds are shown to approach ½ or even 1 in large samples, depending on the situation considered. Similar impossibility results are also obtained for the distribution of linear functions (e.g., predictors) of the post-model-selection estimator.The research of the first author was supported by the Max Kade Foundation and by the Austrian National Science Foundation (FWF), Grant no. P13868-MAT. A preliminary draft of the material in this paper was written in 1999.

Copyright
Corresponding author
Address correspondence to Benedikt M. Pötscher, Department of Statistics, University of Vienna, Universitätsstrasse 5, A-1010 Vienna, Austria; e-mail: Benedikt.Poetscher@univie.ac.at
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

Ahmed, S.E. & A.K. Basu (2000) Least squares, preliminary test and Stein-type estimation in general vector AR(p) models. Statistica Neerlandica 54, 4766.

Bauer, P., B.M. Pötscher, & P. Hackl (1988) Model selection by multiple test procedures. Statistics 19, 3944.

Brownstone, D. (1990) Bootstrapping improved estimators for linear regression models. Journal of Econometrics 44, 171187.

Danilov, D.L. & J.R. Magnus (2004) On the harm that ignoring pre-testing can cause. Journal of Econometrics 122, 2746.

Dijkstra, T.K. & J.H. Veldkamp (1988) Data-driven selection of regressors and the bootstrap. Lecture Notes in Economics and Mathematical Systems 307, 1738.

Dukić, V.M. & E.A. Peña (2005) Variance estimation in a model with Gaussian submodel. Journal of the American Statistical Association 100, 296309.

Freedman, D.A., W. Navidi, & S.C. Peters (1988) On the impact of variable selection in fitting regression equations. Lecture Notes in Economics and Mathematical Systems 307, 116.

Hjort, N.L. & G. Claeskens (2003) Frequentist model average estimators. Journal of the American Statistical Association 98, 879899.

Kabaila, P. (1995) The effect of model selection on confidence regions and prediction regions. Econometric Theory 11, 537549.

Kapetanios, G. (2001) Incorporating lag order selection uncertainty in parameter inference for AR models. Economics Letters 72, 137144.

Kilian, L. (1998) Accounting for lag order uncertainty in autoregressions: The endogenous lag order bootstrap algorithm. Journal of Time Series Analysis 19, 531548.

Kulperger, R.J. & S.E. Ahmed (1992) A bootstrap theorem for a preliminary test estimator. Communications in Statistics: Theory and Methods 21, 20712082.

Leeb, H. (2005) The distribution of a linear predictor after model selection: Conditional finite-sample distributions and asymptotic approximations. Journal of Statistical Planning and Inference 134, 6489.

Leeb, H. & B.M. Pötscher (2003) The finite-sample distribution of post-model-selection estimators and uniform versus nonuniform approximations. Econometric Theory 19, 100142.

Leeb, H. & B.M. Pötscher (2005a) Model selection and inference: Facts and fiction. Econometric Theory 21, 2159.

Leeb, H. & B.M. Pötscher (2006a) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk bound results. Econometric Theory 22, 6997. (Corrigendum, Econometric Theory, forthcoming.)

Leeb, H. & B.M. Pötscher (2006b) Can one estimate the conditional distribution of post-model-selection estimators? Annals of Statistics 34, 25542591.

Pötscher, B.M. (1991) Effects of model selection on inference. Econometric Theory 7, 163185.

Pötscher, B.M. (1995) Comment on “The effect of model selection on confidence regions and prediction regions” by P. Kabaila. Econometric Theory 11, 550559.

Pötscher, B.M. & A.J. Novak (1998) The distribution of estimators after model selection: Large and small sample results. Journal of Statistical Computation and Simulation 60, 1956.

Robinson, G.K. (1979) Conditional properties of statistical procedures. Annals of Statistics 7, 742755.

Sen, P.K. (1979) Asymptotic properties of maximum likelihood estimators based on conditional specification. Annals of Statistics 7, 10191033.

Sen, P.K. & A.K.M.E. Saleh (1987) On preliminary test and shrinkage M-estimation in linear models. Annals of Statistics 15, 15801592.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Econometric Theory
  • ISSN: 0266-4666
  • EISSN: 1469-4360
  • URL: /core/journals/econometric-theory
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 1
Total number of PDF views: 27 *
Loading metrics...

Abstract views

Total abstract views: 132 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 27th May 2017. This data will be updated every 24 hours.