Hostname: page-component-848d4c4894-wg55d Total loading time: 0 Render date: 2024-05-01T07:52:58.678Z Has data issue: false hasContentIssue false
  • English
  • Français

AN ASYMPTOTIC THEORY FOR LEAST SQUARES MODEL AVERAGING WITH NESTED MODELS

Published online by Cambridge University Press:  08 February 2022

Fang Fang ,
Chaoxia Yuan  and
Wenling Tian
Fang Fang*
Affiliation:
East China Normal University
Chaoxia Yuan
Affiliation:
East China Normal University
Wenling Tian
Affiliation:
East China Normal University
*
Address correspondence to Fang Fang, Key Laboratory for Advanced Theory and Application in Statistics and Data Science—MOE, Faculty of Economics and Management, East China Normal University, 3663 North Zhongshan Road, Shanghai 200062, China; e-mail: ffang@sfs.ecnu.edu.cn.
Get access Rights & Permissions [Opens in a new window]

Abstract

Theoretical results of frequentist model averaging mainly focus on asymptotic optimality and asymptotic distribution of the model averaging estimator. However, even for basic least squares model averaging, many theoretical problems have not been well addressed yet. This article discusses asymptotic properties of a class of least squares model averaging methods with nested candidate models that includes the Mallows model averaging (MMA) of Hansen (2007, Econometrica 75, 1175–1189) as a special case. Two scenarios are considered: (i) all candidate models are under-fitted; and (ii) the true model is included in the candidate models. We find that in the first scenario, the least squares model averaging method asymptotically assigns weight one to the largest candidate model and the resulting model averaging estimator is asymptotically normal. In the second scenario with a slightly special weight space, if the penalty factor in the weight selection criterion is diverging with certain order, the model averaging estimator is asymptotically optimal by putting weight one to the true model. However, MMA with fixed model dimensions is not asymptotically optimal since it puts nonnegligible weights to over-fitted models. The theoretical results are clearly summarized with their restrictions, and some critical implications are discussed. Monte Carlo simulations confirm our theoretical results.

Type
ARTICLES
Information
Econometric Theory , Volume 39 , Issue 2 , April 2023 , pp. 412 - 441
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We would like to thank the Editor (Peter C.B. Phillips), the Co-Editor (Michael Jansson), and the anonymous referees for many constructive comments and suggestions that led to a much improved paper. Fang gratefully acknowledges the research support from National Key R&D Program of China (2021YFA1000100 and 2021YFA1000101) and the National Natural Science Foundation of China (12071143, 11831008, 11771146).

References

REFERENCES

Akaike, H. (1973) Information theory and an extension of the maximum likelihood principle. In Petroc, B. and Csake, F. (eds.), Second International Symposium on Information Theory , pp. 267–281. Akademiai Kiado.Google Scholar
Ando, T. & Li, K.-C. (2014) A model-averaging approach for high-dimensional regression. Journal of the American Statistical Association 109, 254–265.CrossRefGoogle Scholar
Ando, T. & Li, K.-C. (2017) A weight-relaxed model averaging approach for high-dimensional generalized linear models. The Annals of Statistics 45, 2654–2679.CrossRefGoogle Scholar
Box, G.E.P. (1976) Science and statistics. Journal of the American Statistical Association 71, 791–799.CrossRefGoogle Scholar
Buckland, S.T., Burnham, K.P., & Augustin, N.H. (1997) Model selection: An integral part of inference. Biometrics 53, 603–618.CrossRefGoogle Scholar
Fang, F., Li, J., & Xia, X. (2020) Semiparametric model averaging prediction for dichotomous response. Journal of Econometrics . https://doi.org/10.1016/j.jeconom.2020.09.008.Google Scholar
Fang, F. & Liu, M. (2020) Limit of the optimal weight in least squares model averaging with non-nested models. Economics Letters 196, 109586.CrossRefGoogle Scholar
Hansen, B.E. (2007) Least squares model averaging. Econometrica 75, 1175–1189.CrossRefGoogle Scholar
Hansen, B.E. (2014) Model averaging, asymptotic risk, and regression groups. Quantitative Economics 5, 495–530.CrossRefGoogle Scholar
Hansen, B.E. & Racine, J.S. (2012) Jackknife model averaging. Journal of Econometrics 167, 38–46.CrossRefGoogle Scholar
Hjort, N.L. & Claeskens, G. (2003a) Frequentist model averaging estimators. Journal of the American Statistical Association 98, 879–899.CrossRefGoogle Scholar
Hjort, N.L. & Claeskens, G. (2003b) Rejoinder to the focused information criterion and frequentist model averaging estimators. Journal of the American Statistical Association 98, 938–945.CrossRefGoogle Scholar
Hoeting, J.A., Madigan, D., Raftery, A.E., & Volinsky, C.T. (1999) Bayesian model averaging: A tutorial. Statistical Science 14, 382–417.Google Scholar
Kitagawa, T. & Muris, C. (2016) Model averaging in semiparametric estimation of treatment effects. Journal of Econometrics 193, 271–289.CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2005) Model selection and inference: Facts and fiction. Econometric Theory 21, 21–59.CrossRefGoogle Scholar
Leung, G. & Barron, A.R. (2006) Information theory and mixing least-squares regressions. IEEE Transactions on Information Theory 52, 3396–3410.CrossRefGoogle Scholar
Li, C., Li, Q., Racine, J.S., & Zhang, D. (2018a) Optimal model averaging of varying coefficient models. Statistica Sinica 28, 2795–2809.Google Scholar
Li, D., Linton, O., & Lu, Z. (2015) A flexible semiparametric forecasting model for time series. Journal of Econometrics 187, 345–357.CrossRefGoogle Scholar
Li, J., Xia, X., Wong, W.K., & Nott, D. (2018b) Varying-coefficient semiparametric model averaging prediction. Biometrics 74, 1417–1428.CrossRefGoogle ScholarPubMed
Liang, H., Zou, G., Wan, A.T.K., & Zhang, X. (2011) Optimal weight choice for frequentist model averaging estimators. Journal of the American Statistical Association 106, 1053–1066.CrossRefGoogle Scholar
Liao, J., Zong, X., Zhang, X., & Zou, G. (2019) Model averaging based on leave-subject-out cross-validation for vector autoregressions. Journal of Econometrics 209, 35–60.CrossRefGoogle Scholar
Liu, C.-A. (2015) Distribution theory of the least squares averaging estimator. Journal of Econometrics 186, 142–159.CrossRefGoogle Scholar
Liu, Q. & Okui, R. (2013) Heteroskedasticity-robust Cp model averaging. The Econometrics Journal 16, 463–472.CrossRefGoogle Scholar
Longford, N.T. (2005) Editorial: Model selection and efficiency—Is “which model…?” the right question? Journal of the Royal Statistical Society, Series A 168, 469–472.CrossRefGoogle Scholar
Peng, J. & Yang, Y. (2021) On improvability of model selection by model averaging. Journal of Econometrics . https://doi.org/10.1016/j.jeconom.2020.12.003.Google Scholar
Phillips, P.C.B. (2005) Automated discovery in econometrics. Econometric Theory 21, 3–20.CrossRefGoogle Scholar
Raftery, A.E. & Zheng, Y. (2003) Discussion: Performance of Bayesian model averaging. Journal of the American Statistical Association 98, 931–938.CrossRefGoogle Scholar
Schwarz, G. (1978) Estimating the dimension of a model. The Annals of Statistics 6, 461–464.CrossRefGoogle Scholar
Shao, J. (1997) An asymptotic theory for linear model selection (with discussion). Statistica Sinica 7, 221–264.Google Scholar
Wan, A.T.K., Zhang, X., & Wang, S. (2014) Frequentist model averaging for multinomial and ordered logit models. International Journal of Forecasting 30, 118–128.CrossRefGoogle Scholar
Wan, A.T.K., Zhang, X., & Zou, G. (2010) Least squares model averaging by Mallows criterion. Journal of Econometrics 156, 277–283.CrossRefGoogle Scholar
Whittle, P. (1960) Bounds for the moments of linear and quadratic forms in independent variables. Theory of Probability and Its Applications 5, 302–305.CrossRefGoogle Scholar
Yang, Y. (2001) Adaptive regression by mixing. Journal of the American Statistical Association 96, 574–586.CrossRefGoogle Scholar
Yang, Y. (2003) Regression with multiple candidate models: Selecting or mixing? Statistica Sinica 13, 783–809.Google Scholar
Yuan, Z. & Yang, Y. (2005) Combining linear regression models: When and how? Journal of the American Statistical Association 100, 1202–1214.CrossRefGoogle Scholar
Zhang, X. (2015) Consistency of model averaging estimators. Economics Letters 130, 120–123.CrossRefGoogle Scholar
Zhang, X. (2021) A new study on asymptotic optimality of least squares model averaging. Econometric Theory 37, 388–407.CrossRefGoogle Scholar
Zhang, X. & Liang, H. (2011) Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics 39, 174–200.CrossRefGoogle Scholar
Zhang, X. & Liu, C.-A. (2019) Inference after model averaging in linear regression models. Econometric Theory 35, 816–841.CrossRefGoogle Scholar
Zhang, X. & Wang, W. (2019) Optimal model averaging estimation for partially linear models. Statistica Sinica 29, 693–718.Google Scholar
Zhang, X., Yu, D., Zou, G., & Liang, H. (2016) Optimal model averaging estimation for generalized linear models and generalized linear mixed-effects models. Journal of the American Statistical Association 111, 1775–1790.CrossRefGoogle Scholar
Zhang, X., Zou, G., & Carroll, R.J. (2015) Model averaging based on Kullback–Leibler distance. Statistica Sinica 25, 1583–1598.Google ScholarPubMed
Zhang, X., Zou, G., & Liang, H. (2014) Model averaging and weight choice in linear mixed effects models. Biometrika 101, 205–218.CrossRefGoogle Scholar
Zhang, X., Zou, G., Liang, H., & Carroll, R.J. (2020) Parsimonious model averaging with a diverging number of parameters. Journal of the American Statistical Association 115, 972–984.CrossRefGoogle ScholarPubMed
Zhang, Y. & Yang, Y. (2015) Cross-validation for selecting a model selection procedure. Journal of Econometrics 187, 95–112.CrossRefGoogle Scholar
Zheng, H., Tsui, K.-W., Kang, X., & Deng, X. (2017) Cholesky-based model averaging for covariance matrix estimation. Statistical Theory and Related Fields 1, 48–58.CrossRefGoogle Scholar
Zhu, R., Wan, A.T.K., Zhang, X., & Zou, G. (2019) A Mallows-type model averaging estimator for the varying-coefficient partially linear model. Journal of the American Statistical Association 114, 882–892.CrossRefGoogle Scholar
Zou, H. & Zhang, H. (2009) On the adaptive elastic-net with a diverging number of parameters. The Annals of Statistics 37, 1733–1751.CrossRefGoogle ScholarPubMed