Hostname: page-component-77c78cf97d-d2fvj Total loading time: 0 Render date: 2026-04-24T15:24:00.281Z Has data issue: false hasContentIssue false
  • English
  • Français

A NOVEL APPROACH TO PREDICTIVE ACCURACY TESTING IN NESTED ENVIRONMENTS

Published online by Cambridge University Press:  17 May 2023

Jean-Yves Pitarakis Open the ORCID record for Jean-Yves Pitarakis [Opens in a new window]
Jean-Yves Pitarakis*
Affiliation:
University of Southampton
*
Address correspondence to Jean-Yves Pitarakis, Department of Economics, University of Southampton, Southampton SO17 1BJ, United Kingdom; e-mail: j.pitarakis@soton.ac.uk.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

We introduce a new approach for comparing the predictive accuracy of two nested models that bypasses the difficulties caused by the degeneracy of the asymptotic variance of forecast error loss differentials used in the construction of commonly used predictive comparison statistics. Our approach continues to rely on the out of sample mean squared error loss differentials between the two competing models, leads to nuisance parameter-free Gaussian asymptotics, and is shown to remain valid under flexible assumptions that can accommodate heteroskedasticity and the presence of mixed predictors (e.g., stationary and local to unit root). A local power analysis also establishes their ability to detect departures from the null in both stationary and persistent settings. Simulations calibrated to common economic and financial applications indicate that our methods have strong power with good size control across commonly encountered sample sizes.

Information

Type
ARTICLES
Information
Econometric Theory , Volume 41 , Issue 1 , February 2025 , pp. 35 - 78
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Footnotes

I wish to thank the Editor, Co-Editor, and three anonymous referees for the quality of the reports I have received and their in-depth review of an earlier version of this paper. I also wish to thank the ESRC for its financial support via grant ES/W000989/1. Any errors are my own responsibility.

References

REFERENCES

Avdis, E. & Wachter, J.A. (2017) Maximum likelihood estimation of the equity premium. Journal of Financial Economics 125, 589–609.CrossRefGoogle Scholar
Berenguer-Rico, V. & Nielsen, B. (2020) Cumulated sum of squares statistics for nonlinear and nonstationary regressions. Econometric Theory 36, 1–47.CrossRefGoogle Scholar
Berkes, I., Hörmann, S., & Horvath, L. (2008) The functional central limit theorem for a family of GARCH observations with applications. Statistics and Probability Letters 78, 2725–2730.CrossRefGoogle Scholar
Clark, T.E. & McCracken, M.W. (2001) Tests of equal forecast accuracy and encompassing for nested models. Journal of Econometrics 105, 85–110.CrossRefGoogle Scholar
Clark, T.E. & McCracken, M.W. (2005) Evaluating direct multistep forecasts. Econometric Reviews 24, 369–404.CrossRefGoogle Scholar
Clark, T.E. & McCracken, M.W. (2013) Advances in forecast evaluation. In Elliott, G. & Timmermann, A. (eds.), Handbook of Economic Forecasting, vol. 2, Part B, pp. 1107–1201. Elsevier.Google Scholar
Clark, T.E. & West, K.D. (2007) Approximately normal tests for equal predictive accuracy in nested models. Journal of Econometrics 138, 291–311.CrossRefGoogle Scholar
Deng, A. & Perron, P. (2008a) The limit distribution of the Cusum of squares test under general mixing conditions. Econometric Theory 24, 809–822.CrossRefGoogle Scholar
Deng, A. & Perron, P. (2008b) A non-local perspective on the power properties of the CUSUM and CUSUM of squares tests for structural change. Journal of Econometrics 142, 212–240.CrossRefGoogle Scholar
Diebold, F.X. (2015) Comparing predictive accuracy, twenty years later: A personal perspective on the use and abuse of Diebold–Mariano tests. Journal of Business and Economic Statistics 33, 1–24.CrossRefGoogle Scholar
Diebold, F.X. & Mariano, R. (1995) Comparing predictive accuracy. Journal of Business and Economic Statistics 13, 253–265.CrossRefGoogle Scholar
Engel, C. & Wu, S. (2021) Forecasting the U.S. Dollar in the 21st Century. NBER Working Paper no. 28447.Google Scholar
Fan, J., Liao, Y., & Yao, J. (2015) Power enhancement in high dimensional cross-sectional tests. Econometrica 83, 1497–1541.CrossRefGoogle ScholarPubMed
Ferson, W., Nallareddy, S., & Biqin, X. (2013) The out-of-sample performance of long run risk models. Journal of Financial Economics 107, 537–556.CrossRefGoogle Scholar
Gades-Riva, M.D. & Gonzalo, J. (2020) Trends in distributional characteristics: Existence of global warming. Journal of Econometrics 214, 153–174.Google Scholar
Giacomini, R. & White, H. (2006) Tests of conditional predictive ability. Econometrica 74, 1545–1578.CrossRefGoogle Scholar
Giraitis, L., Kokoszka, P., & Leipus, R. (2000) Stationary ARCH models: Dependence structure and central limit theorems. Econometric Theory 16, 3–22.CrossRefGoogle Scholar
Giraitis, L., Kokoszka, P., & Leipus, R. (2001) Testing for long memory in the presence of a general trend. Journal of Applied Probability 38, 1033–1054.CrossRefGoogle Scholar
Granziera, E., Hubrich, K., & Moon, H. (2014) Predictability tests for a small number of nested models. Journal of Econometrics 182, 174–185.CrossRefGoogle Scholar
Hansen, P.R. & Timmermann, A. (2015) Equivalence between out-of-sample forecast comparisons and Wald statistics. Econometrica 83, 2485–2505.CrossRefGoogle Scholar
Ince, O., Molodotsova, T., & Papell, D.H. (2016) Taylor rule deviations and out-of-sample exchange rate predictability. Journal of International Money and Finance 69, 22–44.CrossRefGoogle Scholar
Jiang, F., Zhao, Z., & Shao, X. (2020) Modelling the COVID-19 infection trajectory: A piecewise linear quantile trend model. Journal of the Royal Statistical Society: Series B 84, 1589–1607.CrossRefGoogle Scholar
Li, S. & Linton, O. (2021) When will the COVID-19 pandemic peak? Journal of Econometrics 220, 130–157.CrossRefGoogle ScholarPubMed
Linder, A.M. (2009) Stationarity, mixing, distributional properties and moments of GARCH(p,q)-processes. In Mikosch, T., Kreiss, J., Richard, A.D., & Andersen, T.G. (eds.), Handbook of Financial Time Series , pp. 43–69. Springer.CrossRefGoogle Scholar
McCracken, M. (2007) Asymptotics for out of sample tests of Granger causality. Journal of Econometrics 140, 719–752.CrossRefGoogle Scholar
Meese, R.A. & Rogoff, K. (1983) Empirical exchange rate models of the seventies. Journal of International Economics 14, 3–24.CrossRefGoogle Scholar
Molodotsova, T. & Papell, D.H. (2009) Out-of-sample exchange rate predictability with Taylor rule fundamentals. Journal of International Economics 77, 167–180.CrossRefGoogle Scholar
Rossi, B. (2005) Testing long-horizon predictive ability with high persistence, and the Meese–Rogoff puzzle. International Economic Review 46, 61–92.CrossRefGoogle Scholar
Schennah, S.M. & Wilhelm, D. (2017) A simple parametric model selection test. Journal of the American Statistical Association 112, 1663–1674.CrossRefGoogle Scholar
Shi, X. (2015) A nondegenerate Vuong test. Quantitative Economics 6, 85–121.CrossRefGoogle Scholar
Stock, J. & Watson, M. (2010) Modeling Inflation after the Crisis. Macroeconomic Policy: Post-Crisis and Risks Ahead, Proceedings of the Federal Reserve Bank of Kansas City 2010 Jackson Hole Symposium.CrossRefGoogle Scholar
Vuong, Q.H. (1989) Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica 57, 307–333.CrossRefGoogle Scholar
West, K. (1996) Asymptotic inference about predictive ability. Econometrica 64, 1067–1084.CrossRefGoogle Scholar
West, K. (2006) Forecast evaluation. In Elliott, G., Granger, C.W.J., & Timmermann, A. (eds.), Handbook of Economic Forecasting , vol. 1, pp. 99–134. Elsevier.CrossRefGoogle Scholar
Wu, W. & Zhao, Z. (2007) Inference of trends in time series. Journal of the Royal Statistical Society: Series B 69, 391–410.CrossRefGoogle Scholar
Supplementary material: PDF

Pitarakis supplementary material

Appendix

Download Pitarakis supplementary material(PDF)
PDF 310.2 KB