Skip to main content
×
×
Home

REGULARIZING PRIORS FOR LINEAR INVERSE PROBLEMS

  • Jean-Pierre Florens (a1) and Anna Simoni (a2)
Abstract

This paper proposes a new Bayesian approach for estimating, nonparametrically, functional parameters in econometric models that are characterized as the solution of a linear inverse problem. By using a Gaussian process prior we propose the posterior mean as an estimator and prove frequentist consistency of the posterior distribution. The latter provides the frequentist validation of our Bayesian procedure. We show that the minimax rate of contraction of the posterior distribution can be obtained provided that either the regularity of the prior matches the regularity of the true parameter or the prior is scaled at an appropriate rate. The scaling parameter of the prior distribution plays the role of a regularization parameter. We propose a new data-driven method for optimally selecting in practice this regularization parameter. We also provide sufficient conditions such that the posterior mean, in a conjugate-Gaussian setting, is equal to a Tikhonov-type estimator in a frequentist setting. Under these conditions our data-driven method is valid for selecting the regularization parameter of the Tikhonov estimator as well. Finally, we apply our general methodology to two leading examples in econometrics: instrumental regression and functional regression estimation.

Copyright
Corresponding author
*Address correspondence to Anna Simoni, CREST, 15 Boulevard Gabriel Péri, 92240 Malakoff, France; e-mail: simoni.anna@gmail.com.
References
Hide All
Agapiou, S., Larsson, S., & Stuart, A.M. (2013) Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems. Stochastic Processes and Applications 123, 38283860.
Ai, C. & Chen, X. (2003) Efficient estimation of models with conditional moment restrictions containing unknown functions. Econometrica 71, 17951843.
Belitser, E. & Ghosal, S. (2003) Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Annals of Statistics 31, 536559.
Bissantz, N., Hohage, T., Munk, A., & Ruymgaart, F. (2007) Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM Journal on Numerical Analysis 45, 26102636.
Carrasco, M. & Florens, J.P. (2000) Generalization of GMM to a continuum of moment conditions. Econometric Theory 16, 797834.
Carrasco, M., Florens, J.P., & Renault, E. (2005) Estimation based on spectral decomposition and regularization. In Heckman, J.J. & Leamer, E. (eds.), Handbook of Econometrics, vol. 6, pp. 56335751. Elsevier.
Cavalier, L. (2011) Inverse problems in statistics. In Alquier, P., Gautier, E., & Stoltz, G. (eds.), Inverse Problems and High-dimensional Estimation, pp. 396. Springer-Verlag.
Cavalier, L. & Tsybakov, A.B. (2002) Sharp adaptation for inverse problems with random noise. Probability Theory and Related Fields 123, 323354.
Chen, X. & Pouzo, D. (2012) Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals. Econometrica 80, 277321.
Chen, X. & Reiss, M. (2011) On rate optimality for ill-posed inverse problems in econometrics. Econometric Theory 27, 497521.
Daouia, A., Florens, J.-P., & Simar, L. (2009) Regularization of nonparametric frontier estimators. Journal of Econometrics 168, 285299.
Darolles, S., Fan, Y., Florens, J.P., & Renault, E. (2011) Nonparametric instrumental regression. Econometrica 79, 15411565.
Diaconis, P.W. & Freedman, D. (1986) On the consistency of Bayes estimates. Annals of Statistics 14, 126.
Engl, H.W., Hanke, M., & Neubauer, A. (2000) Regularization of Inverse Problems. Kluwer Academic.
Florens, J.-P. (2003) Inverse problems and structural econometrics: The example of instrumental variables. Invited Lectures to the World Congress of the Econometric Society, Seattle 2000. In Dewatripont, M., Hansen, L.-P., & Turnovsky, S.J. (eds.), Advances in Economics end Econometrics: Theory and Applications, vol. II, pp. 284311. Cambridge University Press.
Florens, J.-P., Johannes, J., & Van Bellegem, S. (2011) Identification and estimation by penalization in nonparametric instrumental regression. Econometric Theory 27, 472496.
Florens, J.P. & Simoni, A. (2010) Regularizing Priors for Linear Inverse Problems. IDEI Working paper, n. 621.
Florens, J.P. & Simoni, A. (2012a) Nonparametric estimation of an instrumental regression: A quasi-Bayesian approach based on regularized posterior. Journal of Econometrics 170, 458475.
Florens, J.P. & Simoni, A. (2012b) Regularized posteriors in linear ill-posed inverse problems. Scandinavian Journal of Statistics 39, 214235.
Ghosh, J.K. & Ramamoorthi, R.V. (2003) Bayesian Nonparametrics. Springer Series in Statistics. Springer.
Hall, P. & Horowitz, J. (2005) Nonparametric methods for inference in the presence of instrumental variables. Annals of Statistics 33, 29042929.
Hall, P. & Horowitz, J.L. (2007) Methodology and convergence rates for functional linear regression. Annals of Statistics 35, 7091.
Helin, T. (2009) On infinite-dimensional hierarchical probability models in statistical inverse problems. Inverse Problems and Imaging 3, 567597.
Hoderlein, S., Nesheim, L., & Simoni, A. (2013) Semiparametric Estimation of Random Coefficients in Structural Economic Models. CEMMAP Working paper, CWP09/12.
Hofinger, A. & Pikkarainen, H.K. (2007) Convergence rate for the Bayesian approach to linear inverse problems. Inverse Problems 23, 24692484.
Hofinger, A. & Pikkarainen, H.K. (2009) Convergence rate for linear inverse problems in the presence of an additive normal noise. Stochastic Analysis and Applications 27, 240257.
Horowitz, J. (2014) Adaptive nonparametric instrumental variables estimation: Empirical choice of the regularisation parameter. Journal of Econometrics 180, 158173.
Johannes, J., Schenk, R., & Simoni, A. (2014) Adaptive Bayesian estimation in Gaussian sequence space models. In Bongiorno, E.G., Goia, A., Salinelli, E., & Vieu, P. (eds.), Contributions in Infinite-Dimensional Statistics and Related Topics, pp. 167172. Società Editrice Esculapio.
Kaipio, J. & Somersalo, E. (2004) Statistical and Computational Inverse Problems. Applied Mathematical Series, 160. Springer.
Kato, T. (1995) Perturbation Theory for Linear Operators. Springer.
Knapik, B.T., Van der Vaart, A.W., & Van Zanten, J.H. (2011) Bayesian inverse problems. Annals of Statistics 39, 26262657.
Krein, S.G. & Petunin, J.I. (1966) Scales of Banach spaces. Russian Mathematical Surveys 21, 85160.
Kress, R. (1999) Linear Integral Equation. Springer-Verlag.
Kuo, H.H. (1975) Gaussian Measures in Banach Spaces. Springer-Verlag.
Le Cam, L. (1986) Asymptotic Methods in Statistical Decision Theory. Springer-Verlag.
Liao, Y. & Jiang, W. (2011) Posterior consistency of nonparametric conditional moment restricted models. Annals of Statistics 39, 30033031.
Linton, O. & Mammen, E. (2005) Estimating semiparametric ARCH(∞) models by kernel smoothing methods. Econometrica 73, 771836.
Luschgy, H. (1995) Linear estimators and radonifying operators. Theory of Probability and Its Applications 40, 167175.
Mandelbaum, A. (1984) Linear estimators and measurable linear transformations on a Hilbert space. Z. Wahrcheinlichkeitstheorie 3, 385398.
Natterer, F. (1984) Error bounds for Tikhonov regularization in Hilbert scale. Applicable Analysis 18, 2937.
Newey, W.K. & Powell, J.L. (2003) Instrumental variable estimation of nonparametric models. Econometrica 71, 15651578.
Ruymgaart, F.H. (1998) A note on weak convergence of density in Hilbert spaces. Statistics 30, 331343.
Van der Vaart, A.W. & Van Zanten, J.H. (2009) Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth. Annals of Statistics 37, 26552675.
Van Rooij, A.C.M. & Ruymgaart, F.H. (1999) On inverse estimation. In Ghosh, S. (ed.), Asymptotic, Nonparametric and Time Series, pp. 579613. Dekker.
Zellner, A. (1986) On assessing prior distributions and Bayesian regression analysis with g-prior distribution. In Goel, P.K. & Zellner, A. (eds.), Bayesian Inference and Decision Techniques: Essays in Honour of Bruno de Finetti, vol. 6, pp. 233243. North Holland.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Econometric Theory
  • ISSN: 0266-4666
  • EISSN: 1469-4360
  • URL: /core/journals/econometric-theory
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 58 *
Loading metrics...

Abstract views

Total abstract views: 334 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 21st June 2018. This data will be updated every 24 hours.