Skip to main content
    • Aa
    • Aa

A new analytical approach to consistency and overfitting in regularized empirical risk minimization


This work considers the problem of binary classification: given training data x 1, . . ., x n from a certain population, together with associated labels y 1,. . ., y n ∈ {0,1}, determine the best label for an element x not among the training data. More specifically, this work considers a variant of the regularized empirical risk functional which is defined intrinsically to the observed data and does not depend on the underlying population. Tools from modern analysis are used to obtain a concise proof of asymptotic consistency as regularization parameters are taken to zero at rates related to the size of the sample. These analytical tools give a new framework for understanding overfitting and underfitting, and rigorously connect the notion of overfitting with a loss of compactness.

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

[1] S. Agapiou , S. Larsson & A. M. Stuart (2013) Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems. Stoch. Process. Appl. 123, 38283860.

[6] G. Dal Maso (1993) An Introduction to Γ-Convergence, Springer, Birkhäuser Boston.

[7] E. Di Nezza , G. Palatucci & E. Valdinoci (2012) Hitchhiker's guide to the fractional Sobolev spaces. Bull. Sci. Math. 136, 521573.

[9] L. C. Evans (1990) Weak Convergence Methods for Nonlinear Partial Differential Equations, vol. 74, American Mathematical Soc, Providence, RI.

[12] N. García Trillos & D. Slepčev (2016) Continuum limit of total variation on point clouds. Arch. Ration. Mech. Anal. 220 (1), 193241.

[15] S. Ghosal , J. K. Ghosh & A. W. van der Vaart (2000) Convergence rates of posterior distributions. Ann. Statist. 28, 500531.

[18] M. Nikolova (2004) A variational approach to remove outliers and impulse noise. J. Math. Imaging Vision 20, 99120.

[19] P. Pedregal (1997) Parametrized Measures and Variational Principles, Progress in Nonlinear Differential Equations and their Applications, vol. 30, Birkhäuser Verlag, Basel.

[22] C. Villani (2003) Topics in Optimal Transportation, Graduate Studies in Mathematics, vol. 58, American Mathematical Society, Providence, RI.

[23] C. R. Vogel & M. E. Oman (1996) Iterative methods for total variation denoising. SIAM J. Sci. Comput. 17, 227238.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

European Journal of Applied Mathematics
  • ISSN: 0956-7925
  • EISSN: 1469-4425
  • URL: /core/journals/european-journal-of-applied-mathematics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 8 *
Loading metrics...

Abstract views

Total abstract views: 46 *
Loading metrics...

* Views captured on Cambridge Core between 20th July 2017 - 23rd September 2017. This data will be updated every 24 hours.