Skip to main content

On the discrepancy principle and generalised maximum likelihood for regularisation

  • Mark A. Lukas (a1)

Let fnλ be the regularised solution of a general, linear operator equation, K f0 = g, from discrete, noisy data yi = g(xi ) + εi, i = 1, …, n, where εi are uncorrelated random errors with variance σ2. In this paper, we consider the two well–known methods – the discrepancy principle and generalised maximum likelihood (GML), for choosing the crucial regularisation parameter λ. We investigate the asymptotic properties as n → ∞ of the “expected” estimates λD and λM corresponding to these two methods respectively. It is shown that if f0 is sufficiently smooth, then λD is weakly asymptotically optimal (ao) with respect to the risk and an L2 norm on the output error. However, λD oversmooths for all sufficiently large n and also for all sufficiently small σ2. If f0 is not too smooth relative to the regularisation space W, then λD can also be weakly ao with respect to a whole class of loss functions involving stronger norms on the input error. For the GML method, we show that if f0 is smooth relative to W (for example f0Wθ, 2, θ > m, if W = Wm, 2), then λM is asymptotically sub-optimal and undersmoothing with respect to all of the loss functions above.

Hide All
[1]Anderssen, R.S. and Bloomfield, P., ‘Numerical differentiation procedures for non-exact data’, Numer. Math. 22 (1974), 157182.
[2]Cox, D.D., ‘Approximation of regularization estimators’, Ann. Statist. 16 (1988), 694712.
[3]Craven, P. and Wahba, G., ‘Smoothing noisy data with spline functions’, Numer. Math. 31 (1979), 377403.
[4]Davies, A.R., ‘On the maximum likelihood regularization of Fredholm convolution equations of the first kind’, in Treatment of integral equations by numerical methods, (Baker, C.T.H. and Miller, G.F., Editors) (Academic Press, London, 1982), pp. 95105.
[5]Davies, A.R., Iqbal, M., Maleknejad, K. and Redshaw, T.C., ‘A comparison of statistical regularization and Fourier extrapolation methods for numerical deconvolution’, in Numerical treatment of inverse problems in differential and integral equations, (Deuflhard, P. and Hairer, E., Editors)(Birkhäuser, Berlin, 1983), pp. 320334.
[6]Davies, A.R. and Anderssen, R.S., ‘Improved estimates of statistical regularization parameters in Fourier differentiation and smoothing’, Numer. Math. 48 (1986), 671697.
[7]Davies, A.R. and Anderssen, R.S., ‘Optimization in the regularization of ill-posed problems’, J. Austral. Math. Soc. Ser. B 28 (1986), 114133.
[8]Engl, H.W. and Gfrerer, H., ‘A posteriori parameter choice for general regularization methods for solving linear ill-posed problems’, Appl. Numer. Math. 4 (1988), 395417.
[9]Kimeldorf, G.S. and Wahba, G., ‘Some results on Tchebycheffian spline functions’, J. Math. Anal. Appl. 33 (1971), 8295.
[10]Lukas, M.A., ‘Regularization’, in The application and numerical solution of integral equations, (Anderssen, R.S., de Hoog, F.R. and Lukas, M.A., Editors)(Sijthoff and Noordhoff, Alphen aan den Rijn, The Netherlands, 1980), pp. 151182.
[11]Lukas, M.A., ‘Convergence rates for regularized solutions’, Math. Comp. 51 (1988), 107131.
[12]Lukas, M.A., ‘Assessing regularised solutions’, J. Austral. Math. Soc. Ser. B 30 (1988), 2442..
[13]Lukas, M.A., ‘Methods for choosing the regularization parameter’, in Inverse problems in partial differential equations, (Pani, A.K. and Anderssen, R.S., Editors), Proceedings of the Centre for Mathematics and its Applications (Australian National University, Canberra, ACT, 1992), pp. 89110.
[14]Lukas, M.A., ‘Asymptotic optimality of generalized cross-validation for choosing the reg-ularization parameter’, Numer. Math. 66 (1993), 4166.
[15]Lukas, M.A., ‘Convergence rates for moment collocation solutions of linear operator equations’, Numer. Funct. Anal. Optim. (to appear).
[16]Morozov, V.A., ‘On the solution of functional equations by the method of regularization’. Soviet Math. Dokl. 7 (1966), 414417.
[17]Morozov, V.A., Methods for solving incorrectly posed problems (Springer-Verlag, Berlin, Heidelberg, New York, 1984).
[18]Nychka, D.W. and Cox, D.D., ‘Convergence rates for regularized solutions of integral equations from discrete noisy data’, Ann. Statist. 17 (1989), 556572.
[19]Wahba, G., ‘Convergence rates of certain approximate solutions to Fredholm integral equations of the first kind’, J. Approx. Theory 7 (1973), 167185.
[20]Wahba, G., ‘Smoothing noisy data by spline functions’, Numer. Math. 24 (1975), 383393.
[21]Wahba, G., ‘Practical approximate solutions to linear operator equations when the data are noisy’, SIAM J. Numer. Anal. 14 (1977), 651667.
[22]Wahba, G., ‘A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem’, Ann. Statist. 13 (1985), 13781402.
[23]Wahba, G., Spline models for observational data, CBMS-NSF Regional Conference Series in Applied Mathematics 59 (SIAM, Philadelphia, 1990).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Bulletin of the Australian Mathematical Society
  • ISSN: 0004-9727
  • EISSN: 1755-1633
  • URL: /core/journals/bulletin-of-the-australian-mathematical-society
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed