Hostname: page-component-8448b6f56d-mp689 Total loading time: 0 Render date: 2024-04-24T09:43:32.858Z Has data issue: false hasContentIssue false
  • English
  • Français

Convergence of the backfitting algorithm for additive models

Part of: Linear inference, regression Numerical approximation and computational geometry Nonparametric inference

Published online by Cambridge University Press:  09 April 2009

Craig F. Ansley  and
Robert Kohn
Craig F. Ansley
Affiliation:
Department of Accounting and Finance, University of Auckland, Private Bag, Auckland, New Zealand
Robert Kohn
Affiliation:
Australian Graduate School of Management, University of NSW, Kensington, New South Wales, Australia
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The backfitting algorithm is an iterative procedure for fitting additive models in which, at each step, one component is estimated keeping the other components fixed, the algorithm proceeding component by component and iterating until convergence. Convergence of the algorithm has been studied by Buja, Hastie, and Tibshirani (1989). We give a simple, but more general, geometric proof of the convergence of the backfitting algorithm when the additive components are estimated by penalized least squares. Our treatment covers spline smoothers and structural time series models, and we give a full discussion of the degenerate case. Our proof is based on Halperin's (1962) generalization of von Neumann's alternating projection theorem.

Keywords

Additive modelalternating projectionconcurvitynonparametric regressionpenalized least squarespartial splinetrendseasonal.

MSC classification

Secondary: 62G05: Estimation 62J02: General nonlinear regression 65D10: Smoothing, curve fitting
Type
Research Article
Information
Journal of the Australian Mathematical Society , Volume 57 , Issue 3 , December 1994 , pp. 316 - 329
Copyright
Copyright © Australian Mathematical Society 1994

References

Buja, A., Hastie, T. & Tibshirani, R. (1989), ‘Linear smoothers and additive models (with discussion)’, Ann. Statist. 17, 453–555.Google Scholar
Cogburn, R. & Davis, H. T. (1974), ‘Periodic spline and spectral estimation’, Ann. Statist. 2, 1108–1126.CrossRefGoogle Scholar
Elfving, T. (1980), ‘Block-iterative methods for consistent and inconsistent linear equations’, Numer. Math. 35, 1–12.CrossRefGoogle Scholar
Friedman, J. H. & Stuetzle, W. (1981), ‘Projection pursuit regression’, J. Amer. Statist. Assoc. 76, 817–823.CrossRefGoogle Scholar
Halperin, I. (1962), ‘The product of projection operators’, Acta Sci. Math. 23, 96–99.Google Scholar
Hutchinson, M. F. & de Hoog, F. R. (1985), ‘Smoothing noisy data with spline functions’, Numer. Math. 47, 99–106.CrossRefGoogle Scholar
Kaczmarz, S. (1937), ‘Angenaherte auflosung von systemen linearer gleichungen’, Bull. Acad. Polon. Sci. pp. 335–357.Google Scholar
Kitagawa, G. & Gersch, W. (1984), ‘A smoothness priors-state space modeling of time series with trend and seasonality’, J. Amer. Statist. Assoc. 79, 378–389.Google Scholar
Kohn, R. & Ansley, C. F. (1991), ‘A signal extraction approach to the estimation of treatment and control curves’, J. Amer. Statist. Assoc. 86, 1034–1041.CrossRefGoogle Scholar
von Neumann, J. (1950), Functional operators Vol. II: The Geometry of Orthogonal Spaces, Vol. 22 of Ann. of Math. Stud., Princeton University Press, Princeton.Google Scholar
Silverman, B. W. & Wood, J. T. (1987), ‘The nonparametric estimation of branching curves’, J. Amer. Statist. Assoc. 82, 551–558.Google Scholar
Wahba, G. (1980), ‘Automatic smoothing of the log periodogram’, J. Amer. Statist. Assoc. 75, 122–132.Google Scholar
Wecker, W. E. & Ansley, C. F. (1982), ‘Nonparametric multiple regression by the alternating projection method’, in Proc. ASA Bus. Econ. Statist. Section, pp. 311–316.Google Scholar
Weinert, H. L. & Sidhu, G. S. (1978), ‘A stochastic framework for recursive computation of spline functions: Part I, Interpolating splines’, IEEE Trans. Inform. Theory 24, 45–50.Google Scholar
Weinert, H. L., Byrd, R. H. & Sidhu, G. S. (1980), ‘A stochastic framework for recursive computation of spline functions: Part II, Smoothing splines’, J. Optim. Theory Appl. 30, 255–268.Google Scholar