Hostname: page-component-6766d58669-mzsfj Total loading time: 0 Render date: 2026-05-22T00:32:14.474Z Has data issue: false hasContentIssue false
  • English
  • Français

Regularized Partial and/or Constrained Redundancy Analysis

Published online by Cambridge University Press:  01 January 2025

Yoshio Takane  and
Sunho Jung
Yoshio Takane*
Affiliation:
McGill University
Sunho Jung
Affiliation:
McGill University
*
Requests for reprints should be sent to Yoshio Takane, Department of Psychology, McGill University, 1205 Dr. Penfield Avenue, Montreal, QC, H3A 1B1, Canada. E-mail: takane@psych.mcgill.ca
Get access Rights & Permissions [Opens in a new window]

Abstract

Methods of incorporating a ridge type of regularization into partial redundancy analysis (PRA), constrained redundancy analysis (CRA), and partial and constrained redundancy analysis (PCRA) were discussed. The usefulness of ridge estimation in reducing mean square error (MSE) has been recognized in multiple regression analysis for some time, especially when predictor variables are nearly collinear, and the ordinary least squares estimator is poorly determined. The ridge estimation method was extended to PRA, CRA, and PCRA, where the reduced rank ridge estimates of regression coefficients were obtained by minimizing the ridge least squares criterion. It was shown that in all cases they could be obtained in closed form for a fixed value of ridge parameter. An optimal value of the ridge parameter is found by G-fold cross validation. Illustrative examples were given to demonstrate the usefulness of the method in practical data analysis situations.

Keywords

reduced rank approximationscovariateslinear constraintsleast squares estimationridge least squares estimationgeneralized singular value decomposition (GSVD)G-fold cross validationbootstrap method

Information

Type
Original Paper
Information
Psychometrika , Volume 73 , Issue 4 , December 2008 , pp. 671 - 690
Copyright
Copyright © 2008 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

We thank Jim Ramsay for his insightful comments on an earlier draft of this paper. The work reported in this paper is supported by Grants 10630 from the Natural Sciences and Engineering Research Council of Canada to the first author.

References

Anderson, S.E., Coffey, B.S., & Byerly, R. (2002). Formal organizational initiatives and informal workplace practices: Links to work-family conflict and job-related outcomes. Journal of Management, 28, 787–810.Google Scholar
Anderson, T.W. (1951). Estimating linear restrictions on regression coefficients for multivariate normal distributions. Annals of Statistics, 22, 327–351.CrossRefGoogle Scholar
Breiman, L., & Friedman, J.H. (1997). Predicting multivariate responses in multiple linear regression. Journal of the Royal Statistical Society, Series B, 59, 3–54.CrossRefGoogle Scholar
Chinchilli, V.M., & Elswick, R.K. (1985). A mixture of the MANOVA and GMANOVA models. Communications in Statistics, Theory and Methods, 14, 3075–3089.CrossRefGoogle Scholar
D’Ambra, L., & Lauro, N. (1989). Non symmetrical analysis of three-way contingency tables. In Coppi, R., & Bolasco, S. (Eds.), Multiway data analysis, Amsterdam: North-Holland.Google Scholar
Duxbury, L., Higgins, C., & Lee, C. (1994). Work-family conflict: A comparison by gender, family type, and perceived control. Journal of Family Issues, 25, 449–466.CrossRefGoogle Scholar
Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap, Boca Raton: CRC Press.CrossRefGoogle Scholar
Hoerl, K.E., & Kennard, R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.CrossRefGoogle Scholar
Groß, J. (2003). Linear regression, Berlin: Springer.CrossRefGoogle Scholar
Lambert, Z.V., Wildt, A.R., & Durand, R.M. (1988). Redundancy analysis: An alternative to canonical correlation and multivariate multiple regression in exploring interset associations. Psychological Bulletin, 104, 282–289.CrossRefGoogle Scholar
Legendre, P., & Legendre, L. (1998). Numerical ecology, (2nd ed.). Oxford: Elsevier.Google Scholar
Reinsel, G.C., & Velu, R.P. (1998). Multivariate reduced-rank regression: Theory and applications, New York: Springer.CrossRefGoogle Scholar
Sparrow, P.R. (1996). Transitions in the psychological contract: Some evidence from the banking sector. Human Resource Management Journal, 6, 75–92.CrossRefGoogle Scholar
Takane, Y., & Hwang, H. (2002). Generalized constrained canonical correlation analysis. Multivariate Behavioral Research, 37, 163–195.CrossRefGoogle Scholar
Takane, Y., Hwang, H. (2006). Regularized multiple correspondence analysis. In Blasius, J., & Greenacre, M.J. (Eds.), Multiple correspondence analysis and related methods (pp. 259–279). London: Chapman and Hall.CrossRefGoogle Scholar
Takane, Y., & Hwang, H. (2007). Regularized linear and kernel redundancy analysis. Computational Statistics and Data Analysis, 52, 394–405.CrossRefGoogle Scholar
Takane, Y., & Shibayama, T. (1991). Principal component analysis with external information on both subjects and variables. Psychometrika, 56, 97–120.CrossRefGoogle Scholar
Takane, Y., & Yanai, H. (2008). On ridge operators. Linear Algebra and Its Applications, 428, 1778–1790.CrossRefGoogle Scholar
ten Berge, J.M.F. (1993). Least squares optimization in multivariate analysis, Leiden: DSWO Press.Google Scholar
ter Braak, C.J.F., & Šmilauer, P. (1998). CANOCO reference manual and user’s guide to Canoco for Windows, Ithaka: Microcomputer Power.Google Scholar
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.CrossRefGoogle Scholar
Van den Wollenberg, A.L. (1977). Redundancy analysis: Alternative for canonical analysis. Psychometrika, 42, 207–219.CrossRefGoogle Scholar
Velu, R.P. (1991). Reduced rank models with two sets of regressors. Applied Statistics, 40, 159–170.CrossRefGoogle Scholar
Yuan, M., Ekici, A., Lu, Z., & Monteiro, R. (2007). Dimension reduction and coefficient estimation in multivariate linear regression. Journal of the Royal Statistical Society, Series B, 69, 329–346.CrossRefGoogle Scholar