Hostname: page-component-89b8bd64d-nlwjb Total loading time: 0 Render date: 2026-05-06T23:30:12.223Z Has data issue: false hasContentIssue false
  • English
  • Français

Semi-sparse PCA

Published online by Cambridge University Press:  01 January 2025

Lars Eldén  and
Nickolay Trendafilov Open the ORCID record for Nickolay Trendafilov [Opens in a new window]
Lars Eldén
Affiliation:
Linköping University
Nickolay Trendafilov*
Affiliation:
The Open University
*
Correspondence should bemade to Nickolay Trendafilov, School of Mathematics and Statistics, The Open University, Milton Keynes, UK. Email:nickolay.trendafilov@open.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

It is well known that the classical exploratory factor analysis (EFA) of data with more observations than variables has several types of indeterminacy. We study the factor indeterminacy and show some new aspects of this problem by considering EFA as a specific data matrix decomposition. We adopt a new approach to the EFA estimation and achieve a new characterization of the factor indeterminacy problem. A new alternative model is proposed, which gives determinate factors and can be seen as a semi-sparse principal component analysis (PCA). An alternating algorithm is developed, where in each step a Procrustes problem is solved. It is demonstrated that the new model/algorithm can act as a specific sparse PCA and as a low-rank-plus-sparse matrix decomposition. Numerical examples with several large data sets illustrate the versatility of the new model, and the performance and behaviour of its algorithmic implementation.

Keywords

alternative factor analysismatrix decompositionsleast squaresStiefel manifoldsparse PCArobust PCA

Information

Type
Original Paper
Information
Psychometrika , Volume 84 , Issue 1 , 15 March 2019 , pp. 164 - 185
Copyright
Copyright © 2018 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Absil, P. -A., Mahony, R., & Sepulchre, R. (2008). Optimization Algorithms on Matrix Manifolds. Princeton: Princeton University Press. CrossRefGoogle Scholar
Adachi, K., & Trendafilov, N. (2017). Sparsest factor analysis for clustering variables: A matrix decomposition approach. Advances in Data Analysis and Classification, 12, 778– 794. Google Scholar
Aravkin, A., Becker, S., Cevher, V., & Olsen, P. (2014). A variational approach to stable principal component pursuit. In Conference on uncertainty in artificial intelligence (UAI).Google Scholar
Armstrong, S. A., Staunton, J. E., Silverman, L. B., Pieters, R., den Boer, M. L., Minden, M. D., Sallan, S. E., Lander, E. S., Golub, T. R., & Korsmeyer, S. J. (2002). MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nature Genetics, 30, 41– 47. CrossRefGoogle ScholarPubMed
Cai, J. -F., Candès, E. J., & Shen, Z. (2008). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20, 1956– 1982. CrossRefGoogle Scholar
Candès, E. J., Li, X., Ma, Y., & Wright, J. (2009). Robust principal component analysis?. Journal of ACM, 58, 1– 37. CrossRefGoogle Scholar
De Leeuw, J. van Montfort, K., Oud, J., & Satorra, A. (2004). Least squares optimal scaling of partially observed linear systems. Recent developments on structural equation models: Theory and applications, Dordrecht, NL: Kluwer Academic Publishers. 121– 134. CrossRefGoogle Scholar
Edelman, A., Arias, T. A., & Smith, S. T. (1998). The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20, 303– 353. CrossRefGoogle Scholar
Eldén, L. (2007). Matrix methods in data mining and pattern recognition, Philadelphia: SIAM. CrossRefGoogle Scholar
Golub, G. H., Van Loan, C. F. (2013). Matrix computations. (4). Baltimore, MD: Johns Hopkins University Press. Google Scholar
Harman, H. H. (1976). Modern factor analysis. (3). Chicago, IL: University of Chicago Press. Google Scholar
Jolliffe, I. T., Trendafilov, N. T., & Uddin, M. (2003). A modified principal component technique based on the LASSO. Journal of Computational and Graphical Statistics, 12, 531– 547. CrossRefGoogle Scholar
Journée, M., Nesterov, Y., Richtárik, P., & Sepulchre, R. (2010). Generalized power method for sparse principal component analysis. Journal of Machine Learning Research, 11, 517– 553. Google Scholar
Lin, Z., Chen, M., Wu, L., & Ma, Y. (2009). The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical Report, UILU-ENG-09-2215, November.Google Scholar
Lin, Z., Ganesh, A., Wright, J., Wu, L., Chen, M.,& Ma, Y. (2009). Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. UIUC Technical Report, UILU-ENG-09-2214, August.Google Scholar
Mulaik, S. A., Maydeu-Olivares, In A., & McArdle, J. J. (2005). Looking back on the factor indeterminacy controversies in factor analysis. Contemporary Psychometrics, Mahwah, NJ: Lawrence Erlbaum Associates Inc.. 174– 206. Google Scholar
Mulaik, S. A. (2010). The foundations of factor analysis, 2 Boca Raton, FL: Chapman and Hall/CRC. Google Scholar
Shen, H., & Huang, J. Z. (2008). Sparse principal component analysis via regularized low-rank matrix approximation. Journal of Multivariate Analysis, 99, 1015– 1034. CrossRefGoogle Scholar
Steiger, J. H. (1979). Factor indeterminacy in the 1930’s and the 1970’s: Some interesting parallels. Psychometrika, 44, 157– 166. CrossRefGoogle Scholar
Steiger, J. H., & Schonemann, P. H. (1978). A history of factor indeterminacy, Chicago, IL: University of Chicago Press. 136– 178. Google Scholar
Trendafilov, N., Fontanella, S., & Adachi, K. (2017). Sparse exploratory factor analysis. Psychometrika, 82, 778– 794. CrossRefGoogle ScholarPubMed
Trendafilov, N. T., & Unkel, S. (2011). Exploratory factor analysis of data matrices with more variables than observations. Journal of Computational and Graphical Statistics, 20, 874– 891. CrossRefGoogle Scholar
Unkel, S., & Trendafilov, N. T. (2010). Simultaneous parameter estimation in exploratory factor analysis: An expository review. International Statistical Review, 78, 363– 382. CrossRefGoogle Scholar
Witten, D. M., Tibshirani, R., & Hastie, T. (2009). A penalized matrix decomposition, with applications to sparse principal components and canonical correlation. Biostatistics, 10, 515– 534. CrossRefGoogle ScholarPubMed
Yuan, X., & Yang, J. (2013). Sparse and low-rank matrix decomposition via alternating direction methods. Pacific Journal of Optimization, 9, 167– 180. Google Scholar