Hostname: page-component-89b8bd64d-z2ts4 Total loading time: 0 Render date: 2026-05-08T09:43:55.595Z Has data issue: false hasContentIssue false
  • English
  • Français

Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper

Published online by Cambridge University Press:  01 January 2025

Kohei Adachi
Kohei Adachi*
Affiliation:
Osaka University
*
Requests for reprints should be sent to Kohei Adachi, Graduate School of Human Sciences, Osaka University, 1-2 Yamadaoka, Suita, Osaka 565-0871, Japan. E-mail: adachi@hus.osaka-u.ac.jp
Get access Rights & Permissions [Opens in a new window]

Abstract

Rubin and Thayer (Psychometrika, 47:69–76, 1982) proposed the EM algorithm for exploratory and confirmatory maximum likelihood factor analysis. In this paper, we prove the following fact: the EM algorithm always gives a proper solution with positive unique variances and factor correlations with absolute values that do not exceed one, when the covariance matrix to be analyzed and the initial matrices including unique variances and inter-factor correlations are positive definite. We further numerically demonstrate that the EM algorithm yields proper solutions for the data which lead the prevailing gradient algorithms for factor analysis to produce improper solutions. The numerical studies also show that, in real computations with limited numerical precision, Rubin and Thayer’s (Psychometrika, 47:69–76, 1982) original formulas for confirmatory factor analysis can make factor correlation matrices asymmetric, so that the EM algorithm fails to converge. However, this problem can be overcome by using an EM algorithm in which the original formulas are replaced by those guaranteeing the symmetry of factor correlation matrices, or by formulas used to prove the above fact.

Keywords

factor analysisEM algorithmimproper solutionsmaximum likelihood method

Information

Type
Original Paper
Information
Psychometrika , Volume 78 , Issue 2 , April 2013 , pp. 380 - 394
Copyright
Copyright © 2012 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

The author would like to thank the editor and anonymous associate editor and reviewers. This research was supported by grant (C)-23500347 from the Japan Society for the Promotion of Science.

References

Anderson, J.C., Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173CrossRefGoogle Scholar
Anderson, T.W., Rubin, H. (1956). Statistical inference in factor analysis. In Neyman, J. (Eds.), Proceedings of the third Berkeley symposium on mathematical statistics and probability, Berkeley: University of California Press 111150Google Scholar
Dempster, A.P., Laird, N.M., Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39, 1–38CrossRefGoogle Scholar
Gerbing, D.W., Anderson, J.C. (1987). Improper solutions in the analysis of covariance structures: their interpretability and a comparison of alternative respecifications. Psychometrika, 52, 99–111CrossRefGoogle Scholar
Jennrich, R.I., Robinson, S.M. (1969). A Newton–Raphson algorithm for maximum likelihood factor analysis. Psychometrika, 34, 111–123CrossRefGoogle Scholar
Jöreskog, K.G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, 443–482CrossRefGoogle Scholar
Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202CrossRefGoogle Scholar
Kano, Y. (1998). Causes and treatment of improper solutions: exploratory factor analysis. Bulletin of the Department of Human Sciences, Osaka University, 24, 303–327 (in Japanese)Google Scholar
Kuroda, M., Sakakihara, M. (2006). Accelerating the convergence of the EM algorithm using the vector epsilon algorithm. Computational Statistics & Data Analysis, 51, 1549–1561CrossRefGoogle Scholar
Lütkepohl, H. (1996). Handbook of matrices, Chichester: WileyGoogle Scholar
Magnus, J.R., Neudecker, H. (1991). Matrix differential calculus with applications in statistics and econometrics, (2nd ed.). Chichester: WileyGoogle Scholar
Maxwell, A.E. (1961). Recent trends in factor analysis. Journal of the Royal Statistical Society. Series A, 124, 49–59CrossRefGoogle Scholar
McDonald, R.P., Hartmann, W.M. (1992). A procedure for obtaining initial values of parameters in the RAM model. Multivariate Behavioral Research, 27, 57–76CrossRefGoogle ScholarPubMed
Minami, M. (2004). Convergence speed and acceleration of the EM algorithm. In Watanabe, M., Yamaguchi, K. (Eds.), The EM algorithm and related statistical models, New York: Marcel Dekker 8594Google Scholar
Mulaik, S.A. (2010). Foundations of factor analysis, (2nd ed.). Boca Raton: CRC PressGoogle Scholar
Rubin, D.B., Thayer, D.T. (1982). EM algorithms for ML factor analysis. Psychometrika, 47, 69–76CrossRefGoogle Scholar
SAS Institute Inc., (2009). SAS/STAT® 9.2 users guide, (2nd ed.). Cary: SAS Institute Inc. versionGoogle Scholar
Sato, M. (1987). Pragmatic treatment of improper solutions in factor analysis. Annals of the Institute of Statistical Mathematics, 39, 443–455CrossRefGoogle Scholar
Savalei, V., Kolenikov, S. (2008). Constrained versus unconstrained estimation in structural equation modeling. Psychological Methods, 13, 150–170CrossRefGoogle ScholarPubMed
Seber, G.A.F. (2008). A matrix handbook for statisticians, Hoboken: WileyGoogle Scholar
SPSS Inc., (1997). SPSS 7.5 statistical algorithm, Chicago: SPSS Inc.Google Scholar
Van Driel, O.P. (1978). On various causes of improper solutions in maximum likelihood factor analysis. Psychometrika, 43, 225–243CrossRefGoogle Scholar
Yanai, H., Ichikawa, M. (2007). Factor analysis. In Rao, C.R., Sinharay, S. (Eds.), Psychometrics, Amsterdam: Elsevier 257296Google Scholar