Hostname: page-component-6766d58669-l4t7p Total loading time: 0 Render date: 2026-05-23T06:40:01.710Z Has data issue: false hasContentIssue false
  • English
  • Français

High-dimensional Exploratory Item Factor Analysis by A Metropolis–Hastings Robbins–Monro Algorithm

Published online by Cambridge University Press:  01 January 2025

Li Cai
Li Cai*
Affiliation:
University of California, Los Angeles
*
Requests for reprints should be sent to Li Cai, GSE & IS, UCLA, Los Angeles, CA, USA 90095-1521. E-mail: lcai@ucla.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

A Metropolis–Hastings Robbins–Monro (MH-RM) algorithm for high-dimensional maximum marginal likelihood exploratory item factor analysis is proposed. The sequence of estimates from the MH-RM algorithm converges with probability one to the maximum likelihood solution. Details on the computer implementation of this algorithm are provided. The accuracy of the proposed algorithm is demonstrated with simulations. As an illustration, the proposed algorithm is applied to explore the factor structure underlying a new quality of life scale for children. It is shown that when the dimensionality is high, MH-RM has advantages over existing methods such as numerical quadrature based EM algorithm. Extensions of the algorithm to other modeling frameworks are discussed.

Keywords

stochastic approximationSAitem response theoryIRTMarkov chain Monte CarloMCMCnumerical integrationcategorical factor analysislatent variable modelingstructural equation modeling

Information

Type
Theory and Methods
Information
Psychometrika , Volume 75 , Issue 1 , March 2010 , pp. 33 - 57
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NC
This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Copyright
Copyright © 2009 The Psychometric Society

Footnotes

I thank the editor, the AE, and the reviewers for helpful suggestions. I am indebted to Drs. Chuanshu Ji, Robert MacCallum, and Zhengyuan Zhu for helpful discussions. I would also like to thank Drs. Mike Edwards and David Thissen for supplying the data sets used in the numerical demonstrations. The author gratefully acknowledges financial support from Educational Testing Service (the Gulliksen Psychometric Research Fellowship program), National Science Foundation (SES-0717941), National Center for Research on Evaluation, Standards and Student Testing (CRESST) through award R305A050004 from the US Department of Education’s Institute of Education Sciences (IES), and a predoctoral advanced quantitative methods training grant awarded to the UCLA Departments of Education and Psychology from IES. The views expressed in this paper are of the author’s alone and do not reflect the views or policies of the funding agencies.

References

Albert, J.H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling. Journal of Educational Statistics, 17, 251–269.CrossRefGoogle Scholar
Aptech Systems, Inc. (2003). GAUSS (Version 6.08) [Computer software], Maple Valley: Author.Google Scholar
Baker, F.B., & Kim, S.-H. (2004). Item response theory: Parameter estimation techniques, New York: Dekker.CrossRefGoogle Scholar
Bartholomew, D.J., & Knott, M. (1999). Latent variable models and factor analysis, (2nd ed.). London: Arnold.Google Scholar
Bartholomew, D.J., & Leung, S.O. (2002). A goodness of fit test for sparse 2p contingency tables. British Journal of Mathematical and Statistical Psychology, 55, 1–15.CrossRefGoogle ScholarPubMed
Béguin, A.A., & Glas, C.A.W. (2001). MCMC estimation and some model-fit analysis of multidimensional IRT models. Psychometrika, 66, 541–561.CrossRefGoogle Scholar
Benveniste, A., Métivier, M., & Priouret, P. (1990). Adaptive algorithms and stochastic approximations, Berlin: Springer.CrossRefGoogle Scholar
Bishop, Y.M.M., Fienberg, S.E., & Holland, P.W. (1975). Discrete multivariate analysis: Theory and practice, Cambridge: MIT Press.Google Scholar
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Bock, R.D., Gibbons, R., & Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261–280.CrossRefGoogle Scholar
Bock, R.D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179–197.CrossRefGoogle Scholar
Bolt, D. (2005). Limited and full information estimation of item response theory models. In Maydeu-Olivares, A., & McArdle, J.J. (Eds.), Contemporary psychometrics (pp. 27–71). Mahwah: Earlbaum.Google Scholar
Booth, J.G., & Hobert, J.P. (1999). Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. Journal of the Royal Statistical Society—Series B, 61, 265–285.CrossRefGoogle Scholar
Borkar, V.S. (2008). Stochastic approximation: A dynamical systems viewpoint, Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Browne, M.W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111–150.CrossRefGoogle Scholar
Browne, M.W., Cudeck, R., Tateneni, K., & Mels, G. (2008). CEFA: Comprehensive Exploratory Factor Analysis (Version 3.02) [Computer software]. Retrieved from http://quantrm2.psy.ohio-state.edu/browne/.Google Scholar
Cai, L. (2006). Full-information item factor analysis by Markov chain Monte Carlo stochastic approximation. Unpublished master’s thesis, Department of Statistics, University of North Carolina at Chapel Hill.Google Scholar
Cai, L. (2008a). A Metropolis–Hastings Robbins–Monro algorithm for maximum likelihood nonlinear latent structure analysis with a comprehensive measurement model. Unpublished doctoral dissertation, Department of Psychology, University of North Carolina at Chapel Hill.Google Scholar
Cai, L. (2008). SEM of another flavour: Two new applications of the supplemented EM algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309–329.CrossRefGoogle ScholarPubMed
Cai, L., du Toit, S.H.C., & Thissen, D. (2009). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling [Computer software], Chicago: SSI International.Google Scholar
Cai, L., Maydeu-Olivares, A., Coffman, D.L., & Thissen, D. (2006). Limited-information goodness-of-fit testing of item response theory models for sparse 2p tables. British Journal of Mathematical and Statistical Psychology, 59, 173–194.CrossRefGoogle ScholarPubMed
Camilli, G. (1994). Origin of the scaling constant d=1.7 in item response theory. Journal of Educational and Behavioral Statistics, 19, 379–388.Google Scholar
Celeux, G., Chauveau, D., & Diebolt, J. (1995). On stochastic versions of the EM algorithm (Tech. Rep. No. 2514). The French National Institute for Research in Computer Science and Control.Google Scholar
Celeux, G., & Diebolt, J. (1991). A stochastic approximation type EM algorithm for the mixture problem (Tech. Rep. No. 1383). The French National Institute for Research in Computer Science and Control.Google Scholar
Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm. The American Statistician, 49, 327–335.CrossRefGoogle Scholar
de Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach, New York: Springer.CrossRefGoogle Scholar
Delyon, B., Lavielle, M., & Moulines, E. (1999). Convergence of a stochastic approximation version of the EM algorithm. The Annals of Statistics, 27, 94–128.CrossRefGoogle Scholar
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society—Series B, 39, 1–38.CrossRefGoogle Scholar
Diebolt, J., & Ip, E.H.S. (1996). Stochastic EM: Method and application. In Gilks, W.R., Richardson, S., & Spiegelhalter, D.J. (Eds.), Markov chain Monte Carlo in practice (pp. 259–273). London: Chapman and Hall.Google Scholar
Dunson, D.B. (2000). Bayesian latent variable models for clustered mixed outcomes. Journal of the Royal Statistical Society—Series B, 62, 355–366.CrossRefGoogle Scholar
Edwards, M.C. (2005). A Markov chain Monte Carlo approach to confirmatory item factor analysis. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill.Google Scholar
Fisher, R.A. (1925). Theory of statistical estimation. Proceedings of the Cambridge Philosophical Society, 22, 700–725.CrossRefGoogle Scholar
Fox, J.-P. (2003). Stochastic EM for estimating the parameters of a multilevel IRT model. British Journal of Mathematical and Statistical Psychology, 56, 65–81.CrossRefGoogle ScholarPubMed
Fox, J.-P. (2005). Multilevel IRT using dichotomous and polytomous response data. British Journal of Mathematical and Statistical Psychology, 58, 145–172.CrossRefGoogle ScholarPubMed
Fox, J.-P., & Glas, C.A.W. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 269–286.CrossRefGoogle Scholar
Gelfand, A.E., & Smith, A.F.M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.CrossRefGoogle Scholar
Gu, M.G., & Kong, F.H. (1998). A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems. The Proceedings of the National Academy of Sciences, 95, 7270–7274.CrossRefGoogle ScholarPubMed
Gu, M.G., Sun, L., & Huang, C. (2004). A universal procedure for parametric frailty models. Journal of Statistical Computation and Simulation, 74, 1–13.CrossRefGoogle Scholar
Gu, M.G., & Zhu, H.-T. (2001). Maximum likelihood estimation for spatial models by Markov chain Monte Carlo stochastic approximation. Journal of the Royal Statistical Society—Series B, 63, 339–355.CrossRefGoogle Scholar
Gueorguieva, R.V., & Agresti, A. (2001). A correlated probit model for joint modeling of clustered binary and continuous responses. Journal of the American Statistical Association, 96, 1102–1112.CrossRefGoogle Scholar
Haberman, S.J. (1977). Log-linear models and frequency tables with small expected cell counts. The Annals of Statistics, 5, 1148–1169.CrossRefGoogle Scholar
Hastings, W.K. (1970). Monte Carlo simulation methods using Markov chains and their applications. Biometrika, 57, 97–109.CrossRefGoogle Scholar
Huber, P., Ronchetti, E., & Victoria-Feser, M.-P. (2004). Estimation of generalized linear latent variable models. Journal of the Royal Statistical Society—Series B, 66, 893–908.CrossRefGoogle Scholar
Jank, W.S. (2004). Quasi-Monte Carlo sampling to improve the efficiency of Monte Carlo EM. Computational Statistics and Data Analysis, 48, 685–701.CrossRefGoogle Scholar
Joe, H. (2008). Accuracy of Laplace approximation for discrete response mixed models. Computational Statistics and Data Analysis, 52, 5066–5074.CrossRefGoogle Scholar
Kass, R., & Steffey, D. (1989). Approximate Bayesian inference in conditionally independent hierarchical models. Journal of the American Statistical Association, 84, 717–726.CrossRefGoogle Scholar
Kuhn, E., & Lavielle, M. (2005). Maximum likelihood estimation in nonlinear mixed effects models. Computational Statistics and Data Analysis, 49, 1020–1038.CrossRefGoogle Scholar
Kullback, S., & Leibler, R.A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22, 79–86.CrossRefGoogle Scholar
Kushner, H.J., & Yin, G.G. (1997). Stochastic approximation algorithms and applications, New York: Springer.CrossRefGoogle Scholar
Lange, K. (1995). A gradient algorithm locally equivalent to the EM algorithm. Journal of the Royal Statistical Society—Series B, 57, 425–437.CrossRefGoogle Scholar
Liu, Q., & Pierce, D.A. (1994). A note on Gauss–Hermite quadrature. Biometrika, 81, 624–629.Google Scholar
Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores, Reading: Addison-Wesley.Google Scholar
Louis, T.A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society—Series B, 44, 226–233.CrossRefGoogle Scholar
Makowski, D., & Lavielle, M. (2006). Using SAEM to estimate parameters of models of response to applied fertilizer. Journal of Agricultural, Biological, and Environmental Statistics, 11, 45–60.CrossRefGoogle Scholar
Mardia, K.V., Kent, J.T., & Bibby, J.M. (1979). Multivariate analysis, San Diego: Academic Press.Google Scholar
Maydeu-Olivares, A., & Cai, L. (2006). A cautionary note on using g 2(dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41, 55–64.CrossRefGoogle Scholar
Maydeu-Olivares, A., & Joe, H. (2005). Limited and full information estimation and testing in 2n contingency tables: A unified framework. Journal of the American Statistical Association, 100, 1009–1020.CrossRefGoogle Scholar
McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society—Series B, 42, 109–142.CrossRefGoogle Scholar
McCullagh, P., & Nelder, J.A. (1989). Generalized linear models, (2nd ed.). London: Chapman & Hall.CrossRefGoogle Scholar
McCulloch, C.E., & Searle, S.R. (2001). Generalized, linear, and mixed models, New York: Wiley.Google Scholar
Meng, X.-L., & Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling. Journal of the American Statistical Association, 91, 1254–1267.CrossRefGoogle Scholar
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., & Teller, E. (1953). Equations of state space calculations by fast computing machines. Journal of Chemical Physics, 21, 1087–1092.CrossRefGoogle Scholar
Mislevy, R.J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177–195.CrossRefGoogle Scholar
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115–132.CrossRefGoogle Scholar
Muthén, , Muthén, (2008). Mplus (Version 5.0) [Computer software], Los Angeles: Author.Google Scholar
Natarajan, R., & Kass, R.E. (2000). Reference Bayesian methods for generalized linear mixed models. Journal of the American Statistical Association, 95, 227–237.CrossRefGoogle Scholar
Naylor, J.C., & Smith, A.F.M. (1982). Applications of a method for the efficient computation of posterior distributions. Journal of the Royal Statistical Society—Series C, 31, 214–225.Google Scholar
Orchard, T., & Woodbury, M.A. (1972). A missing information principle: Theory and application. In Lecam, L.M., Neyman, J., & Scott, E.L. (Eds.), Proceedings of the sixth Berkeley symposium on mathematical statistics and probability (pp. 697–715). Berkeley: University of California Press.Google Scholar
Patz, R.J., & Junker, B.W. (1999). A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146–178.CrossRefGoogle Scholar
Patz, R.J., & Junker, B.W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342–366.CrossRefGoogle Scholar
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika, 69, 167–190.CrossRefGoogle Scholar
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2005). Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. Journal of Econometrics, 128, 301–323.CrossRefGoogle Scholar
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004a). GLLAMM manual (U.C. Berkeley Division of Biostatistics Working Paper Series, 160).Google Scholar
Raudenbush, S.W., Yang, M.-L., & Yosef, M. (2000). Maximum likelihood for generalized linear models with nested random effects via high-order, multivariate Laplace approximation. Journal of Computational and Graphical Statistics, 9, 141–157.CrossRefGoogle Scholar
Reeve, B.B., Hays, R.D., Bjorner, J.B., Cook, K.F., Crane, P.K., & Teresi, J.A.et al. (2007). Psychometric evaluation and calibration of health-related quality of life items banks: Plans for the patient-reported outcome measurement information system (PROMIS). Medical Care, 45, S22–31.CrossRefGoogle Scholar
Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 22, 400–407.CrossRefGoogle Scholar
Roberts, G.O., & Rosenthal, J.S. (2001). Optimal scaling for various Metropolis-Hastings algorithms. Statistical Science, 16, 351–367.CrossRefGoogle Scholar
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monographs, 17.CrossRefGoogle Scholar
Savalei, V. (2006). Logistic approximation to the normal: The KL rationale. Psychometrika, 71, 763–767.CrossRefGoogle Scholar
Schilling, S., & Bock, R.D. (2005). High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika, 70, 533–555.CrossRefGoogle Scholar
Segall, D.O. (1998). IFACT computer program Version 1.0: Full information confirmatory item factor analysis using Markov chain Monte Carlo estimation [Computer software], Seaside: Defense Manpower Data Center.Google Scholar
Shi, J.-Q., & Lee, S.-Y. (1998). Bayesian sampling-based approach for factor analysis models with continuous and polytomous data. British Journal of Mathematical and Statistical Psychology, 51, 233–252.CrossRefGoogle Scholar
Song, X.-Y., & Lee, S.-Y. (2005). A multivariate probit latent variable model for analyzing dichotomous responses. Statistica Sinica, 15, 645–664.Google Scholar
te Marvelde, J., Glas, v.G.C., & van Damme, J. (2006). Application of multidimensional item response theory models to longitudinal data. Educational and Psychological Measurement, 66, 5–34.CrossRefGoogle Scholar
Thissen, D. (2003). MULTILOG 7 user’s guide, Chicago: SSI International.Google Scholar
Thomas, N. (1993). Asymptotic corrections for multivariate posterior moments with factored likelihood functions. Journal of Computational and Graphical Statistics, 2, 309–322.CrossRefGoogle Scholar
Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). The Annals of Statistics, 22, 1701–1762.Google Scholar
Tierney, L., & Kadane, J.B. (1986). Accurate approximations for posterior moments and marginal densities. Journal of the American Statistical Association, 81, 82–86.CrossRefGoogle Scholar
Titterington, D.M. (1984). Recursive parameter estimation using incomplete data. Journal of the Royal Statistical Society—Series B, 46, 257–267.CrossRefGoogle Scholar
Wainer, H., Kiely, G. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–202.CrossRefGoogle Scholar
Wei, G.C.G., & Tanner, M.A. (1990). A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithm. Journal of the American Statistical Association, 85, 699–704.CrossRefGoogle Scholar
Wirth, R.J., & Edwards, M.C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.CrossRefGoogle ScholarPubMed
Zhu, H.-T., & Lee, S.-Y. (2002). Analysis of generalized linear mixed models via a stochastic approximation algorithm with Markov chain Monte-Carlo method. Statistics and Computing, 12, 175–183.CrossRefGoogle Scholar
Zimowski, M.F., Muraki, E., Mislevy, R.J., & Bock, R.D. (2003). BILOG-MG3 user’s guide, Chicago: SSI International.Google Scholar