Skip to main content

Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models

  • Daniel L. Oberski (a1)

Latent variable models can only be compared across groups when these groups exhibit measurement equivalence or “invariance,” since otherwise substantive differences may be confounded with measurement differences. This article suggests examining directly whether measurement differences present could confound substantive analyses, by examining the expected parameter change (EPC)-interest. The EPC-interest approximates the change in parameters of interest that can be expected when freeing cross-group invariance restrictions. Monte Carlo simulations suggest that the EPC-interest approximates these changes well. Three empirical applications show that the EPC-interest can help avoid two undesirable situations: first, it can prevent unnecessarily concluding that groups are incomparable, and second, it alerts the user when comparisons of interest may still be invalidated even when the invariance model appears to fit the data. R code and data for the examples discussed in this article are provided in the electronic appendix (

Hide All
Armstrong, D. A. 2011. Stability and change in the Freedom House political rights and civil liberties measures. Journal of Peace Research 48: 653–62.
Bartholomew, D. J., Knott, M., and Moustaki, I. 2011. Latent variable models and factor analysis: A unified approach. New York: John Wiley & Sons.
Bentler, P. M., and Chou, C. P. 1992. Some new covariance structure model improvement statistics. Sociological Methods & Research 21: 259–82.
Bollen, K. A. 1989. Structural equations with latent variables. New York: John Wiley & Sons.
Byrne, B. M., Shavelson, R. J., and Muthén, Bengt. 1989. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin 105: 456.
Chen, F. F. 2007. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling 14: 464504.
Cheung, G. W., and Rensvold, R. B. 2002. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling 9: 233–55.
Chou, C. P., and Bentler, P. M. 1993. Invariant standardized estimated parameter change for model modification in covariance structure analysis. Multivariate Behavioral Research 28: 97110.
Clinton, J., Jackman, S., and Rivers, D. 2004. The statistical analysis of roll call data. American Political Science Review 98: 355–70.
Davidov, E. 2009. Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis 17: 6482.
Davidov, E., Meuleman, B., Billiet, J., and Schmidt, P. 2008. Values and support for immigration: A cross-country comparison. European Sociological Review 24: 583–99.
Fan, Xitao, Thompson, Bruce, and Wang, Lin. 1999. Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal 6: 5683.
Fan, Weihua, and Hancock, Gregory R. 2006. Impact of post hoc measurement model overspecification on structural parameter integrity. Educational and Psychological Measurement 66: 748–64.
French, B. F., and Finch, W. H. 2006. Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling 13: 378402.
Hancock, Gregory R. 1999. A sequential Scheffé-type respecification procedure for controlling type I error in exploratory structural equation model modification. Structural Equation Modeling: A Multidisciplinary Journal 6: 158–68.
Hancock, Gregory R., Stapleton, Laura M., and Arnold-Berkovits, Ilona. 2009. The tenuousness of invariance tests within multisample covariance and mean structure models. In structural equation modeling in educational research: Concepts and applications, eds. Teo, T. and Khine, M. S., 137–74. Rotterdam, The Netherlands: Sense Publishers.
Hausman, J. A. 1978. Specification tests in econometrics. Econometrica: Journal of the Econometric Society 46(6): 1251–71.
Hu, L., and Bentler, P. M. 1998. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods 3: 424.
Imai, K., and Yamamoto, T. 2010. Causal inference with differential measurement error: Nonparametric identification and sensitivity analysis. American Journal of Political Science 54: 543–60.
Jackman, S. 2001. Multidimensional analysis of roll call data via Bayesian simulation: Identification, estimation, inference, and model checking. Political Analysis 9: 227–41.
Jowell, Roger, Roberts, Caroline, Fitzgerald, Rory, and Eva, Gillian. 2007. Measuring attitudes cross-nationally: Lessons from the European Social Survey. Thousand Oaks, CA: SAGE.
Kaplan, D. 1989. Model modification in covariance structure analysis: Application of the expected parameter change statistic. Multivariate Behavioral Research 24: 285305.
King, B. L. 2011. Unbiased measurement of health-related quality-of-life. Unpublished PhD dissertation, University of Amsterdam. (accessed June 1, 2013).
Kolenikov, S. 2009. Biases of parameter estimates in misspecified structural equation models. Sociological Methodology 41: 119–57.
Kwok, Oi-Man, Luo, Wen, and West, Stephen G. 2010. Using modification indexes to detect turning points in longitudinal data: A Monte Carlo study. Structural Equation Modeling 17: 216–40.
Lord, F. M., and Novick, M. R. 1968. Statistical theories of mental scores. Reading, MA: Addison-Wesley.
MacCallum, R. C., Roznowski, M., and Necowitz, L. B. 1992. Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin; Psychological Bulletin 111: 490.
Magnus, J. R., and Vasnev, A. L. 2007. Local sensitivity and diagnostic tests. Econometrics Journal 10: 166–92.
Meredith, W. 1993. Measurement invariance, factor analysis and factorial invariance. Psychometrika 58: 525–43.
Meuleman, B. 2012. When are item intercept differences substantively relevant in measurement invariance testing? In Methods, theories, and empirical applications in the social sciences: Festschrift for Peter Schmidt, eds. Salzborn, S., Davidov, E., and Reinecke, J., 97104. Wiesbaden, Germany: Springer-Verlag.
Millsap, R. E. 1997. Invariance in measurement and prediction: Their relationship in the single-factor case. Psychological Methods 2: 248.
Millsap, R. E. 2007. Invariance in measurement and prediction revisited. Psychometrika 72: 461–73.
Millsap, R. E., and Everson, H. T. 1993. Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement 17: 297334.
Millsap, R. E., and Kwok, O. M. 2004. Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods 9: 93.
Neudecker, H., and Satorra, A. 1991. Linear structural relations: Gradient and Hessian of the fitting function. Statistics and Probability Letters 11: 5761.
Oberski, D. L. 2012. Comparability of survey measurements. In Handbook of survey methodology for the social sciences, ed. Gideon, Lior, 477–98. New York: Springer-Verlag.
R Core Team. 2012. R: A language and environment for statistical computing . Vienna, Austria: R Foundation for Statistical Computing. (accessed June 1, 2013).
Rosseel, Y. 2012. lavaan: An R package for structural equation modeling. Journal of Statistical Software 48: 136.
Saris, W. E., Satorra, A., and Sörbom, D. 1987. The detection and correction of specification errors in structural equation models. Sociological Methodology 17: 105–29.
Saris, W. E., Satorra, A., and Van der Veld, W. M. 2009. Testing structural equation models or detection of misspecifications? Structural Equation Modeling 16: 561–82.
Satorra, A. 1989. Alternative test criteria in covariance structure analysis: A unified approach. Psychometrika 54: 131–51.
Schmitt, N., and Kuljanin, G. 2008. Measurement invariance: Review of practice and implications. Human Resource Management Review 18: 210–22.
Schwartz, S. H., and Bilsky, W. 1987. Toward a universal psychological structure of human values. Journal of Personality and Social Psychology 53: 550.
Schwartz, S. H., and Rubel, T. 2005. Sex differences in value priorities: Cross-cultural and multimethod studies. Journal of Personality and Social Psychology 89: 1010–28.
Steenkamp, J. B. E. M., and Baumgartner, H. 1998. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research 25: 78107.
Treier, S., and Jackman, S. 2008. Democracy as a latent variable. American Journal of Political Science 52: 201–17.
Vandenberg, R. J., and Lance, C. E. 2000. A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods 3: 470.
Whittaker, T. A. 2012. Using the modification index and standardized expected parameter change for model modification. Journal of Experimental Education 80: 2644.
Yoon, M., and Millsap, R. E. 2007. Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling 14: 435–63.
Yuan, K. H., Marshall, L. L., and Bentler, P. M. 2003. Assessing the effect of model misspecifications on parameter estimates in structural equation models. Sociological Methodology 33: 241–65.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
Type Description Title
Supplementary materials

Oberski supplementary material

 PDF (117 KB)
117 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed