Skip to main content Accessibility help
×
Home
Hostname: page-component-66d7dfc8f5-4n6vw Total loading time: 0.412 Render date: 2023-02-08T08:39:34.174Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models

Published online by Cambridge University Press:  04 January 2017

Daniel L. Oberski*
Affiliation:
Department of Methodology and Statistics, Tilburg University, Room P 1105, PO Box 90153, 5000 LE Tilburg, The Netherlands e-mail: d.oberski@tilburguniversity.edu

Abstract

Latent variable models can only be compared across groups when these groups exhibit measurement equivalence or “invariance,” since otherwise substantive differences may be confounded with measurement differences. This article suggests examining directly whether measurement differences present could confound substantive analyses, by examining the expected parameter change (EPC)-interest. The EPC-interest approximates the change in parameters of interest that can be expected when freeing cross-group invariance restrictions. Monte Carlo simulations suggest that the EPC-interest approximates these changes well. Three empirical applications show that the EPC-interest can help avoid two undesirable situations: first, it can prevent unnecessarily concluding that groups are incomparable, and second, it alerts the user when comparisons of interest may still be invalidated even when the invariance model appears to fit the data. R code and data for the examples discussed in this article are provided in the electronic appendix (http://hdl.handle.net/1902.1/21816).

Type
Research Article
Copyright
Copyright © The Author 2013. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Armstrong, D. A. 2011. Stability and change in the Freedom House political rights and civil liberties measures. Journal of Peace Research 48: 653–62.CrossRefGoogle Scholar
Bartholomew, D. J., Knott, M., and Moustaki, I. 2011. Latent variable models and factor analysis: A unified approach. New York: John Wiley & Sons.CrossRefGoogle Scholar
Bentler, P. M., and Chou, C. P. 1992. Some new covariance structure model improvement statistics. Sociological Methods & Research 21: 259–82.CrossRefGoogle Scholar
Bollen, K. A. 1989. Structural equations with latent variables. New York: John Wiley & Sons.Google Scholar
Byrne, B. M., Shavelson, R. J., and Muthén, Bengt. 1989. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin 105: 456.CrossRefGoogle Scholar
Chen, F. F. 2007. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling 14: 464504.CrossRefGoogle Scholar
Cheung, G. W., and Rensvold, R. B. 2002. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling 9: 233–55.CrossRefGoogle Scholar
Chou, C. P., and Bentler, P. M. 1993. Invariant standardized estimated parameter change for model modification in covariance structure analysis. Multivariate Behavioral Research 28: 97110.CrossRefGoogle ScholarPubMed
Clinton, J., Jackman, S., and Rivers, D. 2004. The statistical analysis of roll call data. American Political Science Review 98: 355–70.CrossRefGoogle Scholar
Davidov, E. 2009. Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis 17: 6482.CrossRefGoogle Scholar
Davidov, E., Meuleman, B., Billiet, J., and Schmidt, P. 2008. Values and support for immigration: A cross-country comparison. European Sociological Review 24: 583–99.CrossRefGoogle Scholar
Fan, Xitao, Thompson, Bruce, and Wang, Lin. 1999. Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal 6: 5683.CrossRefGoogle Scholar
Fan, Weihua, and Hancock, Gregory R. 2006. Impact of post hoc measurement model overspecification on structural parameter integrity. Educational and Psychological Measurement 66: 748–64.CrossRefGoogle Scholar
French, B. F., and Finch, W. H. 2006. Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling 13: 378402.CrossRefGoogle Scholar
Hancock, Gregory R. 1999. A sequential Scheffé-type respecification procedure for controlling type I error in exploratory structural equation model modification. Structural Equation Modeling: A Multidisciplinary Journal 6: 158–68.CrossRefGoogle Scholar
Hancock, Gregory R., Stapleton, Laura M., and Arnold-Berkovits, Ilona. 2009. The tenuousness of invariance tests within multisample covariance and mean structure models. In structural equation modeling in educational research: Concepts and applications, eds. Teo, T. and Khine, M. S., 137–74. Rotterdam, The Netherlands: Sense Publishers.Google Scholar
Hausman, J. A. 1978. Specification tests in econometrics. Econometrica: Journal of the Econometric Society 46(6): 1251–71.CrossRefGoogle Scholar
Hu, L., and Bentler, P. M. 1998. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods 3: 424.CrossRefGoogle Scholar
Imai, K., and Yamamoto, T. 2010. Causal inference with differential measurement error: Nonparametric identification and sensitivity analysis. American Journal of Political Science 54: 543–60.CrossRefGoogle Scholar
Jackman, S. 2001. Multidimensional analysis of roll call data via Bayesian simulation: Identification, estimation, inference, and model checking. Political Analysis 9: 227–41.CrossRefGoogle Scholar
Jowell, Roger, Roberts, Caroline, Fitzgerald, Rory, and Eva, Gillian. 2007. Measuring attitudes cross-nationally: Lessons from the European Social Survey. Thousand Oaks, CA: SAGE.Google Scholar
Kaplan, D. 1989. Model modification in covariance structure analysis: Application of the expected parameter change statistic. Multivariate Behavioral Research 24: 285305.CrossRefGoogle ScholarPubMed
King, B. L. 2011. Unbiased measurement of health-related quality-of-life. Unpublished PhD dissertation, University of Amsterdam. http://dare.uva.nl/document/342053 (accessed June 1, 2013).Google Scholar
Kolenikov, S. 2009. Biases of parameter estimates in misspecified structural equation models. Sociological Methodology 41: 119–57.Google Scholar
Kwok, Oi-Man, Luo, Wen, and West, Stephen G. 2010. Using modification indexes to detect turning points in longitudinal data: A Monte Carlo study. Structural Equation Modeling 17: 216–40.CrossRefGoogle Scholar
Lord, F. M., and Novick, M. R. 1968. Statistical theories of mental scores. Reading, MA: Addison-Wesley.Google Scholar
MacCallum, R. C., Roznowski, M., and Necowitz, L. B. 1992. Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin; Psychological Bulletin 111: 490.CrossRefGoogle ScholarPubMed
Magnus, J. R., and Vasnev, A. L. 2007. Local sensitivity and diagnostic tests. Econometrics Journal 10: 166–92.CrossRefGoogle Scholar
Meredith, W. 1993. Measurement invariance, factor analysis and factorial invariance. Psychometrika 58: 525–43.CrossRefGoogle Scholar
Meuleman, B. 2012. When are item intercept differences substantively relevant in measurement invariance testing? In Methods, theories, and empirical applications in the social sciences: Festschrift for Peter Schmidt, eds. Salzborn, S., Davidov, E., and Reinecke, J., 97104. Wiesbaden, Germany: Springer-Verlag.CrossRefGoogle Scholar
Millsap, R. E. 1997. Invariance in measurement and prediction: Their relationship in the single-factor case. Psychological Methods 2: 248.CrossRefGoogle Scholar
Millsap, R. E. 2007. Invariance in measurement and prediction revisited. Psychometrika 72: 461–73.CrossRefGoogle Scholar
Millsap, R. E., and Everson, H. T. 1993. Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement 17: 297334.CrossRefGoogle Scholar
Millsap, R. E., and Kwok, O. M. 2004. Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods 9: 93.CrossRefGoogle ScholarPubMed
Neudecker, H., and Satorra, A. 1991. Linear structural relations: Gradient and Hessian of the fitting function. Statistics and Probability Letters 11: 5761.CrossRefGoogle Scholar
Oberski, D. L. 2012. Comparability of survey measurements. In Handbook of survey methodology for the social sciences, ed. Gideon, Lior, 477–98. New York: Springer-Verlag.Google Scholar
R Core Team. 2012. R: A language and environment for statistical computing . Vienna, Austria: R Foundation for Statistical Computing. http://www.R-project.org/ (accessed June 1, 2013).Google Scholar
Rosseel, Y. 2012. lavaan: An R package for structural equation modeling. Journal of Statistical Software 48: 136.CrossRefGoogle Scholar
Saris, W. E., Satorra, A., and Sörbom, D. 1987. The detection and correction of specification errors in structural equation models. Sociological Methodology 17: 105–29.CrossRefGoogle Scholar
Saris, W. E., Satorra, A., and Van der Veld, W. M. 2009. Testing structural equation models or detection of misspecifications? Structural Equation Modeling 16: 561–82.CrossRefGoogle Scholar
Satorra, A. 1989. Alternative test criteria in covariance structure analysis: A unified approach. Psychometrika 54: 131–51.CrossRefGoogle Scholar
Schmitt, N., and Kuljanin, G. 2008. Measurement invariance: Review of practice and implications. Human Resource Management Review 18: 210–22.CrossRefGoogle Scholar
Schwartz, S. H., and Bilsky, W. 1987. Toward a universal psychological structure of human values. Journal of Personality and Social Psychology 53: 550.CrossRefGoogle Scholar
Schwartz, S. H., and Rubel, T. 2005. Sex differences in value priorities: Cross-cultural and multimethod studies. Journal of Personality and Social Psychology 89: 1010–28.CrossRefGoogle ScholarPubMed
Steenkamp, J. B. E. M., and Baumgartner, H. 1998. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research 25: 78107.CrossRefGoogle Scholar
Treier, S., and Jackman, S. 2008. Democracy as a latent variable. American Journal of Political Science 52: 201–17.CrossRefGoogle Scholar
Vandenberg, R. J., and Lance, C. E. 2000. A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods 3: 470.CrossRefGoogle Scholar
Whittaker, T. A. 2012. Using the modification index and standardized expected parameter change for model modification. Journal of Experimental Education 80: 2644.CrossRefGoogle Scholar
Yoon, M., and Millsap, R. E. 2007. Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling 14: 435–63.CrossRefGoogle Scholar
Yuan, K. H., Marshall, L. L., and Bentler, P. M. 2003. Assessing the effect of model misspecifications on parameter estimates in structural equation models. Sociological Methodology 33: 241–65.CrossRefGoogle Scholar
Supplementary material: PDF

Oberski supplementary material

Appendix

Download Oberski supplementary material(PDF)
PDF 117 KB
50
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *