Skip to main content Accessibility help
×
×
Home

Comparative Research is Harder Than We Thought: Regional Differences in Experts’ Understanding of Electoral Integrity Questions

  • Bruno Castanho Silva (a1) and Levente Littvay (a2)

Abstract

Expert evaluations about countries form the backbone of comparative political research. It is reasonable to assume that such respondents, no matter the region they specialize in, will have a comparable understanding of the phenomena tapped by expert surveys. This is necessary to get results that can be compared across countries, which is the fundamental goal of these measurement activities. We empirically test this assumption using measurement invariance techniques which have not been applied to expert surveys before. Used most often to test the cross-cultural validity and translation effects of public opinion scales, the measurement invariance tests evaluate the comparability of scale items across any groups. We apply them to the Perceptions of Electoral Integrity (PEI) dataset. Our findings suggest that cross-regional comparability fails for all eleven dimensions identified in PEI. Results indicate which items remain comparable, at least across most regions, and point to the need of more rigorous procedures to develop expert survey questions.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Comparative Research is Harder Than We Thought: Regional Differences in Experts’ Understanding of Electoral Integrity Questions
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Comparative Research is Harder Than We Thought: Regional Differences in Experts’ Understanding of Electoral Integrity Questions
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Comparative Research is Harder Than We Thought: Regional Differences in Experts’ Understanding of Electoral Integrity Questions
      Available formats
      ×

Copyright

Corresponding author

Footnotes

Hide All

Authors’ note: We owe a debt of gratitude to the PEI team, especially Thomas Wynter and Kristina Gushchina who put us in contact. Wynter and his team’s responsiveness to our inquiries were exemplary, he explicitly encouraged us to research language effects and worked with us to get what we needed. We are also thankful for comments on earlier versions of this manuscript by the three anonymous reviewers and Jeff Gill. The replication files for this article are available on the Political Analysis Dataverse (Castanho Silva and Littvay 2019).

Contributing Editor: Jeff Gill

Footnotes

References

Hide All
Asparouhov, T., and Muthén, B.. 2014. “Multiple-group Factor Analysis Alignment.” Structural Equation Modeling: A Multidisciplinary Journal 21(4):495508.
Bakker, R., Edwards, E., Jolly, S., Polk, J., Rovny, J., and Steenbergen, M.. 2014. “Anchoring the Experts: Using Vignettes to Compare Party Ideology Across Countries.” Research & Politics 1(3):18.
Castanho Silva, B., Bosancianu, C. M., and Littvay, L.. 2019. Multilevel Structural Equation Modeling . Thousand Oaks, CA: Sage.
Castanho Silva, B., and Levente, L.. 2019. “Replication Data for: Comparative Research is Harder than We Thought: Regional Differences in Experts’ Understanding of Electoral Integrity Questions.” https://doi.org/10.7910/DVN/HG2CAJ, Harvard Dataverse, V1.
Davidov, E., Muthén, B., and Schmidt, P.. 2018. “Measurement Invariance in Cross-National Studies: Challenging Traditional Approaches and Evaluating New Ones.” Sociological Methods & Research 47(4):631636.
von Davier, M., Shin, H.-J., Khorramdel, L., and Stankov, L.. 2017. “The Effects of Vignette Scoring on Reliability and Validity of Self-Reports.” Applied Psychological Measurement 42(4):291306.
Hu, L.-t., and Bentler, P. M.. 1999. “Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus New Alternatives.” Structural Equation Modeling: A Multidisciplinary Journal 6:155.
Jöreskog, K. G. 1971. “Simultaneous Factor Analysis in Several Populations.” Psychometrika 36:409426.
King, G., Murray, C. J. L., Salomon, J. A., and Tandon, A.. 2004. “Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research.” American Political Science Review 98(1):191207.
Kline, R. B. 2016. Principles and Practice of Structural Equation Modeling . 4th edn. New York: The Guilford Press.
Lindstädt, R., Proksch, S.-O., and Slapin, J. B.. 2018. “When Experts Disagree: Response Aggregation and its Consequences in Expert Surveys.” Political Science Research and Methods, https://doi.org/10.1017/psrm.2018.52.
Maestas, C. D., Buttice, M. K., and Stone, W. J.. 2014. “Extracting Wisdom from Experts and Small Crowds: Strategies for Improving Informant-based Measures of Political Concepts.” Political Analysis 22(3):354373.
Marquardt, K. L., and Pemstein, D.. 2018. “IRT Models for Expert-Coded Panel Data.” Political Analysis 26(4):431456.
Millsap, R. E. 2011. Statistical Approaches to Measurement Invariance . New York: Routledge.
Muthén, B. O., and Christoffersson, A.. 1981. “Simultaneous Factor Analysis of Dichotomous Variables in Several Groups.” Psychometrika 46(4):407419.
Muthén, B. O., and Asparouhov, T.. 2018. “Recent Methods for the Study of Measurement Invariance With Many Groups: Alignment and Random Effects.” Sociological Methods & Research 47(4):637664.
Norris, P., Wynter, T., and Grömping, M.. 2017. Codebook – The Expert Survey of Perceptions of Electoral Integrity, Release 5.5, (PEI_5.5). The University of Sidney: The Electoral Integrity Project.
Oberski, D. L. 2014. “Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models.” Political Analysis 22(1):4560.
Raju, N. S., Laffitte, L. J., and Byrne, B. M.. 2002. “Measurement Equivalence: A Comparison of Methods Based on Confirmatory Factor Analysis and Item Response Theory.” Journal of Applied Psychology 87(3):517529.
Satorra, A., and Bentler, P. M.. 2001. “A Scaled Difference chi-Square Test Statistic for Moment Structure Analysis.” Psychometrika 66(4):507514.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax

Keywords

Type Description Title
UNKNOWN
Supplementary materials

Castanho Silva and Littvay supplementary material
Castanho Silva and Littvay supplementary material 1

 Unknown (162 KB)
162 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed