Hostname: page-component-77f85d65b8-6c7dr Total loading time: 0 Render date: 2026-03-28T13:07:20.077Z Has data issue: false hasContentIssue false
  • English
  • Français

Measurement validity and the integrative approach

Published online by Cambridge University Press:  05 February 2024

Wendy C. Higgins Open the ORCID record for Wendy C. Higgins [Opens in a new window] ,
Alexander J. Gillett Open the ORCID record for Alexander J. Gillett [Opens in a new window] ,
Eliane Deschrijver Open the ORCID record for Eliane Deschrijver [Opens in a new window]  and
Robert M. Ross Open the ORCID record for Robert M. Ross [Opens in a new window]
Wendy C. Higgins*
Affiliation:
School of Psychological Sciences, Macquarie University, Sydney, NSW, Australia wendy.higgins@mq.edu.au, https://researchers.mq.edu.au/en/persons/wendy-higgins eliane.deschrijver@mq.edu.au, https://researchers.mq.edu.au/en/persons/eliane-deschrijver
Alexander J. Gillett
Affiliation:
Department of Philosophy, Macquarie University, Sydney, NSW, Australia alexander.gillett@mq.edu.au robross46@gmail.com, https://researchers.mq.edu.au/en/persons/robert-ross
Eliane Deschrijver
Affiliation:
School of Psychological Sciences, Macquarie University, Sydney, NSW, Australia wendy.higgins@mq.edu.au, https://researchers.mq.edu.au/en/persons/wendy-higgins eliane.deschrijver@mq.edu.au, https://researchers.mq.edu.au/en/persons/eliane-deschrijver
Robert M. Ross
Affiliation:
Department of Philosophy, Macquarie University, Sydney, NSW, Australia alexander.gillett@mq.edu.au robross46@gmail.com, https://researchers.mq.edu.au/en/persons/robert-ross
*
*Corresponding author.
Get access Rights & Permissions [Opens in a new window]

Abstract

Almaatouq et al. propose a novel integrative approach to experiments. We provide three examples of how unaddressed measurement issues threaten the feasibility of the approach and its promise of promoting commensurability and knowledge integration.

Information

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 47 , 2024 , e46
Check for updates
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Alexandrova, A., & Haybron, D. M. (2016). Is construct validation valid? Philosophy of Science, 83(5), 1098–1109. https://doi.org/10.1086/687941CrossRefGoogle Scholar
Almaatouq, A., Alsobay, M., Yin, M., & Watts, D. J. (2021). Task complexity moderates group synergy. Proceedings of the National Academy of Sciences of the United States of America, 118(36), Article e2101062118. https://doi.org/10.1073/pnas.2101062118CrossRefGoogle ScholarPubMed
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., … Rahwan, I. (2018). The moral machine experiment. Nature, 563(7729), 59–64. https://doi.org/10.1038/s41586-018-0637-6CrossRefGoogle ScholarPubMed
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., … Rahwan, I. (2020). Reply to: Life and death decisions of autonomous vehicles. Nature, 579(7797), E3–E5. https://doi.org/10.1038/s41586-020-1988-3CrossRefGoogle ScholarPubMed
Bauman, C. W., McGraw, A. P., Bartels, D. M., & Warren, C. (2014). Revisiting external validity: Concerns about trolley problems and other sacrificial dilemmas in moral psychology. Social and Personality Psychology Compass, 8(9), 536–554. https://doi.org/10.1111/spc3.12131CrossRefGoogle Scholar
Bigman, Y. E., & Gray, K. (2020). Life and death decisions of autonomous vehicles. Nature, 579(7797), E1–E2. https://doi.org/10.1038/s41586-020-1987-4CrossRefGoogle ScholarPubMed
Bostyn, D. H., Sevenhant, S., & Roets, A. (2018). Of mice, men, and trolleys: Hypothetical judgment versus real-life behavior in trolley-style moral dilemmas. Psychological Science, 29(7), 1084–1093. https://doi.org/10.1177/0956797617752640CrossRefGoogle ScholarPubMed
Engel, D., Woolley, A. W., Jing, L. X., Chabris, C. F., & Malone, T. W. (2014). Reading the mind in the eyes or reading between the lines? Theory of mind predicts collective intelligence equally well online and face-to-face. PLoS ONE, 9(12), e115212. https://doi.org/10.1371/journal.pone.0115212CrossRefGoogle ScholarPubMed
Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393CrossRefGoogle Scholar
Fried, E. I., Flake, J. K., & Robinaugh, D. J. (2022). Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology, 1(6), 358–368. https://doi.org/10.1038/s44159-022-00050-2CrossRefGoogle ScholarPubMed
Herrman, H., Kieling, C., McGorry, P., Horton, R., Sargent, J., & Patel, V. (2019). Reducing the global burden of depression: A Lancet–World Psychiatric Association Commission. The Lancet, 393(10189), e42–e43. https://doi.org/10.1016/S0140-6736(18)32408-5CrossRefGoogle ScholarPubMed
Hester, N., Axt, J. R., Siemers, N., & Hehman, E. (2023). Evaluating validity properties of 25 race-related scales. Behavior Research Methods, 55(4), 1758–1777. https://doi.org/10.3758/s13428-022-01873-wCrossRefGoogle ScholarPubMed
Higgins, W. C., Ross, R. M., Langdon, R., & Polito, V. (2023). The “reading the mind in the eyes” test shows poor psychometric properties in a large, demographically representative U.S. sample. Assessment, 30(6), 1777–1789. https://doi.org/10.1177/10731911221124342CrossRefGoogle Scholar
Higgins, W. C., Ross, R. M., Polito, V., & Kaplan, D. M. (2023). Three threats to the validity of the reading the mind in the eyes test: A commentary on Pavolova and Sokolov (2022). Neuroscience and Biobehavioral Reviews, 147, 105088. https://doi.org/10.1016/j.neubiorev.2023.105088CrossRefGoogle Scholar
Kim, Y. J., Engel, D., Woolley, A. W., Lin, J. Y.-T., McArthur, N., & Malone, T. W. (2017). What makes a strong team? Using collective intelligence to predict team performance in League of Legends. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing (pp. 2316–2329). Association for Computing Machinery, Portland, Oregon, USA. https://doi.org/10.1145/2998181.2998185CrossRefGoogle Scholar
Kittel, A. F. D., Olderbak, S., & Wilhelm, O. (2022). Sty in the mind's eye: A meta-analytic investigation of the nomological network and internal consistency of the “reading the mind in the eyes” test. Assessment, 29(5), 872–895. https://doi.org/10.1177/1073191121996469CrossRefGoogle Scholar
Lilienfeld, S. O., & Strother, A. N. (2020). Psychological measurement and the replication crisis: Four sacred cows. Canadian Psychology, 61(4), 281–288. https://doi.org/10.1037/cap0000236CrossRefGoogle Scholar
Oakley, B. F., Brewer, R., Bird, G., & Catmur, C. (2016). Theory of mind is not theory of emotion: A cautionary note on the reading the mind in the eyes test. Journal of Abnormal Psychology, 125(6), 818–823. https://doi.org/10.1037/abn0000182CrossRefGoogle Scholar
Olderbak, S., Wilhelm, O., Olaru, G., Geiger, M., Brenneman, M. W., & Roberts, R. D. (2015). A psychometric analysis of the reading the mind in the eyes test: Toward a brief form for research and applied settings. Frontiers in Psychology, 6, 1503. https://doi.org/10.3389/fpsyg.2015.01503CrossRefGoogle Scholar
Silverman, C. (2022). How to read “reading the mind in the eyes”. Notes and Records of the Royal Society of London, 76(4), 683–697. https://doi.org/10.1098/rsnr.2021.0058CrossRefGoogle Scholar
Stevanovic, D., Jafari, P., Knez, R., Franic, T., Atilola, O., Davidovic, N., … Lakic, A. (2017). Can we really use available scales for child and adolescent psychopathology across cultures? A systematic review of cross-cultural measurement invariance data. Transcultural Psychiatry, 54(1), 125–152. https://doi.org/10.1177/1363461516689215CrossRefGoogle ScholarPubMed
Vazire, S., Schiavone, S. R., & Bottesini, J. G. (2022). Credibility beyond replicability: Improving the four validities in psychological science. Current Directions in Psychological Science, 31(2), 162–168. https://doi.org/10.1177/09637214211067779CrossRefGoogle Scholar
Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science (New York, N.Y.), 330(6004), 686–688. https://doi.org/10.1126/science.1193147CrossRefGoogle ScholarPubMed