No CrossRef data available.
Article contents
Assume a can opener
Published online by Cambridge University Press: 05 February 2024
Abstract
We propose a friendly amendment to integrative experiment design (IED), adversarial-collaboration IED, that incentivizes research teams from competing theoretical perspectives to identify zones of the design space where they possess an explanatory edge. This amendment is especially critical in debates that have high policy stakes and carry a strong normative-political charge that might otherwise prevent free exchange of ideas.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2024. Published by Cambridge University Press
References
Abele, A. E., Ellemers, N., Fiske, S. T., Koch, A., & Yzerbyt, V. (2021). Navigating the social world: Toward an integrated framework for evaluating self, individuals, and groups. Psychological Review, 128(2), 290–314.CrossRefGoogle ScholarPubMed
Bowes, S., Clark, C. J., Conway, L. G. III, Costello, T. H., Osborne, D., Tetlock, P., & van Prooijen, J. (2023). An adversarial collaboration on the rigidity-of-the-right, rigidity-of-extremes, or symmetry: The answer depends on the question. https://doi.org/10.31234/osf.io/4wmx2CrossRefGoogle Scholar
Buss, D. M., & von Hippel, W. (2018). Psychological barriers to evolutionary psychology: Ideological bias and coalitional adaptations. Archives of Scientific Psychology, 6(1), 148–158.CrossRefGoogle Scholar
Clark, C. J., Costello, T., Mitchell, G., & Tetlock, P. E. (2022). Keep your enemies close: Adversarial collaborations will improve behavioral science. Journal of Applied Research in Memory and Cognition, 11(1), 1–18.CrossRefGoogle Scholar
Clark, C. J., Fjeldmark, M., Lu, L., Baumeister, R. F., Ceci, S., German, K., … Tetlock, P. E. (2023, February 24). Taboos and self-censorship among psychology professors (Conference presentation). Society for Open Inquiry in Behavioral Science, Behavioral Science Speakeasy, Atlanta, GA, USA.Google Scholar
Clark, C. J., & Tetlock, P. E. (2023). Adversarial collaboration: The next science reform. In Frisby, C. L., Redding, R. E., O’Donohue, W. T., & Lilienfeld, S. O. (Eds.), Ideological and political bias in psychology: Nature, scope, and solutions. Springer.Google Scholar
Clark, C. J., & Winegard, B. M. (2020). Tribalism in war and peace: The nature and evolution of ideological epistemology and its significance for modern social science. Psychological Inquiry, 31(1), 1–22.CrossRefGoogle Scholar
Eagly, A. H. (2016). When passionate advocates meet research on diversity, does the honest broker stand a chance?. Journal of Social Issues, 72(1), 199–222.CrossRefGoogle Scholar
Harden, K. P. (2021). The genetic lottery: Why DNA matters for social equality. Princeton University Press.Google Scholar
Kahneman, D. (2003). Experiences of collaborative research. American Psychologist, 58(9), 723–730.CrossRefGoogle ScholarPubMed
Killingsworth, M. A., Kahneman, D., & Mellers, B. (2023). Income and emotional well-being: A conflict resolved. Proceedings of the National Academy of Sciences of the United States of America, 120(10), e2208661120.CrossRefGoogle ScholarPubMed
Mellers, B., Hertwig, R., & Kahneman, D. (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science, 12(4), 269–275.CrossRefGoogle ScholarPubMed
Merton, R. K. (1942/1973). The sociology of science: Theoretical and empirical investigations. University of Chicago Press.Google Scholar
Nemeth, C., Brown, K., & Rogers, J. (2001). Devil's advocate versus authentic dissent: Stimulating quantity and quality. European Journal of Social Psychology, 31(6), 707–720.CrossRefGoogle Scholar
Tetlock, P. E. (1994). Political psychology or politicized psychology: Is the road to scientific hell paved with good moral intentions?. Political Psychology, 15, 509–529.CrossRefGoogle Scholar
Tetlock, P. E., Kristel, O. V., Elson, S. B., Green, M. C., & Lerner, J. S. (2000). The psychology of the unthinkable: Taboo trade-offs, forbidden base rates, and heretical counterfactuals. Journal of Personality and Social Psychology, 78(5), 853–870.CrossRefGoogle ScholarPubMed
Target article
Beyond playing 20 questions with nature: Integrative experiment design in the social and behavioral sciences
Related commentaries (31)
Against naïve induction from experimental data
Are language–cognition interactions bigger than a breadbox? Integrative modeling and design space thinking temper simplistic questions about causally dense phenomena
Assume a can opener
Beyond integrative experiment design: Systematic experimentation guided by causal discovery AI
Commensurability engineering is first and foremost a theoretical exercise
Confidence in research findings depends on theory
Consensus meetings will outperform integrative experiments
Dimensional versus conceptual incommensurability in the social and behavioral sciences
Discovering the unknown unknowns of research cartography with high-throughput natural description
Diversity of contributions is not efficient but is essential for science
Don't let perfect be the enemy of better: In defense of unparameterized megastudies
Eliminativist induction cannot be a solution to psychology's crisis
Experiment commensurability does not necessitate research consolidation
Explore your experimental designs and theories before you exploit them!
Getting lost in an infinite design space is no solution
Individual differences do matter
Integrative design for thought-experiments
Integrative experiments require a shared theoretical and methodological basis
Is generalization decay a fundamental law of psychology?
Measurement validity and the integrative approach
Neuroadaptive Bayesian optimisation can allow integrative design spaces at the individual level in the social and behavioural sciences… and beyond
Phenomena complexity, disciplinary consensus, and experimental versus correlational research in psychological science
Representative design: A realistic alternative to (systematic) integrative design
Sampling complex social and behavioral phenomena
Some problems with zooming out as scientific reform
Test many theories in many ways
The elephant's other legs: What some sciences actually do
The future of experimental design: Integrative, but is the sample diverse enough?
The miss of the framework
The social sciences needs more than integrative experimental designs: We need better theories
There are no shortcuts to theory
Author response
Replies to commentaries on beyond playing 20 questions with nature