Skip to main content

Do We Really Know the WTO Cures Cancer?

  • Stephen Chaudoin, Jude Hays and Raymond Hicks

This article uses a replication experiment of ninety-four specifications from sixteen different studies to show the severity of the problem of selection on unobservables. Using a variety of approaches, it shows that membership in the General Agreement on Tariffs and Trade/World Trade Organization has a significant effect on a surprisingly high number of dependent variables (34 per cent) that have little or no theoretical relationship to the WTO. To make the exercise even more conservative, the study demonstrates that membership in a low-impact environmental treaty, the Convention on Trade in Endangered Species, yields similarly high false positive rates. The authors advocate theoretically informed sensitivity analysis, showing how prior theoretical knowledge conditions the crucial choice of covariates for sensitivity tests. While the current study focuses on international institutions, the arguments also apply to other subfields and applications.

Hide All

Department of Political Science, University of Illinois at Urbana-Champaign (email:; Department of Political Science, University of Pittsburgh (email:; Department of Politics, Princeton University (email: We appreciate helpful advice from Marc Busch, Jake Bowers, William Clark, Mark Fredrickson, Kristian Skrede Gleditsch, Emilie Hafner-Burton, Sarah Hummel, In Song Kim, Moritz Marbach, Michael Miller, Dan Nielson, Dominik Schraff, Christopher Stanton and Dustin Tingley. We also appreciate comments from audiences at the International Political Economy Society, American Political Science Association, Political Economy of International Organizations and International Studies Association conferences, as well as the UCSD Workshop on International Law and Regulation and the Harvard Government Department 3005 Seminar participants. We appreciate the constructive comments from our editor at BJPS, Sona Golder, as well as from the anonymous reviewers. Data replication sets are available at, and online appendices are available at

Hide All
Allee Todd L., and Scalera Jamie E.. 2012. The Divergent Effects of Joining International Organizations: Trade Gains and the Rigors of WTO Accession. International Organization 66 (2):243276.
Altonji Joseph G., Elder Todd E., and Taber Christopher R.. 2002. Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools. Evanston, IL: Northwestern University.
Altonji Joseph G., Elder Todd E., and Taber Christopher R.. 2005. Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools. Journal of Political Economy 113 (1):151184.
Angrist Joshua D., and Krueger Alan B.. 1999. Empirical Strategies in Labor Economics. Handbook of Labor Economics 3:12771366.
Bertrand Marianne, Duflo Esther, and Mullainathan Sendhil. 2004. How Much Should We Trust Difference-in-Differences Estimates? Quarterly Journal of Economics 119 (1):249275.
Blackwell Matthew. 2014. A Selection Bias Approach to Sensitivity Analysis for Causal Effects. Political Analysis 22 (2):169182.
Clarke Kevin A. 2005. The Phantom Menace: Omitted Variable Bias in Econometric Research. Conflict Management and Peace Science 22 (4):341352.
Clarke Kevin A. 2009. Return of the Phantom Menace: Omitted Variable Bias in Political Research. Conflict Management and Peace Science 26 (1):4666.
Davis Christina, and Wilf Meredith. 2011. Joining the Club: Accession to the GATT/WTO, Working Paper. Princeton, NJ: Princeton University.
Downs George W., Rocke David M., and Barsoom Peter N.. 1996. Is the Good News About Compliance Good News About Cooperation? International Organization 50 (3):379406.
Gerring John, Thacker Strom C., and Moreno Carola. 2005. Centripetal Democratic Governance: A Theory and Global Inquiry. The American Political Science Review 99 (4):567581.
Ho Daniel E., Imai Kosuke, King Gary, and Stuart Elizabeth A.. 2007. Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis 15 (3):199236.
Imai Kosuke, Keele Luke, Tingley Dustin, and Yamamoto Teppei. 2011. Unpacking the Black Box of Causality: Learning About Causal Mechanisms From Experimental and Observational Studies. American Political Science Review 105:765789.
Imai Kosuke, Keele Luke, and Yamamoto Teppei. 2010. Identification, Inference and Sensitivity Analysis for Causal Mediation Effects. Statistical Science 25 (1):5171.
Imbens Guido W. 2003. Sensitivity to Exogeneity Assumptions in Program Evaluation. The American Economic Review 93 (2):126132.
Keele Luke. 2015. The Statistics of Causal Inference: A View From Political Methodology. Political Analysis 23 (3):313335.
Leuven Edwin, and Siansei Barbara. 2003. Psmatch2: Stata Module to Perform Full Mahalanobis and Propensity Score Matching, Common Support Graphing, and Covariate Imbalance Testing (Version 4.0.11).
Lupu Yonatan. 2013. The Informative Power of Treaty Commitment: Using the Spatial Model to Address Selection Effects. American Journal of Political Science 57 (4):912925.
Manski Charles F. 1990. Nonparametric Bounds on Treatment Effects. The American Economic Review 80 (2):319323.
Mebane Walter, and Poast Paul. 2013. Causal Inference Without Ignorability: Identification With Nonrandom Assignment and Missing Treatment Data. Political Analysis 21 (2):233251.
Miller Michael. 2015. The Uses and Abuses of Matching in Political Science, Working Paper. Washington, DC: George Washington University.
Oster Emily. 2014. Unobservable Selection and Coefficient Stability: Theory and Evidence, Working Paper. Chicago: University of Chicago Booth School of Business.
Pearl Judea. 2000. Causality: Models, Reasoning, and Inference. New York and Cambridge: Cambridge University Press.
Pelc Krzysztof J. 2011. Why do Some Countries get Better WTO Accession Terms Than Others? International Organization 65 (4):639672.
Plumper Thomas, and Troeger Vera E.. 2013. Not so Harmless After All: Fixed Effects as Identification Strategy. EPSA Conference Paper.
Rosenbaum Paul R. 2002. Observational Studies. New York: Springer.
Rosenbaum Paul R., and Rubin Donald B.. 1983. The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika 70 (1):4155.
Sekhon Jasjeet S. 2009. Opiates for the Matches: Matching Methods for Causal Inference. Annual Review of Political Science 12 (1):487508.
Simmons Beth A. 2000. International Law and State Behavior: Commitment and Compliance in International Monetary Affairs. American Political Science Review 94 (4):819835.
Simmons Beth A., and Hopkins Daniel J.. 2005. The Constraining Power of International Treaties: Theory and Methods. American Political Science Review 99 (4):623631.
Spirtes Peter, Glymour Clark, and Scheines Richard. 1993. Causation, Prediction, and Search. New York: Springer-Verlag.
Von Stein Jana. 2005. Do Treaties Constrain or Screen? Selection Bias and Treaty Compliance. American Political Science Review 99 (4):611622.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

British Journal of Political Science
  • ISSN: 0007-1234
  • EISSN: 1469-2112
  • URL: /core/journals/british-journal-of-political-science
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Type Description Title
Supplementary materials

Chaudoin supplementary material
Chaudoin supplementary material 1

 PDF (179 KB)
179 KB


Altmetric attention score

Full text views

Total number of HTML views: 17
Total number of PDF views: 233 *
Loading metrics...

Abstract views

Total abstract views: 1169 *
Loading metrics...

* Views captured on Cambridge Core between 14th November 2016 - 25th February 2018. This data will be updated every 24 hours.