Skip to main content Accessibility help

The Commensurability Problem: Conceptual Difficulties in Estimating the Effect of Behavior on Behavior



We pose the commensurability problem: When do the estimates generated by actual research designs correspond to quantities of theoretical interest? We study this question in settings where both treatment and outcome are behavior and the treatment effect of interest is decomposable into direct and informational channels. We establish two results. First, the quantity estimated by an actual research design is only commensurate with the total effect in the ideal experiment if treatment status in the research design is a sufficient statistic for the decision-makers’ information. Second, a research design corresponding to a nonideal experiment isolates just the direct effect in the ideal experiment if two conditions hold: (i) there is no information effect in the nonideal experiment and (ii) the decision-maker’s response function is additively separable in treatment and information. We apply our results to three substantive literatures: the efficacy of protest, the empowerment of female candidates, and indiscriminate violence in counterinsurgency.


Corresponding author

*Ethan Bueno de Mesquita, Harris School, University of Chicago,
Scott A. Tyson, Department of Political Science, and W. Allen Wallis Institute of Political Economy, University of Rochester,


Hide All

We have benefited from comments by Scott Ashworth, Mark Fey, Anthony Fowler, Glyn Morgan, Cyrus Samii, and Stephane Wolton, and participants in the Princeton Institutions & Inference Conference and the Petralia Sottana Applied Economics Workshop.



Hide All
Angeletos, George-Marios, and Pavan, Alessandro. 2013. “Selection-Free Predictions in Global Games with Endogenous Information and Multiple Equilibria.” Theoretical Economics 8 (3): 883–938.
Angeletos, George-Marios, Hellwig, Christian, and Pavan, Alessandro. 2006. “Signaling in a Global Game: Coordination and Policy Traps.” Journal of Political Economy 114 (3): 452–84.
Angeletos, George-Marios, Hellwig, Christian, and Pavan, Alessandro. 2007. “Dynamic Global Games of Regime Change: Learning, Multiplicity, and the Timing of Attacks.” Econometrica 75 (3): 711–56.
Angrist, Joshua D., Imbens, Guido W., and Rubin, Donald B.. 1996. “Identification of Causal Effects Using Instrumental Variables.” Journal of the American Statistical Association 91 (434): 444–55.
Angrist, Joshua D., and Pischke, Jörn-Steffen. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton, NJ: Princeton University Press.
Ashworth, Scott, and Bueno de Mesquita, Ethan. 2014. “Is Voter Competence Good for Voters? Information, Rationality, and Democratic Performance.” American Political Science Review 108 (3): 565–87.
Ashworth, Scott, Bueno de Mesquita, Ethan, and Friedenberg, Amanda. 2018. “Learning about Voter Rationality.” American Journal of Political Science 32 (1): 37–54.
Banerjee, Abhijit V., Chassang, Sylvain, and Snowberg, Erik. 2017. “Decision Theoretic Approaches to Experiment Design and External Validity.” In Handbook of Economic Field Experiments. Vol. 1, eds. Abhijit Vinayak Banerjee and Esther Duflo. Elsevier, 141–74.
Baskaran, Thushyanthan, and Hessami, Zohal. 2018. “Does the Election of a Female Leader Clear the Way for More Women in Politics?American Economic Journal: Economic Policy 10 (3): 95–121.
Benmelech, Efraim, Berrebi, Claude, and Klor, Esteban F.. 2014. “Counter-Suicide-Terrorism: Evidence from House Demolitions.” The Journal of Politics 77 (1): 27–43.
Blackwell, David. 1951. “Comparison of Experiments.” In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, ed. J. Neyman. Berkeley, CA: University of California Press, 93102.
Broockman, David E. 2014. “Do Female Politicians Empower Women to Vote or Run for Office? A Regression Discontinuity Approach.” Electoral Studies 34: 190–204.
Bueno de Mesquita, Ethan. 2010. “Regime Change and Revolutionary Entrepreneurs.” American Political Science Review 104 (3): 446–65.
Casper, Brett Allen, and Tyson, Scott A.. 2014. “Popular Protest and Elite Coordination in a Coup D’etat.” The Journal of Politics 76 (2): 548–64.
Chassang, Sylvain, Padró i Miquel, Gerard, and Snowberg, Erik. 2012. “Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments.” The American Economic Review 102 (4): 1279–309.
Collins, William J., and Margo, Robert A.. 2007. “The Economic Aftermath of the 1960s Riots in American Cities: Evidence from Property Values.” The Journal of Economic History 67 (4): 849.
Condra, Luke N., and Shapiro, Jacob N.. 2012. “Who Takes the Blame? The Strategic Effects of Collateral Damage.” American Journal of Political Science 56 (1): 167–87.
Conlisk, John. 1973. “Choice of Response Functional Form in Designing Subsidy Experiments.” Econometrica 41 (4): 643.
Deaton, Angus. 2010. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.
Dell, Melissa. 2012. “Path Dependence in Development: Evidence from the Mexican Revolution.” Harvard typescript.
Dell, Melissa, and Querubín, Pablo. 2017. “Nation Building through Foreign Intervention: Evidence from Discontinuities in Military Strategies.” Quarterly Journal of Economics 133 (2): 701–64.
Edmond, Chris. 2013. “Information Manipulation, Coordination, and Regime Change.” The Review of Economic Studies 80 (4): 1422–58.
Eggers, Andrew C. 2017. “Quality-Based Explanations of Incumbency Effects.” The Journal of Politics 79 (4): 1315–28.
Feyerabend, Paul K. 1962. “Explanation, Reduction and Empiricism.” In Scientific Explanation, Space, and Time (Minnesota Studies in the Philosophy of Science, Volume III), eds. Feigl, Herbert and Maxwell, Grover. Minneapolis: University of Minneapolis Press, 28–97.
Fowler, Anthony. 2018. “A Bayesian Explanation for the Effect of Incumbency.” Electoral Studies 53: 66–78.
Fudenberg, Drew, and Levine, David K.. 2019. “Learning in Games and the Interpretation of Natural Experiments”. MIT typescript.
Goldberger, Arthur S. 1972. “Structural Equation Methods in the Social Sciences.” Econometrica: Journal of the Econometric Society 40 (6): 979–1001.
Gould, Eric D., and Klor, Esteban. 2010. “Does Terrorism Work?Quarterly Journal of Economics 125 (4): 1459–510.
Grimmer, Justin, Messing, Solomon, and Westwood, Sean J.. 2012. “How Words and Money Cultivate a Personal Vote: The Effect of Legislator Credit Claiming on Constituent Credit Allocation.” American Political Science Review 106 (4): 703–19.
Haavelmo, Trygve. 1944. “The Probability Approach in Econometrics.” Econometrica: Journal of the Econometric Society 12: iii–115.
Heckman, James J. 2000. “Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective.” Quarterly Journal of Economics 115 (1): 45–97.
Heckman, James J., and Urzua, Sergio. 2010. “Comparing IV with Structural Models: What Simple IV Can and Cannot Identify.” Journal of Econometrics 156 (1): 27–37.
Henderson, John, and Brooks, John. 2016. “Mediating the Electoral Connection: The Information Effects of Voter Signals on Legislative Behavior.” The Journal of Politics 78 (3): 653–69.
Hendrix, Cullen S., and Salehyan, Idean. 2012. “Climate Change, Rainfall, and Social Conflict in Africa.” Journal of Peace Research 49 (1): 35–50.
Huet-Vaughn, Emiliano. 2013. “Quiet Riot: The Causal Effect of Protest Violence.” UC Berkeley typescript.
Imbens, Guido W., and Angrist, Joshua D.. 1994. “Identification and Estimation of Local Average Treatment Effects.” Econometrica 62 (2): 467–75.
Izzo, Federical, Dewan, Torun, and Wolton, Stephane. 2018. “Cumulative Knowledge in the Social Sciences: The Case of Improving Voters? Information.” Available at SSRN
Jaeger, David A., Klor, Esteban F., Miaari, Sami H., and Paserman, M.Daniele. 2012. “The Struggle for Palestinian Hearts and Minds: Violence and Public Opinion in the Second Intifada.” Journal of Public Economics 96 (3–4): 354–68.
Koopmans, Tjalling C., and Reiersol, Olav. 1950. “The Identification of Structural Characteristics.” The Annals of Mathematical Statistics 21 (2): 165–81.
Kreps, David M. 2012. Microeconomic Foundations I: Choice and Competitive Markets. Princeton, NJ: Princeton University Press.
Kuhn, Thomas S. 1962. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.
Ladam, Christina, Harden, Jeffrey J., and Windett, Jason H.. 2018. “Prominent Role Models: High-Profile Female Politicians and the Emergence of Women as Candidates for Public Office.” American Journal of Political Science 62 (2): 369–81.
Lyall, Jason. 2009. “Does Indiscriminate Violence Incite Insurgent Attacks? Evidence from Chechnya.” Journal of Conflict Resolution 53 (3): 331–62.
Madestam, Andreas, Shoag, Daniel, Veuger, Stan, and Yanagizawa-Drott, David. 2013. “Do Political Protests Matter? Evidence from the Tea Party Movement.” Harvard University typescript.
Mas-Collel, Andreu, Whiston, Michael D., and Green, Jerry R.. 1995. Microeconomic Theory. Oxford: Oxford University Press.
Milnor, John Willard. 1997. Topology from the Differentiable Viewpoint. Princeton, NJ: Princeton University Press.
Prato, Carlo, and Wolton, Stephane. 2018. “Electoral Imbalances and Their Consequences.” The Journal of Politics 80 (4): 1168–82.
Ritter, Emily Hencken, and Conrad, Cortenay R.. 2016. “Preventing and Responding to Dissent: The Observational Challenges of Explaining Strategic Repression.” American Political Science Review 110 (1): 85–99.
Sarsons, Heather. 2015. “Rainfall and Conflict: A Cautionary Tale.” Journal of Eevelopment Economics 115: 62–72.
Shadmehr, Mehdi, and Bernhardt, Dan. 2011. “Collective Action with Uncertain Payoffs: Coordination, Public Signals, and Punishment Dilemmas.” American Political Science Review 105: 829–51.
Shapiro, Jacob N., and Weidmann, Nils B.. 2015. “Is the Phone Mightier Than the Sword? Cellphones and Insurgent Violence in Iraq.” International Organization 69 (2): 247–74.
Sun, Jessica, and Tyson, Scott A.. 2019. “Theoretical Implications of Empirical Models: An Application to Conflict Studies.” Available at SSRN 3258725.
Tyson, Scott A., and Smith, Alastair. 2018. “Dual-Layered Coordination and Political Instability: Repression, Co-optation, and the Role of Information.” The Journal of Politics 80 (1): 44–58.
Wolton, Stephane. 2019. “Are Biased Media Bad for Democracy?American Journal of Political Science 63 (3): 548–562.

The Commensurability Problem: Conceptual Difficulties in Estimating the Effect of Behavior on Behavior



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed