Skip to main content Accessibility help
×
Home

Analyzing Causal Mechanisms in Survey Experiments

  • Avidit Acharya (a1), Matthew Blackwell (a2) and Maya Sen (a3)

Abstract

Researchers investigating causal mechanisms in survey experiments often rely on nonrandomized quantities to isolate the indirect effect of treatment through these variables. Such an approach, however, requires a “selection-on-observables” assumption, which undermines the advantages of a randomized experiment. In this paper, we show what can be learned about casual mechanisms through experimental design alone. We propose a factorial design that provides or withholds information on mediating variables and allows for the identification of the overall average treatment effect and the controlled direct effect of treatment fixing a potential mediator. While this design cannot identify indirect effects on its own, it avoids making the selection-on-observable assumption of the standard mediation approach while providing evidence for a broader understanding of causal mechanisms that encompasses both indirect effects and interactions. We illustrate these approaches via two examples: one on evaluations of US Supreme Court nominees and the other on perceptions of the democratic peace.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Analyzing Causal Mechanisms in Survey Experiments
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Analyzing Causal Mechanisms in Survey Experiments
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Analyzing Causal Mechanisms in Survey Experiments
      Available formats
      ×

Copyright

Corresponding author

Footnotes

Hide All

Contributing Editor: R. Michael Alvarez

Many thanks to John Ahlquist, Josh Kertzer, Ryan T. Moore, Paul Testa, and Teppei Yamamoto for helpful feedback. Special thanks to Jessica Weeks and Mike Tomz for sharing their survey instrument with us. Replication data and code can be found in Acharya, Blackwell, and Sen (2018).

Footnotes

References

Hide All
Acharya, Avidit, Blackwell, Matthew, and Sen, Maya. 2016. Explaining causal findings without bias: detecting and assessing direct effects. American Political Science Review 110(3):512529.
Acharya, Avidit, Blackwell, Matthew, and Sen, Maya. 2018. Replication data for: analyzing causal mechanisms in survey experiments. doi:10.7910/DVN/KHE44F, Harvard Dataverse, V1, UNF:6:VaeMnsesFmBVwg98eK/heA==.
Alesina, Alberto, Giuliano, Paola, and Nunn, Nathan. 2013. On the origins of gender roles: women and the plough. Quarterly Journal of Economics 128(2):469530.
Bansak, Kirk, Hainmueller, Jens, Hopkins, Daniel J., and Yamamoto, Teppei. 2017. Beyond the breaking point? Survey satisficing in conjoint experiments. Stanford University Graduate School of Business Research Paper No. 17-33; MIT Political Science Department Research Paper No. 2017-16.
Cochran, William G., and Cox, Gertrude M.. 1957. Experimental designs . New York: John Wiley & Sons.
Dafoe, Allan, Zhang, Baobao, and Caughey, Devin. 2017. Information equivalence in survey experiments. Political Analysis , forthcoming.
Gerber, Alan S., and Green, Donald P.. 2012. Field experiments: design, analysis, and interpretation . New York: W.W. Norton.
Hainmueller, Jens, Hopkins, Daniel J., and Yamamoto, Teppei. 2013. Causal inference in conjoint analysis: understanding multidimensional choices via stated preference experiments. Political Analysis 22(1):130.
Holland, Paul W. 1986. Statistics and causal inference. Journal of the American Statistical Association 81(396):945960.
Imai, Kosuke, Tingley, Dustin, and Yamamoto, Teppei. 2013. Experimental designs for identifying causal mechanisms. Journal of the Royal Statistical Society: Series A 176(1):551.
Imai, Kosuke, Keele, Luke, Tingley, Dustin, and Yamamoto, Teppei. 2011. Unpacking the black box of causality: learning about causal mechanisms from experimental and observational studies. American Political Science Review 105(4):765789.
Imai, Kosuke, Keele, Luke, and Yamamoto, Teppei. 2010. Identification, inference and sensitivity analysis for causal mediation effects. Statistical Science 25(1):5171.
Kirkland, Patricia, and Coppock, Alexander. 2017. Candidate choice without party labels: new insights from conjoint survey experiments. Political Behavior . Epub ahead of print, doi:10.1007/s11109-017-9414-8.
McDermott, Monika L. 1998. Race and gender cues in low-information elections. Political Research Quarterly 51(4):895918.
Neyman, Jerzy. 1923. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science 5:465480, translated in 1990 with discussion.
Petersen, Maya L., Sinisi, Sandra E., and van der Laan, Mark J.. 2006. Estimation of direct causal effects. Epidemiology 17(3):276284.
Robins, James M. 2003. Semantics of causal DAG models and the identification of direct and indirect effects. In Highly structured stochastic systems , ed. Green, P. J., Hjort, N. L., and Richardson, S.. Oxford: Oxford University Press, pp. 7081.
Robins, James M., and Greenland, Sander. 1992. Identifiability and exchangeability for direct and indirect effects. Epidemiology 3(2):143155.
Rosenbaum, Paul R. 1984. The consquences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society. Series A (General) 147(5):656666.
Rubin, Donald B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 6(5):688701.
Sen, Maya. 2017. How political signals affect public support for judicial nominations: evidence from a conjoint experiment. Political Research Quarterly 70(2):374393.
Tomz, Michael R., and Weeks, Jessica L. P.. 2013. Public opinion and the democratic peace. American Political Science Review 107(04):849865.
VanderWeele, Tyler J. 2014. A unification of mediation and interaction: a 4-way decomposition. Epidemiology 25(5):749761.
VanderWeele, Tyler J. 2015. Explanation in causal inference: methods for mediation and interaction . New York: Oxford University Press.
VanderWeele, Tyler J., and Tchetgen Tchetgen, Eric J.. 2014. Attributing effects to interactions. Epidemiology 25(5):711722.
MathJax
MathJax is a JavaScript display engine for mathematics. For more information see http://www.mathjax.org.

Keywords

Related content

Powered by UNSILO
Type Description Title
UNKNOWN
Supplementary materials

Acharya et al. supplementary material
Acharya et al. supplementary material 1

 Unknown (90 KB)
90 KB

Analyzing Causal Mechanisms in Survey Experiments

  • Avidit Acharya (a1), Matthew Blackwell (a2) and Maya Sen (a3)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.