Hostname: page-component-89b8bd64d-shngb Total loading time: 0 Render date: 2026-05-10T18:18:14.742Z Has data issue: false hasContentIssue false
  • English
  • Français

Spillover Effects in the Presence of Unobserved Networks

Published online by Cambridge University Press:  18 November 2020

Naoki Egami Open the ORCID record for Naoki Egami [Opens in a new window]
Naoki Egami*
Affiliation:
Department of Political Science, Columbia University, New York, NY 10027, USA. Email: naoki.egami@columbia.edu, URL: https://naokiegami.com
*
Corresponding author Naoki Egami
Get access Rights & Permissions [Opens in a new window]

Abstract

When experimental subjects can interact with each other, the outcome of one individual may be affected by the treatment status of others. In many social science experiments, such spillover effects may occur through multiple networks, for example, through both online and offline face-to-face networks in a Twitter experiment. Thus, to understand how people use different networks, it is essential to estimate the spillover effect in each specific network separately. However, the unbiased estimation of these network-specific spillover effects requires an often-violated assumption that researchers observe all relevant networks. We show that, unlike conventional omitted variable bias, bias due to unobserved networks remains even when treatment assignment is randomized and when unobserved networks and a network of interest are independently generated. We then develop parametric and nonparametric sensitivity analysis methods, with which researchers can assess the potential influence of unobserved networks on causal findings. We illustrate the proposed methods with a simulation study based on a real-world Twitter network and an empirical application based on a network field experiment in China.

Keywords

causal inferenceinterferencepotential outcomesSUTVAnetwork experiments

Information

Type
Article
Information
Political Analysis , Volume 29 , Issue 3 , July 2021 , pp. 287 - 316
Copyright
© The Author(s) 2020. Published by Cambridge University Press on behalf of the Society for Political Methodology

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Edited by Betsy Sinclair

References

Aral, S. (2016). “Networked Experiments.” In The Oxford Handbook of the Economics of Networks. Oxford, UK: Oxford University Press.Google Scholar
Aronow, P. M. (2012). “A General Method for Detecting Interference Between Units in Randomized Experiments.” Sociological Methods & Research 41(1):3–16.CrossRefGoogle Scholar
Aronow, P. M., and Samii, C.. (2017). “Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment.” Annals of Applied Statistics 11(4):1912–1947.CrossRefGoogle Scholar
Aronow, P. M., Crawford, F. W., and Zubizarreta, J. R.. (2016). “Confidence Intervals for Means Under Constrained Dependence.” Preprint, arXiv:1602.00359.Google Scholar
Asunka, J., Brierley, S., Golden, M., Kramon, E., and Ofosu, G.. (2019). “Electoral Fraud or Violence: The Effect of Observers on Party Manipulation Strategies.” British Journal of Political Science 49(1):129–151.CrossRefGoogle Scholar
Athey, S., Eckles, D., and Imbens, G. W.. (2018). “Exact P-values for Network Interference.” Journal of the American Statistical Association 113(521):230–240.CrossRefGoogle Scholar
Basse, G., Feller, A., and Toulis, P.. (2019). “Randomization Tests of Causal Effects Under Interference.” Biometrika 106(2):487–494.CrossRefGoogle Scholar
Basse, G. W., and Airoldi, E. M.. (2018). “Model-Assisted Design of Experiments in the Presence of Network-Correlated Outcomes.” Biometrika 105(4):849–858.CrossRefGoogle Scholar
Berger, R. L., and Boos, D. D.. (1994). “P Values Maximized Over a Confidence Set for the Nuisance Parameter.” Journal of the American Statistical Association 89(427):1012–1016.Google Scholar
Bhattacharya, R., Malinsky, D., and Shpitser, I.. (2019). “Causal Inference Under Interference and Network Uncertainty”. In Proceedings of the 35th Uncertainty Intelligence Conference, edited by R. P. Adams and V. Gogate, 1028–1038. Proceedings of Machine Learning Research, vol. 115.Google Scholar
Bisbee, J., and Larson, J. M.. (2017). “Testing Social Science Network Theories with Online Network Data: An Evaluation of External Validity.” American Political Science Review 111(3):502–521.CrossRefGoogle Scholar
Bond, R. M. et al. (2012). “A 61-Million-Person Experiment in Social Influence and Political Mobilization.” Nature 489(7415):295–298.CrossRefGoogle ScholarPubMed
Bowers, J., Fredrickson, M. M., and Panagopoulos, C.. (2013). “Reasoning About Interference Between Units: A General Framework.” Political Analysis 21(1):97–124.CrossRefGoogle Scholar
Bowers, J., Desmarais, B. A., Frederickson, M., Ichino, N., Lee, H.-W., and Wang, S.. (2018). “Models, Methods and Network topology: Experimental Design for The Study of Interference.” Social Networks 54:196–208.CrossRefGoogle Scholar
Cai, J., De Janvry, A., and Sadoulet, E.. (2015). “Social Networks and the Decision to Insure.” American Economic Journal: Applied Economics 7(2):81–108.Google Scholar
Choi, D. S. (2017). “Estimation of Monotone Treatment Effects in Network Experiments.” Journal of the American Statistical Association 112(519):1147–1155.CrossRefGoogle Scholar
Coppock, A. (2014). “Information Spillovers: Another Look at Experimental Estimates of Legislator Responsiveness.” Journal of Experimental Political Science 1(2):159–169.CrossRefGoogle Scholar
Coppock, A., Guess, A., and Ternovski, J.. (2016). “When Treatments Are Tweets: A Network Mobilization Experiment Over Twitter.” Political Behavior 38(1):105–128.CrossRefGoogle Scholar
Cox, D. R. (1958). Planning of Experiments. Hoboken, NJ: Wiley.Google Scholar
Ding, P., and VanderWeele, T. J.. (2016). “Sensitivity Analysis Without Assumptions.” Epidemiology 27(3):368.CrossRefGoogle ScholarPubMed
Eckles, D., Karrer, B., and Ugander, J.. (2017). “Design and Analysis of Experiments in Networks: Reducing Bias from Interference.” Journal of Causal Inference 5(1).CrossRefGoogle Scholar
Egami, N. (2017). “Unbiased Estimation and Sensitivity Analysis for Network-Specific Spillover Effects: Application to an Online Network Experiment.” https://arxiv.org/abs/1708.08171.Google Scholar
Egami, N. (2020). “Replication Data for: Spillover Effects in the Presence of Unobserved Networks.” https://doi.org/10.7910/DVN/G63OG9, Harvard Dataverse, V1.CrossRefGoogle Scholar
Foos, F., and de Rooij, E. A.. (2017). “All in the Family: Partisan Disagreement and Electoral Mobilization in Intimate Networks—A Spillover Experiment.” American Journal of Political Science 61(2):289–304.CrossRefGoogle Scholar
Forastiere, L., Airoldi, E. M., and Mealli, F.. (2016). “Identification and Estimation of Treatment and Interference Effects in Observational Studies on Networks.” Preprint, arXiv:1609.06245.Google Scholar
Fowler, J. H., Heaney, M. T., Nickerson, D. W., Padgett, J. F., and Sinclair, B.. (2011). “Causality in Political Networks.” American Politics Research 39(2):437–480.CrossRefGoogle Scholar
Halloran, M. E., and Hudgens, M. G.. (2016). “Dependent Happenings: A Recent Methodological Review.” Current Epidemiology Reports 3(4):297–305.CrossRefGoogle ScholarPubMed
Holland, P. W. (1986). “Statistics and Causal Inference.” Journal of the American statistical Association 81(396):945–960.CrossRefGoogle Scholar
Hong, G., and Raudenbush, S. W.. (2006). “Evaluating Kindergarten Retention Policy.” Journal of the American Statistical Association 101(475):901–910.CrossRefGoogle Scholar
Horvitz, D. G., and Thompson, D. J.. (1952). “A generalization of Sampling Without Replacement from a Finite Universe.” Journal of the American statistical Association 47(260):663–685.CrossRefGoogle Scholar
Hudgens, M. G., and Halloran, M. E.. (2008). “Toward Causal Inference with Interference.” Journal of the American Statistical Association 103(482):832–842.CrossRefGoogle ScholarPubMed
Ichino, N., and Schündeln, M.. (2012). “Deterring or Displacing Electoral Irregularities? Spillover Effects of Observers in a Randomized Field Experiment in Ghana.” The Journal of Politics 74(1):292–307.CrossRefGoogle Scholar
Imbens, G. W., and Rubin, D. B.. (2015). Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Jagadeesan, R., Pillai, N., and Volfovsky, A.. (2020). “Designs for Estimating the Treatment Effect in Networks with Interference.” Annals of Statistics 48(2):679–712.CrossRefGoogle Scholar
Kang, H., and Imbens, G.. (2016). “Peer Encouragement Designs in Causal Inference with Partial Interference and Identification of Local Average Network Effects.” Preprint, arXiv:1609.04464.Google Scholar
Larson, J. M., and Lewis, J. I. (2017). “Ethnic Networks.” American Journal of Political Science 61(2):350–364.CrossRefGoogle Scholar
Larson, J. M., and Lewis, J. I. (2020). “Measuring Networks in the Field.” Political Science Research and Methods 8:123–135.CrossRefGoogle Scholar
Liu, L., and Hudgens, M. G. (2014). “Large Sample Randomization Inference of Causal Effects in the Presence of Interference.” Journal of the American Statistical Association 109(505):288–301.CrossRefGoogle Scholar
Luo, X., Small, D. S., Li, C.-S. R., and Rosenbaum, P. R. (2012). “Inference with Interference Between Units in an FMRI Experiment of Motor Inhibition.” Journal of the American Statistical Association 107(498):530–541.CrossRefGoogle Scholar
Manski, C. F. (2013). “Identification of Treatment Response with Social Interactions.” The Econometrics Journal 16(1):1–23.CrossRefGoogle Scholar
Neyman, J. (1923). “On the Application of Probability Theory to Agricultural Experiments. Essay on Principles (with Discussion). Section 9 (Translated).” Statistical Science 5(4):465–472.Google Scholar
Nickerson, D. W. (2008). “Is Voting Contagious? Evidence from Two Field Experiments.” American Political Science Review 102(01):49–57.CrossRefGoogle Scholar
Ogburn, E. L. and VanderWeele, T. J. (2014). “Causal Diagrams for Interference.” Statistical Science 29(4):559–578.CrossRefGoogle Scholar
Paluck, E. L., Shepherd, H., and Aronow, P. M. (2016). “Changing Climates of Conflict: A Social Network Experiment in 56 Schools.” Proceedings of the National Academy of Sciences 113(3):566–571.CrossRefGoogle ScholarPubMed
Rosenbaum, P. R. (2002). “Covariance Adjustment in Randomized Experiments and Observational Studies.” Statistical Science 17(3):286–327.CrossRefGoogle Scholar
Rosenbaum, P. R. (2007). “Interference Between Units in Randomized Experiments.” Journal of the American Statistical Association 102(477):191–200.CrossRefGoogle Scholar
Rubin, D. B. (1974). “Estimating Causal Effects of treatments in randomized and nonrandomized studies.” Journal of educational Psychology 66(5):688.CrossRefGoogle Scholar
Rubin, D. B. (1980). “Discussion of ‘Randomization Analysis of Experimental Data: The Fisher Randomization Test Comment’ by Basu.” Journal of the American Statistical Association 75(371):591–593.Google Scholar
Särndal, C.-E., Swensson, B., and Wretman, J.. (1992). Model Assisted Survey Sampling. New York: Springer.CrossRefGoogle Scholar
Sävje, F. (2019). “Causal Inference with Misspecified Exposure Mappings.” Technical report, Yale University.Google Scholar
Sävje, F., Aronow, P. M., and Hudgens, M. G.. (2017). “Average Treatment Effects in the Presence of Unknown Interference.” Preprint, arXiv:1711.06399.Google Scholar
Sinclair, B. (2012). The Social Citizen: Peer Networks and Political Behavior. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Sinclair, B., McConnell, M., and Green, D. P. (2012). “Detecting Spillover Effects: Design and Analysis of Multilevel Experiments.” American Journal of Political Science 56(4):1055–1069.CrossRefGoogle Scholar
Sobel, M. E. (2006). “What Do Randomized Studies of Housing Mobility Demonstrate? Causal Inference in the Face of Interference.” Journal of the American Statistical Association 101(476):1398–1407.CrossRefGoogle Scholar
Sussman, D. L., and Airoldi, E. M.. (2017). “Elements of Estimatie on Theory for Causal Effects in the Presence of Network Interference.” Preprint, arXiv:1702.03578.Google Scholar
Taylor, S. J., and Eckles, D.. (2017). “Randomized Experiments to Detect and Estimate Social Influence in Networks.” In Spreading Dynamics in Social Systems, edited by Lehmann, S. and Ahn, Y.-Y.. New York: Springer.Google Scholar
Tchetgen Tchetgen, E. J. and VanderWeele, T. J.. (2010). “On Causal Inference in the Presence of Interference.” Statistical Methods in Medical Research 21(1):55–75.CrossRefGoogle Scholar
Tchetgen Tchetgen, E. J., Fulcher, I., and Shpitser, I.. (2017). “Auto-g-computation of Causal Effects on a Network.” Preprint, arXiv:1709.01577.Google Scholar
Toulis, P., and Kao, E. K.. (2013). “Estimation of Causal Peer Influence Effects.” In Proceedings of the 30th International Conference on Machine Learning, edited by S. Dasgupta and D. McAllester, 1489–1497.Google Scholar
VanderWeele, T. J., and Arah, O. A.. (2011). “Bias Formulas for Sensitivity Analysis of Unmeasured Confounding for General Outcomes, Treatments, and Confounders.” Epidemiology 22(1):42–52.CrossRefGoogle ScholarPubMed
Xiong, H., and Payne, D.. (2017). “Characteristics of Chinese Rural Networks: Evidence from Villages in Central China.” Chinese Journal of Sociology 3(1):74–97.CrossRefGoogle Scholar
Supplementary material: Link

Egami Dataset

Dataset

https://doi.org/10.7910/DVN/G63OG9
Link
Supplementary material: PDF

Egami supplementary material

Egami supplementary material
Download Egami supplementary material(PDF)
PDF 261.5 KB