Skip to main content
×
Home

Testing Social Science Network Theories with Online Network Data: An Evaluation of External Validity

  • JAMES BISBEE (a1) and JENNIFER M. LARSON (a1)
Abstract

To answer questions about the origins and outcomes of collective action, political scientists increasingly turn to datasets with social network information culled from online sources. However, a fundamental question of external validity remains untested: are the relationships measured between a person and her online peers informative of the kind of offline, “real-world” relationships to which network theories typically speak? This article offers the first direct comparison of the nature and consequences of online and offline social ties, using data collected via a novel network elicitation technique in an experimental setting. We document strong, robust similarity between online and offline relationships. This parity is not driven by shared identity of online and offline ties, but a shared nature of relationships in both domains. Our results affirm that online social tie data offer great promise for testing long-standing theories in the social sciences about the role of social networks.

Copyright
Corresponding author
James Bisbee is Ph.D. Candidate, NYU Department of Politics, 19 W. 4th St., New York, NY 10012. (james.bisbee@nyu.edu).
Jennifer M. Larson is Assistant Professor, NYU Department of Politics, 19 W. 4th St., New York, NY 10012. Corresponding Author (jenn.larson@nyu.edu).
Footnotes
Hide All

The authors thank Neal Beck, Eric Dickson, Sean Kates, and Xiang Zhou, as well as the participants of the New York University Empirical Research for Advanced Students seminar and the participants of the 2017 Harvard Experimental Political Science Graduate Student Conference, for their valuable feedback on the project.

Footnotes
References
Hide All
Acemoglu Daron, Ozdaglar Asuman, and ParandehGheibi Ali. 2010. “Spread of (Mis) Information in Social Networks.” Games and Economic Behavior 70 (2): 194227.
Ahn Yong-Yeol, Han Seungyeop, Kwak Haewoon, Moon Sue, and Jeong Hawoong. 2007. “Analysis of Topological Characteristics of Huge Online Social Networking Services.” In Proceedings of the 16th International Conference on World Wide Web. New York, NY: ACM Press, 835–44.
Amir Ofra, Rand David G., and Gal Ya’akov Kobi. 2012. “Economic Games on the Internet: The Effect of $1 Stakes.” PLoS One 7 (2): e31461.
Aral Sinan, and Walker Dylan. 2011. “Creating Social Contagion Through Viral Product Design: A Randomized Trial of Peer Influence in Networks.” Management Science 57 (9): 1623–39.
Bartneck Christoph, Duenser Andreas, Moltchanova Elena, and Zawieska Karolina. 2015. “Comparing the Similarity of Responses Received from Studies in Amazon’s Mechanical Turk to Studies Conducted Online and with Direct Recruitment.” PloS One 10 (4): e0121595.
Berinsky Adam J., Huber Gregory A., and Lenz Gabriel S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk.” Political Analysis 20 (3): 351–68.
Bond Robert M., Fariss Christopher J., Jones Jason J., Kramer Adam D. I., Marlow Cameron, Settle Jaime E., and Fowler James H.. 2012. “A 61-Million-Person Experiment in Social Influence and Political Mobilization.” Nature 489 (7415): 295–98.
Bramoullé Yann, and Kranton Rachel. 2007. “Public Goods in Networks.” Journal of Economic Theory 135 (1): 478–94.
Calvo-Armengol Antoni, and Jackson Matthew O.. 2004. “The Effects of Social Networks on Employment and Inequality.” American Economic Review 94 (3): 426–54.
Centola Damon. 2010. “The Spread of Behavior in an Online Social Network Experiment.” Science 329 (5996): 1194–97.
Centola Damon, and Macy Michael. 2007. “Complex Contagions and the Weakness of Long Ties.” American Journal of Sociology 113 (3): 702–34.
Chandler Jesse, and Shapiro Danielle. 2016. “Conducting Clinical Science Research on Amazon Mechanical Turk.” Annual Review of Clinical Psychology 12 (1): 35.
Christakis Nicholas A., and Fowler James H.. 2013. “Social Contagion Theory: Examining Dynamic Social Networks and Human Behavior.” Statistics in Medicine 32 (4): 556–77.
Chwe Michael Suk-Young. 2000. “Communication and Coordination in Social Networks.” Review of Economic Studies 67 (1): 1 16.
Clifford Scott, Jewell Ryan M., and Waggoner Philip D.. 2015. “Are Samples Drawn from Mechanical Turk Valid for Research on Political Methodology?Research and Politics 2 (4): 205316801562272.
Conley Timothy G., and Udry Christopher R.. 2010. “Learning About a New Technology: Pineapple in Ghana.” American Economic Review 100 (1): 3569.
Crump Matthew J. C., McDonnell John V., and Gureckis Todd M.. 2013. “Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research.” PloS One 8 (3): e57410.
Ellison Nicole B., Steinfield Charles, and Lampe Cliff. 2007. “The Benefits of Facebook ‘Friends:’ Social Capital and College Students’ Use of Online Social Network Sites.” Journal of Computer-Mediated Communication 12 (4): 1143–68.
Ferrali R. 2016. “Partners in Crime? A Theory of Corruption as a Criminal Network.” Working Paper. http://scholar.princeton.edu/sites/default/files/ferrali/files/ferrali_corruption.pdf.
Gilbert Eric, and Karahalios Karrie. 2009. “Predicting Tie Strength with Social Media.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM Press, 211–20.
González-Bailón Sandra, and Wang Ning. 2016. “Networked Discontent: The Anatomy of Protest Campaigns in Social Media.” Social Networks 44: 95104.
Goodman Joseph K., Cryder Cynthia E., and Cheema Amar. 2013. “Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples.” Journal of Behavioral Decision Making 26 (3): 213–24.
Grabowicz Przemyslaw A., Ramasco José J., Moro Esteban, Pujol Josep M., and Eguiluz Victor M.. 2012. “Social Features of Online Networks: The Strength of Intermediary Ties in Online Social Media.” PloS One 7 (1): e29358.
Granovetter Mark S. 1973. “The Strength of Weak Ties.” American Journal of Sociology 78 (6): 1360–80.
Horton John J., Rand David G., and Zeckhauser Richard J.. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399425.
Huff Connor, and Tingley Dustin. 2015. “‘Who Are These People?’ Evaluating the Demographic Characteristics and Political Preferences of MTurk Survey Respondents.” Research and Politics 2 (1): 112.
Jackson Matthew O., Rodriguez-Barraquer Tomas, and Tan Xu. 2012. “Social Capital and Social Quilts: Network Patterns of Favor Exchange.” American Economic Review 102 (5): 1857–97.
Joinson Adam. 1999. “Social Desirability, Anonymity, and Internet-Based Questionnaires.” Behavior Research Methods, Instruments, and Computers 31 (3): 433–38.
Jones Jason J., Settle Jaime E., Bond Robert M., Fariss Christopher J., Marlow Cameron, and Fowler James H.. 2013. “Inferring Tie Strength from Online Directed Behavior.” PloS One 8 (1): e52168.
Larson Jennifer M. 2016. “Interethnic Conflict and the Potential Dangers of Cross-Group Ties.” Journal of Peace Research 53 (3): 459–71.
Larson Jennifer M. 2017. “Networks and Interethnic Cooperation.” Journal of Politics 79 (2): 546–59.
Larson Jennifer M., and Lewis Janet I.. 2017. “Measuring Networks in the Field.” Working Paper. http://goo.gl/cRWP3j.
Lazer David, Pentland Alex Sandy, Adamic Lada, Aral Sinan, Barabasi Albert Laszlo, Brewer Devon, Christakis Nicholas, Contractor Noshir, Fowler James, Gutmann Myron, et al. 2009. “Life in the Network: The Coming Age of Computational Social Science.” Science 323 (5915): 721.
Levay Kevin E., Freese Jeremy, and Druckman James N.. 2016. “The Demographic and Political Composition of Mechanical Turk Samples.” SAGE Open 6 (1): 116.
Marwell Gerald, Oliver Pamela E., and Prahl Ralph. 1988. “Social Networks and Collective Action: A Theory of the Critical Mass. III.” American Journal of Sociology 94 (3): 502–34.
McAdam Doug, and Paulsen Ronnelle. 1993. “Specifying the Relationship Between Social Ties and Activism.” American Journal of Sociology 99 (3): 640–67.
McClurg Scott D. 2006. “The Electoral Relevance of Political Talk: Examining Disagreement and Expertise Effects in Social Networks on Political Participation.” American Journal of Political Science 50 (3): 737–54.
Milgram Stanley, and Gudehus Christian. 1978. Obedience to Authority. New York, NY: Ziff-Davis Publishing Company.
Opp Karl-Dieter, and Gern Christiane. 1993. “Dissident Groups, Personal Networks, and Spontaneous Cooperation: The East German Revolution of 1989.” American Sociological Review 58 (5): 659–80.
Paolacci Gabriele, Chandler Jesse, and Ipeirotis Panagiotis G.. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision Making 5 (5): 411–19.
Parkinson Sarah Elizabeth. 2013. “Organizing Rebellion: Rethinking High-Risk Mobilization and Social Networks in War.” American Political Science Review 107 (03): 418–32.
Patty John W., and Penn Elizabeth Maggie. 2014. “Sequential Decision Making and Information Aggregation in Small Networks.” Political Science Research and Methods 2 (02): 243–71.
Putnam Robert D. 2001. Bowling Alone: The Collapse and Revival of American Community. New York, NY: Simon & Schuster.
Rao Neel, Mobius Markus M., and Rosenblat Tanya. 2007. “Social Networks and Vaccination Decisions.” Working Paper (Series Federal Reserve Bank of Boston No. 07-12). http://hdl.handle.net/10419/55601.
Salganik Matthew J., Dodds Peter Sheridan, and Watts Duncan J.. 2006. “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.” Science 311 (5762): 854– 56.
Siegel David A. 2009. “Social Networks and Collective Action.” American Journal of Political Science 53 (1): 122–38.
Sinclair Betsy. 2012. The Social Citizen: Peer Networks and Political Behavior. Chicago, IL: University of Chicago Press.
Staniland Paul. 2014. Networks of Rebellion: Explaining Insurgent Cohesion and Collapse. Ithaca, NY: Cornell University Press.
Steinert-Threlkeld Zachary C. 2017. “Spontaneous Collective Action: Peripheral Mobilization during the Arab Spring.” American Political Science Review 111 (2): 379403.
Steinert-Threlkeld Zachary, Mocanu Delia, Vespignani Alessandro, and Fowler James. 2015. “Online Social Networks and Offline Protest.” EPJ Data Science 4 (1): 19.
Stewart Neil, Ungemach Christoph, Harris Adam J. L., Bartels Daniel M., Newell Ben R., Paolacci Gabriele, and Chandler Jesse. 2015. “The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers.” Judgment and Decision Making 10 (5): 479–91.
Subrahmanyam Kaveri, Reich Stephanie M., Waechter Natalia, and Espinoza Guadalupe. 2008. “Online and Offline Social Networks: Use of Social Networking Sites by Emerging Adults.” Journal of Applied Developmental Psychology 29 (6): 420–33.
Tezcür Güneş Murat. 2016. “Ordinary People, Extraordinary Risks: Participation in an Ethnic Rebellion.” American Political Science Review 110 (02): 247–64.
Watt Susan Ellen, Lea Martin, Spears Russell, and Rogers Paul. 2002. How Social Is Internet Communication? Anonymity Effects in Computer-Mediated Groups. In Virtual Society? Technology, Cyberbole, Reality, ed. Woolgar Steve. Oxford, UK: Oxford University Press, 6177.
Weinberg Jill D., Freese Jeremy, and McElhattan David. 2015. “Comparing Data Characteristics and Results of an Online Factorial Survey Between a Population-Based and a Crowdsource-Recruited Sample.” Sociological Science 1 (doi:10.15195/v1.a19): 292310.
Westfall Peter H., and Young S. Stanley. 1993. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. Vol. 279. New York, NY: John Wiley & Sons.
Wilcox Keith, and Stephen Andrew T.. 2013. “Are Close Friends the Enemy? Online Social Networks, Self-Esteem, and Self-Control.” Journal of Consumer Research 40 (1): 90103.
Wolitzky Alexander. 2013. “Cooperation with Network Monitoring.” Review of Economic Studies 80 (1): 395427.
Zimbardo Philip G. 1969. “The Human Choice: Individuation, Reason, and Order Versus Deindividuation, Impulse, and Chaos.” In Nebraska Symposium on Motivation. Lincoln, NE: University of Nebraska Press, 237307.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary Materials

Bisbee and Larson supplementary material
Online Appendix

 PDF (302 KB)
302 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 39
Total number of PDF views: 499 *
Loading metrics...

Abstract views

Total abstract views: 1677 *
Loading metrics...

* Views captured on Cambridge Core between 13th June 2017 - 20th November 2017. This data will be updated every 24 hours.