Skip to main content
×
×
Home

Testing Social Science Network Theories with Online Network Data: An Evaluation of External Validity

  • JAMES BISBEE (a1) and JENNIFER M. LARSON (a1)
Abstract

To answer questions about the origins and outcomes of collective action, political scientists increasingly turn to datasets with social network information culled from online sources. However, a fundamental question of external validity remains untested: are the relationships measured between a person and her online peers informative of the kind of offline, “real-world” relationships to which network theories typically speak? This article offers the first direct comparison of the nature and consequences of online and offline social ties, using data collected via a novel network elicitation technique in an experimental setting. We document strong, robust similarity between online and offline relationships. This parity is not driven by shared identity of online and offline ties, but a shared nature of relationships in both domains. Our results affirm that online social tie data offer great promise for testing long-standing theories in the social sciences about the role of social networks.

Copyright
Corresponding author
James Bisbee is Ph.D. Candidate, NYU Department of Politics, 19 W. 4th St., New York, NY 10012. (james.bisbee@nyu.edu).
Jennifer M. Larson is Assistant Professor, NYU Department of Politics, 19 W. 4th St., New York, NY 10012. Corresponding Author (jenn.larson@nyu.edu).
Footnotes
Hide All

The authors thank Neal Beck, Eric Dickson, Sean Kates, and Xiang Zhou, as well as the participants of the New York University Empirical Research for Advanced Students seminar and the participants of the 2017 Harvard Experimental Political Science Graduate Student Conference, for their valuable feedback on the project.

Footnotes
References
Hide All
Acemoglu, Daron, Ozdaglar, Asuman, and ParandehGheibi, Ali. 2010. “Spread of (Mis) Information in Social Networks.” Games and Economic Behavior 70 (2): 194227.
Ahn, Yong-Yeol, Han, Seungyeop, Kwak, Haewoon, Moon, Sue, and Jeong, Hawoong. 2007. “Analysis of Topological Characteristics of Huge Online Social Networking Services.” In Proceedings of the 16th International Conference on World Wide Web. New York, NY: ACM Press, 835–44.
Amir, Ofra, Rand, David G., and Gal, Ya’akov Kobi. 2012. “Economic Games on the Internet: The Effect of $1 Stakes.” PLoS One 7 (2): e31461.
Aral, Sinan, and Walker, Dylan. 2011. “Creating Social Contagion Through Viral Product Design: A Randomized Trial of Peer Influence in Networks.” Management Science 57 (9): 1623–39.
Bartneck, Christoph, Duenser, Andreas, Moltchanova, Elena, and Zawieska, Karolina. 2015. “Comparing the Similarity of Responses Received from Studies in Amazon’s Mechanical Turk to Studies Conducted Online and with Direct Recruitment.” PloS One 10 (4): e0121595.
Berinsky, Adam J., Huber, Gregory A., and Lenz, Gabriel S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk.” Political Analysis 20 (3): 351–68.
Bond, Robert M., Fariss, Christopher J., Jones, Jason J., Kramer, Adam D. I., Marlow, Cameron, Settle, Jaime E., and Fowler, James H.. 2012. “A 61-Million-Person Experiment in Social Influence and Political Mobilization.” Nature 489 (7415): 295–98.
Bramoullé, Yann, and Kranton, Rachel. 2007. “Public Goods in Networks.” Journal of Economic Theory 135 (1): 478–94.
Calvo-Armengol, Antoni, and Jackson, Matthew O.. 2004. “The Effects of Social Networks on Employment and Inequality.” American Economic Review 94 (3): 426–54.
Centola, Damon. 2010. “The Spread of Behavior in an Online Social Network Experiment.” Science 329 (5996): 1194–97.
Centola, Damon, and Macy, Michael. 2007. “Complex Contagions and the Weakness of Long Ties.” American Journal of Sociology 113 (3): 702–34.
Chandler, Jesse, and Shapiro, Danielle. 2016. “Conducting Clinical Science Research on Amazon Mechanical Turk.” Annual Review of Clinical Psychology 12 (1): 35.
Christakis, Nicholas A., and Fowler, James H.. 2013. “Social Contagion Theory: Examining Dynamic Social Networks and Human Behavior.” Statistics in Medicine 32 (4): 556–77.
Chwe, Michael Suk-Young. 2000. “Communication and Coordination in Social Networks.” Review of Economic Studies 67 (1): 1 16.
Clifford, Scott, Jewell, Ryan M., and Waggoner, Philip D.. 2015. “Are Samples Drawn from Mechanical Turk Valid for Research on Political Methodology?Research and Politics 2 (4): 205316801562272.
Conley, Timothy G., and Udry, Christopher R.. 2010. “Learning About a New Technology: Pineapple in Ghana.” American Economic Review 100 (1): 3569.
Crump, Matthew J. C., McDonnell, John V., and Gureckis, Todd M.. 2013. “Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research.” PloS One 8 (3): e57410.
Ellison, Nicole B., Steinfield, Charles, and Lampe, Cliff. 2007. “The Benefits of Facebook ‘Friends:’ Social Capital and College Students’ Use of Online Social Network Sites.” Journal of Computer-Mediated Communication 12 (4): 1143–68.
Ferrali, R. 2016. “Partners in Crime? A Theory of Corruption as a Criminal Network.” Working Paper. http://scholar.princeton.edu/sites/default/files/ferrali/files/ferrali_corruption.pdf.
Gilbert, Eric, and Karahalios, Karrie. 2009. “Predicting Tie Strength with Social Media.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM Press, 211–20.
González-Bailón, Sandra, and Wang, Ning. 2016. “Networked Discontent: The Anatomy of Protest Campaigns in Social Media.” Social Networks 44: 95104.
Goodman, Joseph K., Cryder, Cynthia E., and Cheema, Amar. 2013. “Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples.” Journal of Behavioral Decision Making 26 (3): 213–24.
Grabowicz, Przemyslaw A., Ramasco, José J., Moro, Esteban, Pujol, Josep M., and Eguiluz, Victor M.. 2012. “Social Features of Online Networks: The Strength of Intermediary Ties in Online Social Media.” PloS One 7 (1): e29358.
Granovetter, Mark S. 1973. “The Strength of Weak Ties.” American Journal of Sociology 78 (6): 1360–80.
Horton, John J., Rand, David G., and Zeckhauser, Richard J.. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399425.
Huff, Connor, and Tingley, Dustin. 2015. “‘Who Are These People?’ Evaluating the Demographic Characteristics and Political Preferences of MTurk Survey Respondents.” Research and Politics 2 (1): 112.
Jackson, Matthew O., Rodriguez-Barraquer, Tomas, and Tan, Xu. 2012. “Social Capital and Social Quilts: Network Patterns of Favor Exchange.” American Economic Review 102 (5): 1857–97.
Joinson, Adam. 1999. “Social Desirability, Anonymity, and Internet-Based Questionnaires.” Behavior Research Methods, Instruments, and Computers 31 (3): 433–38.
Jones, Jason J., Settle, Jaime E., Bond, Robert M., Fariss, Christopher J., Marlow, Cameron, and Fowler, James H.. 2013. “Inferring Tie Strength from Online Directed Behavior.” PloS One 8 (1): e52168.
Larson, Jennifer M. 2016. “Interethnic Conflict and the Potential Dangers of Cross-Group Ties.” Journal of Peace Research 53 (3): 459–71.
Larson, Jennifer M. 2017. “Networks and Interethnic Cooperation.” Journal of Politics 79 (2): 546–59.
Larson, Jennifer M., and Lewis, Janet I.. 2017. “Measuring Networks in the Field.” Working Paper. http://goo.gl/cRWP3j.
Lazer, David, Pentland, Alex Sandy, Adamic, Lada, Aral, Sinan, Barabasi, Albert Laszlo, Brewer, Devon, Christakis, Nicholas, Contractor, Noshir, Fowler, James, Gutmann, Myron, et al. 2009. “Life in the Network: The Coming Age of Computational Social Science.” Science 323 (5915): 721.
Levay, Kevin E., Freese, Jeremy, and Druckman, James N.. 2016. “The Demographic and Political Composition of Mechanical Turk Samples.” SAGE Open 6 (1): 116.
Marwell, Gerald, Oliver, Pamela E., and Prahl, Ralph. 1988. “Social Networks and Collective Action: A Theory of the Critical Mass. III.” American Journal of Sociology 94 (3): 502–34.
McAdam, Doug, and Paulsen, Ronnelle. 1993. “Specifying the Relationship Between Social Ties and Activism.” American Journal of Sociology 99 (3): 640–67.
McClurg, Scott D. 2006. “The Electoral Relevance of Political Talk: Examining Disagreement and Expertise Effects in Social Networks on Political Participation.” American Journal of Political Science 50 (3): 737–54.
Milgram, Stanley, and Gudehus, Christian. 1978. Obedience to Authority. New York, NY: Ziff-Davis Publishing Company.
Opp, Karl-Dieter, and Gern, Christiane. 1993. “Dissident Groups, Personal Networks, and Spontaneous Cooperation: The East German Revolution of 1989.” American Sociological Review 58 (5): 659–80.
Paolacci, Gabriele, Chandler, Jesse, and Ipeirotis, Panagiotis G.. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision Making 5 (5): 411–19.
Parkinson, Sarah Elizabeth. 2013. “Organizing Rebellion: Rethinking High-Risk Mobilization and Social Networks in War.” American Political Science Review 107 (03): 418–32.
Patty, John W., and Penn, Elizabeth Maggie. 2014. “Sequential Decision Making and Information Aggregation in Small Networks.” Political Science Research and Methods 2 (02): 243–71.
Putnam, Robert D. 2001. Bowling Alone: The Collapse and Revival of American Community. New York, NY: Simon & Schuster.
Rao, Neel, Mobius, Markus M., and Rosenblat, Tanya. 2007. “Social Networks and Vaccination Decisions.” Working Paper (Series Federal Reserve Bank of Boston No. 07-12). http://hdl.handle.net/10419/55601.
Salganik, Matthew J., Dodds, Peter Sheridan, and Watts, Duncan J.. 2006. “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.” Science 311 (5762): 854– 56.
Siegel, David A. 2009. “Social Networks and Collective Action.” American Journal of Political Science 53 (1): 122–38.
Sinclair, Betsy. 2012. The Social Citizen: Peer Networks and Political Behavior. Chicago, IL: University of Chicago Press.
Staniland, Paul. 2014. Networks of Rebellion: Explaining Insurgent Cohesion and Collapse. Ithaca, NY: Cornell University Press.
Steinert-Threlkeld, Zachary C. 2017. “Spontaneous Collective Action: Peripheral Mobilization during the Arab Spring.” American Political Science Review 111 (2): 379403.
Steinert-Threlkeld, Zachary, Mocanu, Delia, Vespignani, Alessandro, and Fowler, James. 2015. “Online Social Networks and Offline Protest.” EPJ Data Science 4 (1): 19.
Stewart, Neil, Ungemach, Christoph, Harris, Adam J. L., Bartels, Daniel M., Newell, Ben R., Paolacci, Gabriele, and Chandler, Jesse. 2015. “The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers.” Judgment and Decision Making 10 (5): 479–91.
Subrahmanyam, Kaveri, Reich, Stephanie M., Waechter, Natalia, and Espinoza, Guadalupe. 2008. “Online and Offline Social Networks: Use of Social Networking Sites by Emerging Adults.” Journal of Applied Developmental Psychology 29 (6): 420–33.
Tezcür, Güneş Murat. 2016. “Ordinary People, Extraordinary Risks: Participation in an Ethnic Rebellion.” American Political Science Review 110 (02): 247–64.
Watt, Susan Ellen, Lea, Martin, Spears, Russell, and Rogers, Paul. 2002. How Social Is Internet Communication? Anonymity Effects in Computer-Mediated Groups. In Virtual Society? Technology, Cyberbole, Reality, ed. Woolgar, Steve. Oxford, UK: Oxford University Press, 6177.
Weinberg, Jill D., Freese, Jeremy, and McElhattan, David. 2015. “Comparing Data Characteristics and Results of an Online Factorial Survey Between a Population-Based and a Crowdsource-Recruited Sample.” Sociological Science 1 (doi:10.15195/v1.a19): 292310.
Westfall, Peter H., and Young, S. Stanley. 1993. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. Vol. 279. New York, NY: John Wiley & Sons.
Wilcox, Keith, and Stephen, Andrew T.. 2013. “Are Close Friends the Enemy? Online Social Networks, Self-Esteem, and Self-Control.” Journal of Consumer Research 40 (1): 90103.
Wolitzky, Alexander. 2013. “Cooperation with Network Monitoring.” Review of Economic Studies 80 (1): 395427.
Zimbardo, Philip G. 1969. “The Human Choice: Individuation, Reason, and Order Versus Deindividuation, Impulse, and Chaos.” In Nebraska Symposium on Motivation. Lincoln, NE: University of Nebraska Press, 237307.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary materials

Bisbee and Larson supplementary material
Online Appendix

 PDF (302 KB)
302 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed