Hostname: page-component-6766d58669-l4t7p Total loading time: 0 Render date: 2026-05-15T11:30:37.173Z Has data issue: false hasContentIssue false
  • English
  • Français

Crowdsourcing Reliable Local Data

Published online by Cambridge University Press:  20 September 2019

Jane Lawrence Sumner Open the ORCID record for Jane Lawrence Sumner [Opens in a new window] ,
Emily M. Farris Open the ORCID record for Emily M. Farris [Opens in a new window]  and
Mirya R. Holman Open the ORCID record for Mirya R. Holman [Opens in a new window]
Jane Lawrence Sumner
Affiliation:
University of Minnesota, Political Science, 1414 Social Sciences, 267 19th Ave S, Minneapolis, MN 55455, USA. Email: jlsumner@umn.edu
Emily M. Farris
Affiliation:
Texas Christian University, Political Science, Scharbauer Hall 2012G, Fort Worth, TX 76129, USA. Email: e.farris@tcu.edu
Mirya R. Holman*
Affiliation:
Tulane University, Political Science, 6823 St. Charles Avenue, Norman Mayer Building, New Orleans, LA 70118, USA. Email: mholman@tulane.edu
*
*Email: mholman@tulane.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

The adage “All politics is local” in the United States is largely true. Of the United States’ 90,106 governments, 99.9% are local governments. Despite variations in institutional features, descriptive representation, and policy-making power, political scientists have been slow to take advantage of these variations. One obstacle is that comprehensive data on local politics is often extremely difficult to obtain; as a result, data is unavailable or costly, hard to replicate, and rarely updated. We provide an alternative: crowdsourcing this data. We demonstrate and validate crowdsourcing data on local politics using two different data collection projects. We evaluate different measures of consensus across coders and validate the crowd’s work against elite and professional datasets. In doing so, we show that crowdsourced data is both highly accurate and easy to use. In doing so, we demonstrate that nonexperts can be used to collect, validate, or update local data.

Keywords

data collectioncrowdsourcingsurveysreplicationlocal politics

Information

Type
Articles
Information
Political Analysis , Volume 28 , Issue 2 , April 2020 , pp. 244 - 262
Copyright
Copyright © The Author(s) 2019. Published by Cambridge University Press on behalf of the Society for Political Methodology.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Authors’ note: Sumner is an assistant professor of political science at the University of Minnesota, Twin Cities. Farris is an associate professor of political science at Texas Christian University. Holman is an associate professor of political science at Tulane University. All authors contributed equally to this work. This paper was previously presented as “Is 2chainz the Mayor of Atlanta? The Crowd Says ‘No”’ at the 2018 Saint Louis Area Methods Meeting (SLAMM) and the 2018 Political Methodology meeting in Provo, Utah. Thanks to Vito D’Orazio, Justin Grimmer, Justin Esarey, participants at 2018 SLAMM and 2018 PolMeth meetings, and four anonymous reviewers for helpful comments on the paper. The University of Minnesota and Tulane University provided monetary support for this research. All errors remain our own. All data from the project are available from Holman, Sumner, and Farris (2019) “Replication Data for: Crowdsourcing reliable local data”, doi:10.7910/DVN/LSEEKB, Harvard Dataverse.

Contributing Editor: Jeff Gill

References

Alvarez, R. M., Key, E. M., and Núñez, L.. 2018. “Research Replication: Practical Considerations.” PS: Political Science & Politics 51(2):422–426.Google Scholar
Andersen, D., and Lau, R.. 2018. “Pay Rates and Subject Performance in Social Science Experiments Using Crowdsourced Online Samples.” Journal of Experimental Political Science 5(3):217–229.Google Scholar
Barnes, T. D. 2016. Gendering Legislative Behavior: Institutional Constraints and Collaboration in Argentina. New York: Cambridge University Press.Google Scholar
Barnes, T. D., and Holman, M. R.. 2018. “Taking Diverse Backgrounds into Account in Studies of Political Ambition and Representation.” Politics, Groups, and Identities, OnlineFirst. doi:10.1080/21565503.2018.1532916.Google Scholar
Barnes, T. D., Tchintian, C., and Alles, S.. 2017. “Assessing Ballot Structure and Split Ticket Voting.” Journal of Politics 79(2):439–456.Google Scholar
Baron, J., and Maxwell, N. P.. 1996. “Cost of Public Goods Affects Willingness to Pay for Them.” Journal of Behavioral Decision Making 9(3):173–183.Google Scholar
Beach, B., and Jones, D. B.. 2017. “Ethnic Diversity in Government and the Provision of Public Goods.” American Economic Journal: Economic Policy 9(1):112–136.Google Scholar
Benoit, K., Conway, D., Lauderdale, B. E., Laver, M., and Mikhaylov, S.. 2016. “Crowd-Sourced Text Analysis: Reproducible and Agile Production of Political Data.” American Political Science Review 110(2):278–295.Google Scholar
Berinsky, A. J., Huber, G. A., and Lenz, G. S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon.Com’s Mechanical Turk.” Political Analysis 20(3):351–368.Google Scholar
Bishin, B. G., and Dennis, C. D.. 2002. “Heterogeneity and Representation Reconsidered: A Replication and Extension.” Political Analysis 10(2):210–210.Google Scholar
Black, R. C., and Johnson, T. R.. 2019. “Behind the Velvet Curtain: Understanding Supreme Court Conference Discussions Through the Justices’ Conference Notes.” Journal of Appellate Practice and Process, Forthcoming.Google Scholar
Boussalis, C., Coan, T. G., and Holman, M. R.. 2018. “Climate Communication from Cities in the United States.” Climate Change 149(2):173–187.Google Scholar
Browning, R. P., Marshall, D. R., and Tabb, D. H.. 1986. “Protest Is Not Enough: A Theory of Political Incorporation.” PS 19(3):576–581.Google Scholar
Burns, P. F. 2006. Electoral Politics Is Not Enough: Racial and Ethnic Minorities and Urban Politics. New York: SUNY Press.Google Scholar
de Benedictis-Kessner, J.2019 “Strategic Government Communication Distorts Reality.” Working paper. Boston, MA.Google Scholar
de Benedictis-Kessner, J., and Warshaw, C.. 2016. “Mayoral Partisanship and Municipal Fiscal Policy.” The Journal of Politics 78(4):1124–1138.Google Scholar
de Benedictis-Kessner, J., and Warshaw, C.. 2018. “Politics in Forgotten Governments: The Partisan Composition of County Legislatures and County Fiscal Policies.” The Journal of Politics, OnlineFirst.Google Scholar
Casey, L., Chandler, J., Levine, A. S., Proctor, A., and Strolovitch, D. Z.. 2017. “Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection.” SAGE Open 7(2):215824401771277.Google Scholar
Collingwood, L., Omar El-Khatib, S., and O’Brien, B. G.. 2019. “American Legislative Exchange Council and the Diffusion of Anti-Sanctuary Policy.” Policy Studies Journal, OnlineFirst.Google Scholar
Crowder-Meyer, M. 2013. “Gendered Recruitment without Trying: How Local Party Recruiters Affect Women’s Representation.” Politics & Gender 9(04):390–413.Google Scholar
Crowder-Meyer, M., Gadarian, S. K., and Trounstine, J.. 2015. “Electoral Institutions, Gender Stereotypes, and Women’s Local Representation.” Politics, Groups, and Identities 3(2):318–334.Google Scholar
Crowder-Meyer, M., and Smith, A. R.. 2015. “How the Strategic Context Affects Women’s Emergence and Success.” Politics, Groups, and Identities 3(2):295–317.Google Scholar
Dahl, R. 1961. Who Governs? Democracy and Power in an American City. New Haven, CT: Yale University Press.Google Scholar
D’Orazio, V., Kenwick, M., Lane, M., Palmer, G., and Reitter, D.. 2016. “Crowdsourcing the Measurement of Interstate Conflict.” PLOS ONE 11(6):e0156527.Google Scholar
Einstein, K. L., and Kogan, V.. 2016. “Pushing the City Limits Policy Responsiveness in Municipal Government.” Urban Affairs Review 52(1):3–32.Google Scholar
Einstein, K. L., Palmer, M., and Glick, D. M.. 2019. “Who Participates in Local Government? Perspectives on Politics 17(1):28–46.Google Scholar
Elman, C., Kapiszewski, D., and Lupia, A.. 2018. “Transparent Social Inquiry: Implications for Political Science.” Annual Review of Political Science 21:29–47.Google Scholar
Farris, E. M., and Holman, M. R.. 2015. “Attitudes and Policies in the County Sheriff Office Regarding Violence Against Women.” Social Science Quarterly 96(4):1117–1135.Google Scholar
Ferreira, F., and Gyourko, J.. 2014. “Does Gender Matter for Political Leadership? The Case of U.S. Mayors.” Journal of Public Economics 112(April):24–39.Google Scholar
Gandhi, J., and Sumner, J. L.. 2019. “Measuring the Consolidation of Power in Non-Democracies.” Working paper.Google Scholar
Gerber, E. R., and Hopkins, D. J.. 2010. “Estimating the Impact of Mayoral Partisanship on City Policy.” American Journal of Political Science 55(2):326–339.Google Scholar
Gillespie, A. 2012. The New Black Politician: Cory Booker, Newark, and Post-Racial America. NYU Press.Google Scholar
Gleibs, I. H. 2017. “Are all “Research Fields” Equal? Rethinking Practice for the use of Data from Crowdsourcing Market Places.” Behavior Research Methods 49:1333–1342.Google Scholar
Goodman, C. B. 2018. “Jurisdictional Overlap and the Size of the Local Public Workforce.” State and Local Government Review 50(1):15–23.Google Scholar
Guhin, J. 2018. “Colorblind Islam: The Racial Hinges of Immigrant Muslims in the United States.” Social Inclusion 6(2):87–97.Google Scholar
Hamidullah, M. F., Riccucci, N. M., and Pandey, S. K.. 2015. “Women in City Hall: Gender Dimensions of Managerial Values.” The American Review of Public Administration 45(3):247–262.Google Scholar
Hansen, E. R., and Clark, C. J.. 2019. “Institutional Barriers to Diversity in Party Leadership: Evidence from State Legislatures.” Working paper.Google Scholar
Holman, M. R. 2014. “Sex and the City: Female Leaders And Spending On Social Welfare Programs In U.S. Municipalities.” Journal of Urban Affairs 36(4):701–715.Google Scholar
Holman, M. R. 2015. Women in Politics in the American City. Philadelphia, PA: Temple University Press.Google Scholar
Holman, M. R. 2016. “Gender, Political Rhetoric, and Moral Metaphors in State of the City Addresses.” Urban Affairs Review 52(4):501–530.Google Scholar
Holman, M. R. 2017. “Women in Local Government: What We Know and Where We Go from Here.” State and Local Government Review 49(4):285–296.Google Scholar
Holman, M. R., and Mahoney, A. M.. 2018. “Stop, Collaborate, and Listen: Women’s Collaboration in US State Legislatures.” Legislative Studies Quarterly, OnlineFirst.Google Scholar
Holman, M. R., Sumner, J. L., and Farris, E.. 2019. “Replication Data for: Crowdsourcing Reliable Local Data.” https://doi.org/10.7910/DVN/LSEEKB, Harvard Dataverse, V1.Google Scholar
Hopkins, D. J., and Pettingill, L.. 2017. “Retrospective Voting in Big-City US Mayoral Elections.” Political Science Research and Methods 6(4):697–714.Google Scholar
Jäger, K. 2017. “The Potential of Online Sampling for Studying Political Activists around the World and across Time.” Political Analysis 25(3):329–343.Google Scholar
Kaufmann, K. M. 2004. The Urban Voter: Group Conflict and Mayoral Voting Behavior in American Cities. Ann Arbor, MI: University of Michigan Press.Google Scholar
Kirkland, P.2018. “The Business of Being Mayor: Mayors and Fiscal Policy in U.S. Cities.” Working paper. Princeton, NJ.Google Scholar
Krause, R. M., Yi, H., and Feiock, R. C.. 2016. “Applying Policy Termination Theory to the Abandonment of Climate Protection Initiatives by U.S. Local Governments.” Policy Studies Journal 44(2):176–195.Google Scholar
Lay, J. C., and Bauman, A.. 2017. “Private Governance of Public Schools: Representation, Priorities, and Compliance in New Orleans Charter School Boards.” Urban Affairs Review, OnlineFirst.Google Scholar
Leeper, T. J. 2016. “Crowdsourced Data Preprocessing with R and Amazon Mechanical Turk.” The R Journal 8(1):276–288.Google Scholar
Litman, L., Robinson, J., and Abberbock, T.. 2017. “TurkPrime.Com: A Versatile Crowdsourcing Data Acquisition Platform for the Behavioral Sciences.” Behavior Research Methods 49(2):433–442, doi:10.3758/s13428-016-0727-z.Google Scholar
Maestas, C. D., Buttice, M. K., and Stone, W. J.. 2014. “Extracting Wisdom from Experts and Small Crowds: Strategies for Improving Informant-Based Measures of Political Concepts.” Political Analysis 22(3):354–373.Google Scholar
Marschall, M., Shah, P., and Ruhil, A.. 2011. “The Study of Local Elections: A Looking Glass into the Future.” PS: Political Science & Politics 44(1):97–100.Google Scholar
Nelson, K. L., and Svara, J. H.. 2010. “Adaptation of Models Versus Variations in Form: Classifying Structures of City Government.” Urban Affairs Review 45(4):544–562.Google Scholar
Oliver, S., and Conroy, M.. 2018. “How Masculinity Predicts Recruitment of City Council Members.” American Politics Research 46(6):1094–1122.Google Scholar
Owens, M. L., and Sumner, J. L.. 2016. “Regional or Parochial? Support for Cross-Community Sharing Within City-Regions.” Journal of Urban Affairs, Online First.Google Scholar
Pandya, S. S. 2014. Trading Spaces: Foreign Direct Investment Regulation, 1970–2000. Cambridge University Press.Google Scholar
Sapotichne, J., Jones, B. D., and Wolfe, M.. 2007. “Is Urban Politics a Black Hole? Analyzing the Boundary Between Political Science and Urban Politics.” Urban Affairs Review 43(1):76–106.Google Scholar
Schwindt-Bayer, L. A. 2011. “Women Who Win: Social Backgrounds, Paths to Power, and Political Ambition in Latin American Legislatures.” Politics & Gender 7(1):1–33.Google Scholar
Searles, K., and Smith, G.. 2016. “Who’s the Boss? Setting the Agenda in a Fragmented Media Environment.” International Journal of Communication 10:22.Google Scholar
Small, M. L., Manduca, R. A., and Johnston, W. R.. 2018. “Ethnography, Neighborhood Effects, and the Rising Heterogeneity of Poor Neighborhoods across Cities.” City & Community 17(3):565–589.Google Scholar
Smith, A. 2014. “Female Political Incorporation and the Allocation of Community Development Block Grant Funding.” Politics & Gender 10(3):313–340.Google Scholar
Stone, C. N. 1989. Regime Politics: Governing Atlanta, 1946–1988. Studies in Government and Public Policy. Lawrence, KS: University of Kansas Press.Google Scholar
Tausanovitch, C., and Warshaw, C.. 2014. “Representation in Municipal Government.” American Political Science Review 108(3):605–641.Google Scholar
Trounstine, J. 2006. “Dominant Regimes and the Demise of Urban Democracy.” Journal of Politics 68(4):879–893.Google Scholar
Trounstine, J. 2008. Political Monopolies in American Cities: The Rise and Fall of Bosses and Reformers. Chicago: University of Chicago Press.Google Scholar
Trounstine, J. 2009. “All Politics Is Local: The Reemergence of the Study of City Politics.” Perspectives on Politics 7(3):611–618.Google Scholar
Trounstine, J. 2018. Segregation by Design Local Politics and Inequality in American Cities. New York: Cambridge University Press.Google Scholar
Vogl, T. S. 2014. “Race and the Politics of Close Elections.” Journal of Public Economics 109:101–113.Google Scholar
Ward, A. 2004. “How One Mistake Leads to Another: On the Importance of Verification/Replication.” Political Analysis 12(2):199–200.Google Scholar
Watson, D. J., and Hassett, W. L.. 2004. “Career Paths of City Managers in America’s Largest Council Manager Cities.” Public Administration Review 64(2):192–199.Google Scholar
Williamson, A. 2018. Welcoming New Americans? Chicago: University of Chicago Press.Google Scholar
Williamson, V. 2016. “On the Ethics of Crowdsourced Research.” PS: Political Science & Politics 49(1):77–81.Google Scholar
Supplementary material: Link

Sumner Dataset

Dataset

https://doi.org/10.7910/DVN/LSEEKB
Link
Supplementary material: File

Sumner et al. supplementary material

Sumner et al. supplementary material

Download Sumner et al. supplementary material(File)
File 780.7 KB