Skip to main content

Comparing Three Methods of Measuring Race/Ethnicity

  • Paru R. Shah (a1) and Nicholas R. Davis (a1)

In this research note, we explore compare and contrast three methods for measuring race. We utilize as our baseline, or “true”, measure expert coded racial categories, and to this compare two alternatives. The first is a hybrid Bayesian analysis of racial/ethnic surname lists and population distributions, which allow us to develop a race probability score for each candidate. The second is a novel and innovative crowdsourcing method that allows many contributors to classify the racial identity of candidates. We analyze and discuss the potential benefits, pitfalls, and tradeoffs of each method. We conclude with the implications of these new measures for future election research as well as race and politics scholarship more broadly.

Corresponding author
Address correspondence and reprint requests to: Paru Shah, University of Wisconsin-Milwaukee, WI. E-mail:
Hide All
Adjaye-Gbewonyo, Dzifa, Bednarczyk, Robert A., Davis, Robert L., and Omer, Saad B.. 2014. “Using the Bayesian Improved Surname Geocoding Method (BISG) to create a working classification of race and ethnicity in a diverse managed care population: a validation study.” Health Services Research 49 (1): 268–83.
Alonso, Omar and Baeza-Yates, Ricardo. 2011. “Design and Implementation of Relevance Assessments using Crowdsourcing.” In Advances in Information Retrieval: 33rd European Conference on IR Research, Dublin, Ireland, April 18–21, 2011. Proceedings. Editors: Clough, Paul, Foley, Colum, Gurrin, Cathal, Jones, Gareth, Kraaij, Wessel, Lee, Hyowon, Mudoch, Vanessa. Springer, 153–64.
Alonso, Omar and Mizzaro, Stefano. 2009. Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation. pp. 1516.
Benoit, Kenneth, Conway, Drew, Lauderdale, Benjamin E., Laver, Michael, and Mikhaylov, Slava. 2016. “Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data.” American Political Science Review 110 (2): 278–95.
Berinsky, Adam J., Huber, Gregory A., and Lenz, Gabriel S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon. com's Mechanical Turk.” Political Analysis 20 (3): 351–68.
Bothwell, Robert K., Brigham, John C., and Malpass, Roy S.. 1989. “Cross-racial Identification.” Personality and Social Psychology Bulletin 15 (1): 1925.
Chiroro, Patrick and Valentine, Tim. 1995. “An Investigation of the Contact Hypothesis of the Own-Race Bias in Face Recognition.” The Quarterly Journal of Experimental Psychology 48 (4): 879–94.
Dehon, Hedwige and Brédart, Serge. 2001. “An Other-Race Effect in Age Estimation from Faces.” Perception 30 (9): 1107–14.
Elliott, Marc N., Fremont, Allen, Morrison, Peter A., Pantoja, Philip, and Lurie, Nicole. 2008. “A New Method for Estimating Race/Ethnicity and Associated Disparities Where Administrative Records Lack Self-Reported Race/Ethnicity.” Health Services Research 43 (5 Part 1): 1722–36.
Enos, Ryan D. 2015. “The Effect of Proximity to African-Americans on Latino Vote Choice in the 2008 Presidential Primary in Los Angeles.” (Accessed August 15, 2016).
Furl, Nicholas, Phillips, P. Jonathan, and O'Toole, Alice J.. 2002. “Face Recognition Algorithms and the Other-Race Effect: Computational Mechanisms for a Developmental Contact Hypothesis.” Cognitive Science 26 (6): 797815.
Goldstone, Robert L. 2003. “Do We All Look Alike to Computers?Trends in Cognitive Sciences 7 (2): 5557.
Hanson, Andrew and Hawley, Zackary. 2011. “Do Landlords Discriminate in the Rental Housing Market? Evidence from an Internet Field Experiment in US Cities.” Journal of Urban Economics 70 (2): 99114.
Horton, John J., Rand, David G., and Zeckhauser, Richard J.. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399425.
Hsueh, Pei-Yun, Melville, Prem, and Sindhwani, Vikas. 2009. Data quality from crowdsourcing: a study of annotation selection criteria. In Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing. Association for Computational Linguistics, pp. 2735.
Imai, Kosuke, and Khanna, Kabir. 2016. “Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records.” Political Analysis 24(2): 263–72.
Juenke, Eric Gonzalez and Shah, Paru. 2015. “Not the Usual Story: The Effect of Candidate Supply on Models of Latino Descriptive Representation.” Politics, Groups, and Identities 3(3): 438–53.
Lauderdale, Diane S. and Bert, Kestenbaum. 2000. “Asian American Ethnic Identification by Surname.” Population Research and Policy Review 19 (3): 283300.
Meissner, Christian A. and Brigham, John C.. 2001. “Thirty Years of Investigating the Own-Race Bias in Memory for Faces: A Meta-Analytic Review.” Psychology, Public Policy, and Law 7 (1): 3.
Morrison, Peter A. and Coleman, Charles D.. 2001. “Using First Names to Estimate Racial Proportions in Populations.” Paper Presented at the Population Association of America Annual Meeting, Washington, DC.
Nicoll, Angus, Bassett, Karen, and Ulijaszek, Stanley J.. 1986. “What's in a Name? Accuracy of using Surnames and Forenames in Ascribing Asian Ethnic Identity in English Populations.” Journal of Epidemiology and Community Health 40 (4): 364–68.
Omi, Michael and Winant, Howard. 1986. Racial Formation in the United States: From the 1960s to the 1980s. New York: Routledge.
O'Toole, Alice J., Peterson, Jennifer, and Deffenbacher, Kenneth A.. 1996. “An ‘Other-Race Effect’ for Categorizing Faces by Sex.” Perception 25: 669–76.
Paolacci, Gabriele, Chandler, Jesse, and Ipeirotis, Panagiotis G.. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision making 5 (5): 411–19.
Rhodes, Gillian, Locke, Vance, Ewing, Louise, and Evangelista, Emma. 2009. “Race Coding and the Other-Race Effect in Face Recognition.” Perception 38 (2): 232.
Rhodes, Gillian, Hayward, William G., and Winkler, Christopher. 2006. “Expert Face Coding: Configural and Component Coding of Own-Race and Other-Race Faces.” Psychonomic Bulletin & Review 13 (3): 499505.
Snow, Rion, O'Connor, Brendan, Jurafsky, Daniel, and Ng, Andrew Y.. 2008. Cheap and Fast—But is it Good?: Evaluating Non-Expert Annotations for Natural Language Tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 254–63.
U.S. Census Bureau. 1990. Selected Population Characteristics, 1990 Census of Population and Housing.
U.S. Census Bureau. 2000. Selected Population Characteristics, 2000 Census of Population and Housing.
Word, David L., Coleman, Charles D., Nunziata, Robert, and Kominski, Robert. 2008. “Demographic aspects of surnames from census 2000.” Unpublished Manuscript, download (Accessed August 15, 2016).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Race, Ethnicity, and Politics
  • ISSN: -
  • EISSN: 2056-6085
  • URL: /core/journals/journal-of-race-ethnicity-and-politics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 13
Total number of PDF views: 97 *
Loading metrics...

Abstract views

Total abstract views: 576 *
Loading metrics...

* Views captured on Cambridge Core between 6th March 2017 - 19th March 2018. This data will be updated every 24 hours.