Skip to main content

Comparing Three Methods of Measuring Race/Ethnicity

  • Paru R. Shah (a1) and Nicholas R. Davis (a1)

In this research note, we explore compare and contrast three methods for measuring race. We utilize as our baseline, or “true”, measure expert coded racial categories, and to this compare two alternatives. The first is a hybrid Bayesian analysis of racial/ethnic surname lists and population distributions, which allow us to develop a race probability score for each candidate. The second is a novel and innovative crowdsourcing method that allows many contributors to classify the racial identity of candidates. We analyze and discuss the potential benefits, pitfalls, and tradeoffs of each method. We conclude with the implications of these new measures for future election research as well as race and politics scholarship more broadly.

Corresponding author
Address correspondence and reprint requests to: Paru Shah, University of Wisconsin-Milwaukee, WI. E-mail:
Hide All
Adjaye-Gbewonyo Dzifa, Bednarczyk Robert A., Davis Robert L., and Omer Saad B.. 2014. “Using the Bayesian Improved Surname Geocoding Method (BISG) to create a working classification of race and ethnicity in a diverse managed care population: a validation study.” Health Services Research 49 (1): 268–83.
Alonso Omar and Baeza-Yates Ricardo. 2011. “Design and Implementation of Relevance Assessments using Crowdsourcing.” In Advances in Information Retrieval: 33rd European Conference on IR Research, Dublin, Ireland, April 18–21, 2011. Proceedings. Editors: Clough Paul, Foley Colum, Gurrin Cathal, Jones Gareth, Kraaij Wessel, Lee Hyowon, Mudoch Vanessa. Springer, 153–64.
Alonso Omar and Mizzaro Stefano. 2009. Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation. pp. 1516.
Benoit Kenneth, Conway Drew, Lauderdale Benjamin E., Laver Michael, and Mikhaylov Slava. 2016. “Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data.” American Political Science Review 110 (2): 278–95.
Berinsky Adam J., Huber Gregory A., and Lenz Gabriel S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon. com's Mechanical Turk.” Political Analysis 20 (3): 351–68.
Bothwell Robert K., Brigham John C., and Malpass Roy S.. 1989. “Cross-racial Identification.” Personality and Social Psychology Bulletin 15 (1): 1925.
Chiroro Patrick and Valentine Tim. 1995. “An Investigation of the Contact Hypothesis of the Own-Race Bias in Face Recognition.” The Quarterly Journal of Experimental Psychology 48 (4): 879–94.
Dehon Hedwige and Brédart Serge. 2001. “An Other-Race Effect in Age Estimation from Faces.” Perception 30 (9): 1107–14.
Elliott Marc N., Fremont Allen, Morrison Peter A., Pantoja Philip, and Lurie Nicole. 2008. “A New Method for Estimating Race/Ethnicity and Associated Disparities Where Administrative Records Lack Self-Reported Race/Ethnicity.” Health Services Research 43 (5 Part 1): 1722–36.
Enos Ryan D. 2015. “The Effect of Proximity to African-Americans on Latino Vote Choice in the 2008 Presidential Primary in Los Angeles.” (Accessed August 15, 2016).
Furl Nicholas, Phillips P. Jonathan, and O'Toole Alice J.. 2002. “Face Recognition Algorithms and the Other-Race Effect: Computational Mechanisms for a Developmental Contact Hypothesis.” Cognitive Science 26 (6): 797815.
Goldstone Robert L. 2003. “Do We All Look Alike to Computers?Trends in Cognitive Sciences 7 (2): 5557.
Hanson Andrew and Hawley Zackary. 2011. “Do Landlords Discriminate in the Rental Housing Market? Evidence from an Internet Field Experiment in US Cities.” Journal of Urban Economics 70 (2): 99114.
Horton John J., Rand David G., and Zeckhauser Richard J.. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399425.
Hsueh Pei-Yun, Melville Prem, and Sindhwani Vikas. 2009. Data quality from crowdsourcing: a study of annotation selection criteria. In Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing. Association for Computational Linguistics, pp. 2735.
Imai Kosuke, and Khanna Kabir. 2016. “Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records.” Political Analysis 24(2): 263–72.
Juenke Eric Gonzalez and Shah Paru. 2015. “Not the Usual Story: The Effect of Candidate Supply on Models of Latino Descriptive Representation.” Politics, Groups, and Identities 3(3): 438–53.
Lauderdale Diane S. and Bert Kestenbaum. 2000. “Asian American Ethnic Identification by Surname.” Population Research and Policy Review 19 (3): 283300.
Meissner Christian A. and Brigham John C.. 2001. “Thirty Years of Investigating the Own-Race Bias in Memory for Faces: A Meta-Analytic Review.” Psychology, Public Policy, and Law 7 (1): 3.
Morrison Peter A. and Coleman Charles D.. 2001. “Using First Names to Estimate Racial Proportions in Populations.” Paper Presented at the Population Association of America Annual Meeting, Washington, DC.
Nicoll Angus, Bassett Karen, and Ulijaszek Stanley J.. 1986. “What's in a Name? Accuracy of using Surnames and Forenames in Ascribing Asian Ethnic Identity in English Populations.” Journal of Epidemiology and Community Health 40 (4): 364–68.
Omi Michael and Winant Howard. 1986. Racial Formation in the United States: From the 1960s to the 1980s. New York: Routledge.
O'Toole Alice J., Peterson Jennifer, and Deffenbacher Kenneth A.. 1996. “An ‘Other-Race Effect’ for Categorizing Faces by Sex.” Perception 25: 669–76.
Paolacci Gabriele, Chandler Jesse, and Ipeirotis Panagiotis G.. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision making 5 (5): 411–19.
Rhodes Gillian, Locke Vance, Ewing Louise, and Evangelista Emma. 2009. “Race Coding and the Other-Race Effect in Face Recognition.” Perception 38 (2): 232.
Rhodes Gillian, Hayward William G., and Winkler Christopher. 2006. “Expert Face Coding: Configural and Component Coding of Own-Race and Other-Race Faces.” Psychonomic Bulletin & Review 13 (3): 499505.
Snow Rion, O'Connor Brendan, Jurafsky Daniel, and Ng Andrew Y.. 2008. Cheap and Fast—But is it Good?: Evaluating Non-Expert Annotations for Natural Language Tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 254–63.
U.S. Census Bureau. 1990. Selected Population Characteristics, 1990 Census of Population and Housing.
U.S. Census Bureau. 2000. Selected Population Characteristics, 2000 Census of Population and Housing.
Word David L., Coleman Charles D., Nunziata Robert, and Kominski Robert. 2008. “Demographic aspects of surnames from census 2000.” Unpublished Manuscript, download (Accessed August 15, 2016).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Race, Ethnicity, and Politics
  • ISSN: -
  • EISSN: 2056-6085
  • URL: /core/journals/journal-of-race-ethnicity-and-politics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 12
Total number of PDF views: 79 *
Loading metrics...

Abstract views

Total abstract views: 478 *
Loading metrics...

* Views captured on Cambridge Core between 6th March 2017 - 14th December 2017. This data will be updated every 24 hours.