Skip to main content Accessibility help
Hostname: page-component-66d7dfc8f5-zf4m4 Total loading time: 0.362 Render date: 2023-02-08T08:22:49.465Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

Comparing Three Methods of Measuring Race/Ethnicity

Published online by Cambridge University Press:  06 March 2017

Paru R. Shah*
University of Wisconsin–Milwaukee
Nicholas R. Davis
University of Wisconsin–Milwaukee
Address correspondence and reprint requests to: Paru Shah, University of Wisconsin-Milwaukee, WI. E-mail:
Get access


In this research note, we explore compare and contrast three methods for measuring race. We utilize as our baseline, or “true”, measure expert coded racial categories, and to this compare two alternatives. The first is a hybrid Bayesian analysis of racial/ethnic surname lists and population distributions, which allow us to develop a race probability score for each candidate. The second is a novel and innovative crowdsourcing method that allows many contributors to classify the racial identity of candidates. We analyze and discuss the potential benefits, pitfalls, and tradeoffs of each method. We conclude with the implications of these new measures for future election research as well as race and politics scholarship more broadly.

Research Article
Copyright © The Race, Ethnicity, and Politics Section of the American Political Science Association 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)



Adjaye-Gbewonyo, Dzifa, Bednarczyk, Robert A., Davis, Robert L., and Omer, Saad B.. 2014. “Using the Bayesian Improved Surname Geocoding Method (BISG) to create a working classification of race and ethnicity in a diverse managed care population: a validation study.” Health Services Research 49 (1): 268–83.CrossRefGoogle Scholar
Alonso, Omar and Baeza-Yates, Ricardo. 2011. “Design and Implementation of Relevance Assessments using Crowdsourcing.” In Advances in Information Retrieval: 33rd European Conference on IR Research, Dublin, Ireland, April 18–21, 2011. Proceedings. Editors: Clough, Paul, Foley, Colum, Gurrin, Cathal, Jones, Gareth, Kraaij, Wessel, Lee, Hyowon, Mudoch, Vanessa. Springer, 153–64.CrossRefGoogle Scholar
Alonso, Omar and Mizzaro, Stefano. 2009. Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation. pp. 1516.Google Scholar
Benoit, Kenneth, Conway, Drew, Lauderdale, Benjamin E., Laver, Michael, and Mikhaylov, Slava. 2016. “Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data.” American Political Science Review 110 (2): 278–95.CrossRefGoogle Scholar
Berinsky, Adam J., Huber, Gregory A., and Lenz, Gabriel S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon. com's Mechanical Turk.” Political Analysis 20 (3): 351–68.CrossRefGoogle Scholar
Bothwell, Robert K., Brigham, John C., and Malpass, Roy S.. 1989. “Cross-racial Identification.” Personality and Social Psychology Bulletin 15 (1): 1925.CrossRefGoogle Scholar
Chiroro, Patrick and Valentine, Tim. 1995. “An Investigation of the Contact Hypothesis of the Own-Race Bias in Face Recognition.” The Quarterly Journal of Experimental Psychology 48 (4): 879–94.CrossRefGoogle Scholar
Dehon, Hedwige and Brédart, Serge. 2001. “An Other-Race Effect in Age Estimation from Faces.” Perception 30 (9): 1107–14.CrossRefGoogle ScholarPubMed
Elliott, Marc N., Fremont, Allen, Morrison, Peter A., Pantoja, Philip, and Lurie, Nicole. 2008. “A New Method for Estimating Race/Ethnicity and Associated Disparities Where Administrative Records Lack Self-Reported Race/Ethnicity.” Health Services Research 43 (5 Part 1): 1722–36.CrossRefGoogle Scholar
Enos, Ryan D. 2015. “The Effect of Proximity to African-Americans on Latino Vote Choice in the 2008 Presidential Primary in Los Angeles.” (Accessed August 15, 2016).Google Scholar
Furl, Nicholas, Phillips, P. Jonathan, and O'Toole, Alice J.. 2002. “Face Recognition Algorithms and the Other-Race Effect: Computational Mechanisms for a Developmental Contact Hypothesis.” Cognitive Science 26 (6): 797815.CrossRefGoogle Scholar
Goldstone, Robert L. 2003. “Do We All Look Alike to Computers?Trends in Cognitive Sciences 7 (2): 5557.CrossRefGoogle Scholar
Hanson, Andrew and Hawley, Zackary. 2011. “Do Landlords Discriminate in the Rental Housing Market? Evidence from an Internet Field Experiment in US Cities.” Journal of Urban Economics 70 (2): 99114.CrossRefGoogle Scholar
Horton, John J., Rand, David G., and Zeckhauser, Richard J.. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399425.CrossRefGoogle Scholar
Hsueh, Pei-Yun, Melville, Prem, and Sindhwani, Vikas. 2009. Data quality from crowdsourcing: a study of annotation selection criteria. In Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing. Association for Computational Linguistics, pp. 2735.Google Scholar
Imai, Kosuke, and Khanna, Kabir. 2016. “Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records.” Political Analysis 24(2): 263–72.CrossRefGoogle Scholar
Juenke, Eric Gonzalez and Shah, Paru. 2015. “Not the Usual Story: The Effect of Candidate Supply on Models of Latino Descriptive Representation.” Politics, Groups, and Identities 3(3): 438–53.CrossRefGoogle Scholar
Lauderdale, Diane S. and Bert, Kestenbaum. 2000. “Asian American Ethnic Identification by Surname.” Population Research and Policy Review 19 (3): 283300.CrossRefGoogle Scholar
Meissner, Christian A. and Brigham, John C.. 2001. “Thirty Years of Investigating the Own-Race Bias in Memory for Faces: A Meta-Analytic Review.” Psychology, Public Policy, and Law 7 (1): 3.CrossRefGoogle Scholar
Morrison, Peter A. and Coleman, Charles D.. 2001. “Using First Names to Estimate Racial Proportions in Populations.” Paper Presented at the Population Association of America Annual Meeting, Washington, DC.Google Scholar
Nicoll, Angus, Bassett, Karen, and Ulijaszek, Stanley J.. 1986. “What's in a Name? Accuracy of using Surnames and Forenames in Ascribing Asian Ethnic Identity in English Populations.” Journal of Epidemiology and Community Health 40 (4): 364–68.CrossRefGoogle Scholar
Omi, Michael and Winant, Howard. 1986. Racial Formation in the United States: From the 1960s to the 1980s. New York: Routledge.Google Scholar
O'Toole, Alice J., Peterson, Jennifer, and Deffenbacher, Kenneth A.. 1996. “An ‘Other-Race Effect’ for Categorizing Faces by Sex.” Perception 25: 669–76.CrossRefGoogle Scholar
Paolacci, Gabriele, Chandler, Jesse, and Ipeirotis, Panagiotis G.. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision making 5 (5): 411–19.Google Scholar
Rhodes, Gillian, Locke, Vance, Ewing, Louise, and Evangelista, Emma. 2009. “Race Coding and the Other-Race Effect in Face Recognition.” Perception 38 (2): 232.CrossRefGoogle Scholar
Rhodes, Gillian, Hayward, William G., and Winkler, Christopher. 2006. “Expert Face Coding: Configural and Component Coding of Own-Race and Other-Race Faces.” Psychonomic Bulletin & Review 13 (3): 499505.CrossRefGoogle ScholarPubMed
Snow, Rion, O'Connor, Brendan, Jurafsky, Daniel, and Ng, Andrew Y.. 2008. Cheap and Fast—But is it Good?: Evaluating Non-Expert Annotations for Natural Language Tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 254–63.Google Scholar
U.S. Census Bureau. 1990. Selected Population Characteristics, 1990 Census of Population and Housing. Scholar
U.S. Census Bureau. 2000. Selected Population Characteristics, 2000 Census of Population and Housing. Scholar
Word, David L., Coleman, Charles D., Nunziata, Robert, and Kominski, Robert. 2008. “Demographic aspects of surnames from census 2000.” Unpublished Manuscript, download (Accessed August 15, 2016).Google Scholar
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the or variations. ‘’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Comparing Three Methods of Measuring Race/Ethnicity
Available formats

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Comparing Three Methods of Measuring Race/Ethnicity
Available formats

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Comparing Three Methods of Measuring Race/Ethnicity
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *