Hostname: page-component-5d59c44645-mhl4m Total loading time: 0 Render date: 2024-03-03T18:51:55.879Z Has data issue: false hasContentIssue false


How Best to Ask the Questions, in One or Two Stages, Across Multiple Racial/Ethnic Groups?1

Published online by Cambridge University Press:  15 April 2011

Salma Shariff-Marco*
Division of Cancer Control and Population Sciences, National Cancer Institute
Nancy Breen
Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute
Hope Landrine
Center for Health Disparities Research, East Carolina University
Bryce B. Reeve
Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute
Nancy Krieger
Harvard School of Public Health, Harvard University
Gilbert C. Gee
School of Public Health, University of California, Los Angeles
David R. Williams
Department of Society, Human Development, and Health, Harvard School of Public Health; Departments of African and African American Studies and of Sociology, Harvard University
Vickie M. Mays
School of Public Health, University of California, Los Angeles and Center for Research, Education, Training, and Strategic Communication on Minority Health Disparities, University of California, Los Angeles
Ninez A. Ponce
School of Public Health, University of California, Los Angeles and UCLA Center for Health Policy and Research
Margarita Alegría
Harvard Medical School, Harvard University
Benmei Liu
Surveillance Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute
Gordon Willis
Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute
Timothy P. Johnson
Survey Research Laboratory, Department of Public Administration and Institute for Health Research and Policy, University of Illinois at Chicago
Salma Shariff-Marco, Cancer Prevention Institute of California, 2201 Walnut Avenue, Suite 300, Fremont, CA 94538. E-mail:


While it is clear that self-reported racial/ethnic discrimination is related to illness, there are challenges in measuring self-reported discrimination or unfair treatment. In the present study, we evaluate the psychometric properties of a self-reported instrument across racial/ethnic groups in a population-based sample, and we test and interpret findings from applying two different widely-used approaches to asking about discrimination and unfair treatment. Even though we found that the subset of items we tested tap into a single underlying concept, we also found that different groups are more likely to report on different aspects of discrimination. Whether race is mentioned in the survey question affects both frequency and mean scores of reports of racial/ethnic discrimination. Our findings suggest caution to researchers when comparing studies that have used different approaches to measure racial/ethnic discrimination and allow us to suggest practical empirical guidelines for measuring and analyzing racial/ethnic discrimination. No less important, we have developed a self-reported measure of recent racial/ethnic discrimination that functions well in a range of different racial/ethnic groups and makes it possible to compare how racial/ethnic discrimination is associated with health disparities among multiple racial/ethnic groups.

Unpacking Racism and its Health Consequences
Copyright © W.E.B. Du Bois Institute for African and African American Research 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)



We would like to thank the NCI Division of Cancer Control and Population Sciences (DCCPS) and the NIH Office of Behavioral Social Sciences Research for co-funding the 2007 field test, and NCI DCCPS for providing staff for this project. We would also like to acknowledge the Cancer Prevention Fellowship Program, Office of Preventive Oncology, National Cancer Institute, for providing financial support for S. Shariff-Marco in her postdoctoral fellowship; Tim McNeel and William Waldron at IMS for their programming support; and Penny Randall-Levy of the Scientific Consulting Group, Inc., for her help with the references. We would also like to acknowledge Drs. David Grant, E. Richard Brown, and David Takeuchi for their participation in this research project. Finally, we would like to acknowledge Drs. William Klein and Rachel Ballard-Barbash for their thoughtful reviews.



Albert, Michelle A., Cozier, Yvette, Ridker, Paul M., Palmer, Julie R., Glynn, Robert J., Rose, Lynda, Halevy, Nitsan, and Rosenberg, Lynn (2010). Perceptions of Race/Ethnic Discrimination in Relation to Mortality Among Black Women: Results From the Black Women's Health Study. Archives of Internal Medicine, 170(10): 896904.Google Scholar
Barnes, Lisa L., de Leon, Carlos F. Mendes, Lewis, Tene T., Bienias, Julia L., Wilson, Robert S., and Evans, Denis A. (2008). Perceived Discrimination and Mortality in a Population-Based Study of Older Adults. American Journal of Public Health, 98(7): 12411247.Google Scholar
Bastos, Joao Luiz, Celeste, Roger Keller, Faerstein, Eduardo, and Barros, Aluisio J. (2010). Racial Discrimination and Health: a Systematic Review of Scales With a Focus on Their Psychometric Properties. Social Science & Medicine, 70(7): 10911099.Google Scholar
Bobo, Lawrence (2001). Racial Attitudes and Relations at the Close of the 20th Century. In Smelser, Neil J., Wilson, William J., and Mitchell, Faith (Eds.), America Becoming: Racial Trends and Their Consequences, Volume 1, pp. 264301. Washington, DC: National Academy Press.Google Scholar
Brown, Tony N. (2001). Measuring Self-Perceived Racial and Ethnic Discrimination in Social Surveys. Sociological Spectrum, 21(3): 377392.Google Scholar
California Health Interview Survey (2009). CHIS 2007 Methodology Series Report 4: Response Rates. Los Angeles, CA: UCLA Center for Health Policy Research.Google Scholar
Cameron, Christopher David Ruiz (1997). How the Garcia Cousins Lost Their Accents: Understanding the Language of Title VII Decisions Approving English-Only Rules As the Product of Racial Dualism, Latino Invisibility, and Legal Indeterminacy. California Law Review, 85: 13471393.Google Scholar
Carney, Dana R., Banaji, Mahzarin R., and Krieger, Nancy (2010). Implicit Measures Reveal Evidence of Personal Discrimination. Self and Identity, 9(2): 162176.Google Scholar
Chae, David H., Takeuchi, David T., Barbeau, Elizabeth M., Bennett, Gary G., Lindsey, Jane, and Krieger, Nancy (2008). Unfair Treatment, Racial/Ethnic Discrimination, Ethnic Identification, and Smoking Among Asian Americans in the National Latino and Asian American Study. American Journal of Public Health, 98(3): 485492.Google Scholar
Clark, Rodney, Anderson, Norman B., Clark, Vernessa R., and Williams, David R. (1999). Racism As a Stressor for African Americans. A Biopsychosocial Model. American Psychologist, 54(10): 805816.Google Scholar
Desmond, Matthew and Emirbayer, Mustafa (2009). What Is Racial Domination? Du Bois Review: Social Science Research on Race, 6(2): 335355.Google Scholar
Epel, Elissa S., Blackburn, Elizabeth H., Lin, Jue, Dhabhar, Firdaus S., Adler, Nancy E., Morrow, Jason D., and Cawthon, Richard M. (2004). Accelerated Telomere Shortening in Response to Life Stress. Proceedings of the National Academy of Sciences of the United States of America, 101(49): 1731217315.Google Scholar
Essed, Philomena (1990). Understanding Everyday Racism: an Interdisciplinary Theory. Newbury Park, CA: Sage Publications, Inc.Google Scholar
Feagin, Joe R. (2006). Systematic Racism. New York: CRC Press.Google Scholar
Feagin, Joe R. and O'Brien, Eileen (2003). White Men on Race: Power, Privilege, and the Shaping of Cultural Consciousness. Boston, MA: Beacon Press.Google Scholar
Gee, Gilbert C., Ro, Annie, Shariff-Marco, Salma, and Chae, David (2009). Racial Discrimination and Health Among Asian Americans: Evidence, Assessment, and Directions for Future Research. Epidemiologic Reviews, 31: 130151.Google Scholar
Geronimus, Arline T. (2001). Understanding and Eliminating Racial Inequalities in Women's Health in the United States: the Role of the Weathering Conceptual Framework. Journal of the American Medical Women's Association 56(4): 133–136, 149–150.Google Scholar
Geronimus, Arline T., Hicken, Margaret, Keene, Danya, and Bound, John (2006). “Weathering” and Age Patterns of Allostatic Load Scores Among Blacks and Whites in the United States. American Journal of Public Health, 96(5): 826833.Google Scholar
Judkins, David, Krenzke, Tom, Piesse, Andrea, Fan, Zizhong, and Haung, Wen-Chau (2007). Preservation of Skip Patterns and Covariate Structure Through Semi-Parametric Whole Questionnaire Imputation Proceedings of the Joint Statistical Meetings of the American Statistical Association, pp. 32113288.Google Scholar
Judkins, David, Piesse, Andrea, and Krenzke, Tom (2008). Multiple Semi-Parametric Imputation. Proceedings of the Joint Statistical Meetings of the American Statistical Association, pp. 4858.Google Scholar
Kessler, Ronald C., Mickelson, Kristin D., and Williams, David R. (1999). The Prevalence, Distribution, and Mental Health Correlates of Perceived Discrimination in the United States. Journal of Health and Social Behavior, 40(3): 208230.Google Scholar
Kluegel, James R. and Bobo, Lawrence D. (2001). Perceived Group Discrimination and Policy Attitudes: the Sources and Consequences of the Race and Gender Gaps. In O'Connor, Alice, Tilly, Chris, and Bobo, Lawrence D. (Eds.), Urban Inequality: Evidence From Four Cities, pp. 163214. New York: Russell Sage Foundation.Google Scholar
Kressin, Nancy R., Raymond, Kristal L., and Manze, Meredith (2008). Perceptions of Race/Ethnicity-Based Discrimination: a Review of Measures and Evaluation of Their Usefulness for the Health Care Setting. Journal of Health Care for the Poor and Underserved, 19(3): 697730.Google Scholar
Krieger, Nancy (2000). Discrimination and Health. In Berkman, Lisa F. and Kawachi, Ichiro (Eds.), Social Epidemiology, pp. 3675. New York: Oxford University Press.Google Scholar
Krieger, Nancy (2010). The Science and Epidemiology of Racism and Health: Racial/Ethnic Categories, Biological Expressions of Racism, and the Embodiment of Inequality—an Ecosocial Perspective. In Whitmarsh, Ian and Jones, David S. (Eds.), What's the Use of Race? Modern Governance and the Biology of Difference, pp. 225255. Cambridge, MA: The MIT Press.Google Scholar
Krieger, Nancy, Carney, Dana, Lancaster, Katie, Waterman, Pamela D., Kosheleva, Anna, and Banaji, Mahzarin (2010). Combining Explicit and Implicit Measures of Racial Discrimination in Health Research. American Journal of Public Health, 100(8): 14851492.Google Scholar
Krieger, Nancy and Sidney, Stephen (1996). Racial Discrimination and Blood Pressure: the CARDIA Study of Young Black and White Adults. American Journal of Public Health, 86(10): 13701378.Google Scholar
Krieger, Nancy, Smith, Kevin, Naishadham, Deepa, Hartman, Cathy, and Barbeau, Elizabeth M. (2005). Experiences of Discrimination: Validity and Reliability of a Self-Report Measure for Population Health Research on Racism and Health. Social Science & Medicine, 61(7): 15761596.Google Scholar
Krysan, Maria and Couper, Mick P. (2003). Race in the Live and the Virtual Interview: Racial Deference, Social Desirability, and Activation Effects in Attitude Surveys. Social Psychology Quarterly, 66(4): 364383.Google Scholar
Landrine, Hope, Klonoff, Elizabeth A., Corral, Irma, Fernandez, Senaida, and Roesch, Scott (2006). Conceptualizing and Measuring Ethnic Discrimination in Health Research. Journal of Behavioral Medicine, 29(1): 7994.Google Scholar
Lee, Sunghee, Satter, Delight E., and Ponce, Ninez A. (2009). Effect of Race and Ethnicity Classification on Survey Estimates: Anomaly of the Weighted Totals of American Indians and Alaska Natives. American Indian and Alaska Native Mental Health Research, 16(3): 115.Google Scholar
Lewis, Tene T., Aiello, Allison E., Leurgans, Sue, Kelly, Jeremiah, and Barnes, Lisa L. (2010). Self-Reported Experiences of Everyday Discrimination Are Associated With Elevated C-Reactive Protein Levels in Older African-American Adults. Brain, Behavior, and Immunity, 24(3): 438443.Google Scholar
Lewis, Tene T., Everson-Rose, Susan A., Powell, Lynda H., Matthews, Karen A., Brown, Charlotte, Karavolos, Kelly, Sutton-Tyrrell, Kim, Jacobs, Elizabeth, and Wesley, Deidre (2006). Chronic Exposure to Everyday Discrimination and Coronary Artery Calcification in African-American Women: the SWAN Heart Study. Psychosomatic Medicine, 68(3): 362368.Google Scholar
Liang, Christopher T., Li, Lisa C., and Kim, Bryan S. (2004). The Asian American Racism-Related Stress Inventory: Development, Factor Analysis, Reliability, and Validity. Journal of Counseling Psychology, 51(1): 103114.Google Scholar
Massey, Douglas S. and Lundy, Garvey (2001). Use of Black English and Racial Discrimination in Urban Housing Markets: New Methods and Findings. Urban Affairs Review, 36(4): 452469.Google Scholar
Mays, Vickie M., Cochran, Susan D., and Barnes, Namdi W. (2007). Race, Race-Based Discrimination, and Health Outcomes Among African Americans. Annual Review of Psychology, 58: 201225.Google Scholar
Mays, Vickie M., Ponce, Ninez A., Washington, Donna L., and Cochran, Susan D. (2003). Classification of Race and Ethnicity: Implications for Public Health. Annual Review of Public Health, 24: 83110.Google Scholar
Muthen, Linda K. and Muthen, Bengt O. (2007). Mplus (Version 4.21) Software. Los Angeles, CA: Muthen & Muthen.Google Scholar
National Research Council (2004). Measuring Racial Discrimination. Panel on Methods for Assessing Discrimination. Washington, DC: The National Academies Press.Google Scholar
Nunnally, Jum C. and Bernstein, Ira H. (1994). Psychometric Theory, 3ed.New York: McGraw Hill Co.Google Scholar
Office of Management and Budget (1997). Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity, Office of Management and Budget Directive 15. U.S. Census Bureau. ⟨⟩ (accessed October 3, 2010).Google Scholar
Paradies, Yin (2006). A Systematic Review of Empirical Research on Self-Reported Racism and Health. International Journal of Epidemiology, 35(4): 888901.Google Scholar
Pascoe, Elizabeth A. and Smart Richman, Laura (2009). Perceived Discrimination and Health: a Meta-Analytic Review. Psychological Bulletin, 135(4): 531554.Google Scholar
Piesse, Andrea, Judkins, David R., and Fan, Zizhong (2005). Item Imputation Made Easy. Proceedings of the Survey Research Methods Section of the American Statistical Association, pp. 25402546.Google Scholar
Pincus, Fred L. (2003). Reverse Discrimination: Dismantling the Myth. Boulder, CO: Lynne Rienner Publishers.Google Scholar
Reeve, Bryce B. and Fayers, Peter (2005). Applying Item Response Theory Modelling for Evaluating Questionnaire Item and Scale Properties. In Fayers, Peter and Hays, Ron (Eds.), Assessing Quality of Life in Clinical Trials: Methods and Practice, 2ed., pp. 5573. New York: Oxford University Press.Google Scholar
Reeve, Bryce B., Hays, Ron D., Bjorner, Jakob B., Cook, Karon F., Crane, Paul K., Teresi, Jeanne A., Thissen, David, Revicki, Dennis A., Weiss, David J., Hambleton, Ronald K., Liu, Honghu, Gershon, Richard, Reise, Steven P., Lai, Jin-shei, and Cella, David (2007). Psychometric Evaluation and Calibration of Health-Related Quality of Life Item Banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1): S22S31.Google Scholar
Reeve, Bryce B., Willis, Gordon, Shariff-Marco, Salma N., Breen, Nancy, Williams, David R., Gee, Gilbert C., Alegria, Margarita, Takeuchi, David T., Kudela, Martha S., and Levin, Kerry Y. (Forthcoming). Comparing Cognitive Interviewing and Psychometric Methods to Evaluate a Racial/Ethnic Discrimination Scale. Field Methods.Google Scholar
Research Triangle Institute (2008). SUDAAN (Release 10.0). Research Triangle Park, NC: Research Triangle Institute.Google Scholar
Roussos, Louis and Stout, William (1996). A Multidimensionality-Based DIF Analysis Paradigm. Applied Psychological Measurement, 20(4): 355371.Google Scholar
Seeman, Teresa E., Singer, Burton H., Ryff, Carol D., Love, Gayle Dienberg, and Levy-Storms, Lene (2002). Social Relationships, Gender, and Allostatic Load Across Two Age Cohorts. Psychosomatic Medicine, 64(3): 395406.Google Scholar
Shariff-Marco, Salma, Gee, Gilbert C., Breen, Nancy, Willis, Gordon, Reeve, Bryce B., Grant, David, Ponce, Ninez A., Krieger, Nancy, Landrine, Hope, Williams, David R., Alegria, Margarita, Mays, Vickie M., Johnson, Timothy P., and Brown, E. Richard (2009). A Mixed-Methods Approach to Developing a Self-Reported Racial/Ethnic Discrimination Measure for Use in Multiethnic Health Surveys. Ethnicity & Disease, 19(4): 447453.Google Scholar
Spencer, Michael S. and Chen, Juan (2004). Effect of Discrimination on Mental Health Service Utilization Among Chinese Americans. American Journal of Public Health, 94(5): 809814.Google Scholar
Swan, Judith, Breen, Nancy, Burhansstipanov, Linda, Satter, Delight E., Davis, William W., McNeel, Timothy, and Snipp, C. Matthew (2006). Cancer Screening and Risk Factor Rates Among American Indians. American Journal of Public Health, 96(2): 340350.Google Scholar
Taylor, Teletia R., Williams, Carla D., Makambi, Kepher H., Mouton, Charles, Harrell, Jules P., Cozier, Yvette, Palmer, Julie R., Rosenberg, Lynn, and Adams-Campbell, Lucile L. (2007). Racial Discrimination and Breast Cancer Incidence in US Black Women: the Black Women's Health Study. American Journal of Epidemiology, 166(1): 4654.Google Scholar
Teresi, Jeanne A. and Fleishman, John A. (2007). Differential Item Functioning and Health Assessment. Quality of Life Research, 16(Suppl 1): 3342.Google Scholar
Thissen, David L. (2001). IRTLRDIF (v. 2.0b): Software for the Computation of the Statistics Involved in Item Response Theory Likelihood-Ratio Tests for Differential Item Functioning. Chapel Hill, NC: Thurstone Psychometric Laboratory.Google Scholar
Thissen, David, Steinberg, Lynne, and Wainer, Howard (1993). Detection of Differential Item Functioning Using the Parameters of Item Response Models. In Holland, Paul W. and Wainer, Howard (Eds.), Differential Item Functioning, pp. 67113. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Tomfohr, Lianne, Cooper, Denise C., Mills, Paul J., Nelesen, Richard A., and Dimsdale, Joel E. (2010). Everyday Discrimination and Nocturnal Blood Pressure Dipping in Black and White Americans. Psychosomatic Medicine, 72(3): 266272.Google Scholar
Tuan, Mia (1998). Forever Foreigners or Honorary Whites? The Asian Ethnic Experience Today. New Brunswick, NJ: Rutgers University Press.Google Scholar
Turner, Margery A. and Skidmore, Felicity (Eds.) (1999). Mortgage Lending Discrimination: A Review of Existing Evidence. Washington, DC: The Urban Institute.Google Scholar
Wildsmith, Elizabeth M. (2002). Testing the Weathering Hypothesis Among Mexican-Origin Women. Ethnicity & Disease, 12(4): 470479.Google Scholar
Williams, David R. and Mohammed, Selina A. (2009). Discrimination and Racial Disparities in Health: Evidence and Needed Research. Journal of Behavioral Medicine, 32(1): 2047.Google Scholar
Williams, David R., Mohammed, Selina A., Leavell, Jacinta, and Collins, Chiquita (2010). Race, Socioeconomic Status, and Health: Complexities, Ongoing Challenges, and Research Opportunities. Annals of the New York Academy of Sciences, 1186: 69101.Google Scholar
Williams, David R. and Neighbors, Harold (2001). Racism, Discrimination and Hypertension: Evidence and Needed Research. Ethnicity & Disease, 11(4): 800816.Google Scholar
Williams, David R., Yu, Yan, Jackson, James S., and Anderson, Norman B. (1997). Racial Differences in Physical and Mental Health: Socio-Economic Status, Stress and Discrimination. Journal of Health Psychology, 2(3): 335351.Google Scholar
Yoo, Hyung Chol, Gee, Gilbert C., Lowthrop, Craig K., and Robertson, Joanne (2010). Self-Reported Racial Discrimination and Substance Use Among Asian Americans in Arizona. Journal of Immigrant and Minority Health, 12(5): 683690.Google Scholar
Yoo, Hyung Chol, Gee, Gilbert C., and Takeuchi, David (2009). Discrimination and Health Among Asian American Immigrants: Disentangling Racial From Language Discrimination. Social Science & Medicine, 68(4): 726732.Google Scholar