Skip to main content
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 3
  • Cited by
    This chapter has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Edwards, Brad Maitland, Aaron and Connor, Sue 2017. Total Survey Error in Practice. p. 253.

    Erdman, Chandra and Bates, Nancy 2017. The Low Response Score (LRS). Public Opinion Quarterly, Vol. 81, Issue. 1, p. 144.

    Eidson, Jillian L. Roman, Caterina G. and Cahill, Meagan 2017. Successes and Challenges in Recruiting and Retaining Gang Members in Longitudinal Research. Youth Violence and Juvenile Justice, Vol. 15, Issue. 4, p. 396.

  • Print publication year: 2014
  • Online publication date: September 2014

5 - A review of quality issues associated with studying hard-to-survey populations



Some populations/groups are defined as hard to survey (H2S) in the research literature because of difficulty sampling the population, accessing the population, contacting members of the population, persuading them to participate, and interviewing group members (see Tourangeau, Chapter 1 in this volume). Reviewing the literature on populations referred to as “hard to survey” indicates the types of studies undertaken with these populations, the survey error challenges, and the methodological solutions used by researchers who study such populations. The H2S literature, however, does have limitations and research gaps exist.

The problems that complicate obtaining high-quality data from H2S populations are diverse and often interrelated. This review uses a quality assurance and quality control framework to discuss the total survey error associated with H2S studies. The discussion is based on an extensive systematic review of the quantitative and qualitative literature related to H2S, as well as our own experiences surveying these populations. (For a discussion and definition of kinds of H2S populations and the characteristics that earn them the H2S label, see Chapter 1 in this volume.)

Recommend this book

Email your librarian or administrator to recommend adding this book to your organisation's collection.

Hard-to-Survey Populations
  • Online ISBN: 9781139381635
  • Book DOI:
Please enter your name
Please enter a valid email address
Who would you like to send this to *
Agustín, L. M. (2007). Questioning solidarity: outreach with migrants who sell sex. Sexualities, 10(4), 519–34.
Andresen, E. M., Diehr, P. H., & Luke, D. A. (2004). Public health surveillance of low-frequency populations. Annual Review of Public Health, 25(1), 25–52.
Armstrong, B. (2003). The young men’s clinic: addressing men’s reproductive health and responsibilities. Perspectives on Sexual and Reproductive Health, 35(5), 220–25.
Barkan, S. E., Melnick, S. L., Preston-Martin, S., Weber, K., Kalish, L. A., Miotti, P., et al. (1998). The women’s interagency HIV study. Epidemiology, 9(2), 117–25.
Bertrand, J. T., Seiber, E., & Escudero, G. (2001). Contraceptive dynamics in Guatemala: 1978–1998. International Family Planning Perspectives, 27(3), 112–18,136.
Bethlehem, J., Cobben, F., & Schouten, B. (2011). Handbook of Nonresponse in Household Surveys. Hoboken, NJ: John Wiley & Sons.
Biemer, P., & Lyberg, L. (2003). Introduction to Survey Quality. Hoboken, NJ: John Wiley & Sons.
Blair, J., & Czaja, R. (1982). Locating a special population using random digit dialing. Public Opinion Quarterly, 46(4), 585.
Boccia, D., Guthmann, J.-P., Klovstad, H., Hamid, N., Tatay, M., Ciglenecki, I., et al. (2006). High mortality associated with an outbreak of Hepatitis E among displaced persons in Darfur, Sudan. Clinical Infectious Diseases, 42(12), 1679–84.
Bodenmann, P., Vaucher, P., Wolff, H., Favrat, B., de Tribolet, F., Masserey, E., et al. (2009). Screening for latent tuberculosis infection among undocumented immigrants in Swiss healthcare centres: a descriptive exploratory study. BMC Infectious Diseases, 9, 1–8.
Bouchard, F. (2007). A capture–recapture model to estimate the size of criminal populations and the risks of detection in a marijuana cultivation industry. Journal of Quantitative Criminology, 23(3), 221–41.
Bowen, D. J., & Boehmer, U. (2007). The lack of cancer surveillance data on sexual minorities and strategies for change. Cancer Causes & Control, 18(4), 343–49.
Cáceres, C. F., & Rosasco, A. M. (1999). The margin has many sides: diversity among gay and homosexually active men in Lima. Culture, Health & Sexuality, 1(3), 261–75.
Callan, L. B. (1971). Adapting the windshield survey model to community health education. HSMHA Health Reports, 86(3), 202–03.
Canales, S., Ganz, P. A., & Coscarelli, C. A. (1995). Translation and validation of a quality of life instrument for Hispanic American cancer patients: methodological considerations. Quality of Life Research, 4(1), 3–11.
Canvin, K., Bartlett, A., & Pinfold, V. (2005). Acceptability of compulsory powers in the community: the ethical considerations of mental health service users on supervised discharge and guardianship. Journal of Medical Ethics, 31(8), 457–62.
Chaturvedi, N., & McKeigue, P. M. (1994). Methods for epidemiological surveys of ethnic minority groups. Journal of Epidemiological & Community Health, 48, 107–11.
Chavez, E. L., Edwards, R., & Oetting, E. R. (1989). Mexican American and white American school dropouts’ drug use, health status, and involvement in violence. Public Health Reports, 104(6), 594–604.
Chavez, L. R., Hubbell, F. A., Mishra, S. I., & Valdez, R. B. (1997). Undocumented Latina immigrants in Orange County, California: a comparative analysis. International Migration Review, 31(1), 88–107.
Cherry, D. J., & Rost, K. (2009). Alcohol use, comorbidities, and receptivity to treatment in Hispanic farmworkers in primary care. Journal of Health Care for the Poor and Underserved, 20(4), 1095–110.
Coady, M. H., Weiss, L., Galea, S., Ompad, D. C., Glidden, K., & Vlahov, D. (2007). Rapid vaccine distribution in nontraditional settings: lessons learned from project VIVA. Journal of Community Health Nursing, 24(2), 79–85.
Coe, C., Gibson, A., Spencer, N., & Stuttaford, M. (2008). Sure start: voices of the hard-to-reach. Child: Care, Health and Development, 34(4), 447–53.
Coleman, S. M., Rajabiun, S., Cabral, H. J., Bradford, J. B., & Tobias, C. R. (2009). Sexual risk behavior and behavior change among persons newly diagnosed with HIV: the impact of targeted outreach interventions among hard-to-reach populations. AIDS Patient Care & STDs, 23(8), 639–45.
Correa-Velez, I., & Gifford, S. M. (2007). When the right to be counted doesn’t count: the politics and challenges of researching the health of asylum seekers. Critical Public Health, 17(3), 273–81.
Creswell, J. W. (2007). Qualitative inquiry & research design: choosing among five approaches. Thousand Oaks, CA: Sage Publications.
Davis, R. E., Couper, M. P., Janz, N. K., Caldwell, C. H., & Resnicow, K. (2010). Interviewer effects in public health surveys. Health Education Research, 25(1), 14–26.
de Leeuw, E. D. (2008). Choosing the method of data collection. In de Leeuw, E. D., Hox, J. J., & Dillman, D. A. (eds.), International Handbook of Survey Methodology (pp. 113–35). Hillsdale, NJ: Lawrence Erlbaum Associates.
de Leeuw, E., Hox, J., & Kef, S. (2003). Computer-assisted self-interviewing tailored for special populations and topics. Field Methods, 15(3), 223–51.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, Mail and Mixed Mode Surveys: The Tailored Design Method (3rd edn.). Hoboken, NJ: John Wiley & Sons.
Dowrick, C., Gask, L., Edwards, S., Aseem, S., Bower, P., Burroughs, H., et al. (2009). Researching the mental health needs of hard-to-reach groups: managing multiple sources of evidence. BMC Health Services Research, 9, 226.
Draus, P. J., Siegal, H. A., Carlson, R. G., Falck, R. S., & Wang, J. (2005). Cracking the cornfields: recruiting illicit stimulant drug users in rural Ohio. Sociological Quarterly, 46(1), 165–89.
Dunlap, E., Golub, A., & Johnson, B. D. (2003). Transient male–female relationships and the violence they bring to girls in the inner city. Journal of African American Studies, 7(2), 19–36.
Elliott, K., & Fuller, J. (1991). Health education and ethnic minorities: needs may differ, the techniques don’t. BMJ, 302(6780), 802–03.
Ennett, S. T., Bailey, S. L., & Federman, E. B. (1999). Social network characteristics associated with risky behaviors among runaway and homeless youth. Journal of Health and Social Behavior, 40(1), 63–78.
Errante, A. (1997). Close to home: comparative perspectives on childhood and community violence. American Journal of Education, 105(4), 355–400.
Faugier, J., & Sargeant, M. (1997). Sampling hard to reach populations. Journal of Advanced Nursing, 26(4), 790–97.
Feild, L., Pruchno, R. A., Bewley, J., LemayJr., E. P., & Levinsky, N. G. (2006). Using probability vs. nonprobability sampling to identify hard-to-access participants for health-related research: costs and contrasts. Journal of Aging and Health, 18(4), 565–83.
Flanagan, S. M., & Hancock, B. (2010). “Reaching the hard to reach” – lessons learned from the VCS (voluntary and community sector). A qualitative study. BMC Health Services Research, 10, 92–100.
Ford, W. L., Weiss, G., Kipke, M. D., Ritt-Olson, A., Iverson, E., & Lopez, D. (2009). The Healthy Young Men’s Study: sampling methods to recruit a random cohort of young men who have sex with men. Journal of Gay & Lesbian Social Services, 21(4), 357–73.
FowlerJr., F. J., & Mangione, T. W. (1990). Standardized Survey Interviewing: Minimizing Interviewer-related Error. Newbury Park, CA: Sage.
Fredriksen-Goldsen, K., & Hooyman, N. R. (2005). Caregiving research, services, and policies in historically marginalized communities: where do we go from here?Journal of Gay & Lesbian Social Services, 18(3), 129–45.
Freimuth, V. S. (1995). Mass media strategies and channels: a review of the use of media in breast and cervical cancers. Wellness Perspectives, 11(2), 79.
Garland, J., Spalek, B., & Chakraborti, N. (2006). Hearing lost voices. British Journal of Criminology, 46(3), 423–37.
Geringer, W., Marks, S., Allen, W., & Armstrong, K. (1993). Knowledge, attitudes, and behavior related to condom use and STDs in a high risk population. Journal of Sex Research, 30(1), 75–83.
Gertler, P., Shah, M., & Bertozzi, S. (2005). Risky business: the market for unprotected commercial sex. Journal of Political Economy, 113(3), 518–50.
Goel, S., & Salganik, M. J. (2010). Assessing respondent-driven sampling. Proceedings of the National Academy of Sciences of the United States of America, 107(15), 6743–47.
Greenfields, M. (2009). Reaching Gypsies and Travellers. Primary Health Care, 19(8), 26–27.
Greenhalgh, T., Collard, A., & Begum, N. (2005). Sharing stories: complex intervention for diabetes education in minority ethnic groups who do not speak English. BMJ, 330(7492), 628–31.
Groves, R. M., FowlerJr., F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey Methodology. Hoboken, NJ: John Wiley & Sons.
Groves, R., & Lyberg, L. (2010). Total survey error: past, present, and future. Public Opinion Quarterly, 74(5), 849–79.
Groves, R., & McGonagle, K. (2001). A theory-guided interviewer training protocol regarding survey participation. Journal of Official Statistics, 17(2), 249–65.
Han, H.-R., Kang, J., Kim, K. B., Ryu, J., & Kim, M. (2007). Barriers to and strategies for recruiting Korean Americans for community-partnered health promotion research. Journal of Immigrant and Minority Health, 9(2), 137–46.
Harkness, J. (2003). Questionnaire translation. In Harkness, J. A., van de Vijver, F. J. R., & Mohler, P. Ph. (eds.), Cross-Cultural Survey Methods (pp. 35–56). Hoboken, NJ: John Wiley & Sons.
Harkness, J. A., Villar, A., & Edwards, B. (2010). Translation, adaptation, and design. In Harkness, J. A., Braun, M., Edwards, B., Johnson, T. P., Lyberg, L., Mohler, P. Ph., Pennell, B-E., & Smith, T. W. (eds.), Survey Methods in Multinational, Multiregional, and Multicultural Contexts (pp. 117–40). Hoboken, NJ: John Wiley & Sons.
Heckathorn, D. (2007). Extensions of respondent-driven sampling: analyzing continuous variables and controlling for differential recruitment. Sociological Methodology, 37(1), 151–208.
Henning, K. J., Pollack, D. M., & Friedman, S. M. (1992). A neonatal hepatitis B surveillance and vaccination program: New York City, 1987 to 1988. American Journal of Public Health, 82(6), 885–88.
Hidaka, Y., & Operario, D. (2006). Attempted suicide, psychological health and exposure to harassment among Japanese homosexual, bisexual or other men questioning their sexual orientation recruited via the internet. Journal of Epidemiology and Community Health, 60(11), 962–67.
Higgins, D. L., O’Reilly, K., Tashima, N., Crain, C., Beeker, C., Goldbaum, G., et al. (1996). Using formative research to lay the foundation for community level HIV prevention efforts: an example from the AIDS community demonstration projects. Public Health Reports, 111(Suppl. 1), 28–35.
Hogue, A., Johnson-Leckrone, J., & Liddle, H. A. (1999). Recruiting high-risk families into family-based prevention and prevention research. Journal of Mental Health Counseling, 21(4), 337.
Horwitz, R. H., Roberts, L. W., & Warner, T. D. (2008). Mexican immigrant women’s perceptions of health care access for stigmatizing illnesses: a focus group study in Albuquerque, New Mexico. Journal of Health Care for the Poor and Underserved, 19(3), 857–73.
Hossain, Z. (2001). Division of household labor and family functioning in off-reservation Navajo Indian families. Family Relations, 50(3), 255–61.
Icard, L. D. (2008). Reaching African-American men on the “down-low”: sampling hidden populations: Implication for HIV prevention. Journal of Homosexuality, 55(3), 437–49.
Johnston, L. G., Trummal, A., Lohmus, L., & Ravalepik, A. (2009). Efficacy of convenience sampling through the internet versus respondent driven sampling among males who have sex with males in Tallinn and Harju County, Estonia: challenges reaching a hidden population. AIDS Care, 21(9), 1195–202.
Jones, L., & Allebone, B. (1999). Researching “hard-to-reach” groups: the crucial role of the research associate. International Journal of Inclusive Education, 3(4), 353–62.
Kalton, G. (2003). Practical methods for sampling rare and mobile populations. Statistics in Transition, 6(4), 491–501.
Kalton, G. (2009). Methods for oversampling rare subpopulations in social surveys. Survey Methodology, 35(2), 125–41.
Kalton, G., & Anderson, D. W. (1986). Sampling rare populations. Journal of the Royal Statistical Society: Series A (General), 149(1), 65–82.
Kandel, D. B. (1991). The social demography of drug use. Milbank Quarterly, 69(3), 365–414.
Kish, L. (1965). Survey Sampling. New York: John Wiley & Sons.
Koné, A., Sullivan, M., Senturia, K. D., Chrisman, N. J., Ciske, S. J., & Krieger, J. W. (2000). Improving collaboration between researchers and communities. Public Health Reports, 115(2/3), 243–48.
Kumar, M. S., Mudaliar, S., & Daniels, D. (1998). Community-based outreach HIV intervention for street-recruited drug-users in Madras, India. In Needle, R. H., Coyle, S., & Cesari, H. (eds.), HIV prevention with drug-using populations – current status and future prospects. Public Health Reports, 113(Suppl. 1), 58–66.
Landry, P. F., & Shen, M. (2005). Reaching migrants in survey research: the use of the global positioning system to reduce coverage bias in China. Political Analysis, 13(1), 1–22.
Lansky, A., Abdul-Quader, A. S., Cribbin, M., Hall, T., Finlayson, T. J., Garfein, R. S., et al. (2007). Developing an HIV behavioral surveillance system for injecting drug users: the national HIV behavioral surveillance system. Public Health Reports, 122(Suppl. 1), 48–55.
Laporte, R. E. (1994). Assessing the human condition: capture–recapture techniques: allows accurate counts of those difficult to reach populations. BMJ, 308(6920), 5–6.
Lavelle, B., Larsen, M. D., & Gundersen, C. (2009). Strategies for surveys of American Indians. Public Opinion Quarterly, 73(2), 385–403.
Lessler, J. T., Eyerman, J., & Wang, K. (2008). Interviewer training. In de Leeuw, E. D., Hox, J. J., & Dillman, D. A. (eds.), International Handbook of Survey Methodology (pp. 442–60). New York: Lawrence Erlbaum Associates.
Loosveldt, G. (2008). Face-to-face interviews. In de Leeuw, E. D., Hox, J. J., & Dillman, D. A. (eds.), International handbook of survey methodology (pp. 201–20). New York: Lawrence Erlbaum Associates.
Lyberg, L., & Biemer, P. (2008). Quality assurance and quality control in surveys. In de Leeuw, E. D., Hox, J. J., & Dillman, D. A. (eds.), International Handbook of Survey Methodology (pp. 421–441). New York: Lawrence Erlbaum Associates.
Lyberg, L., & Couper, M. (2005). The use of paradata in survey research. Invited paper presented at the ISI Meeting, Sydney, April.
MacKellar, D. A., Gallagher, K. M., Finlayson, T., Sanchez, T., Lansky, A., & Sullivan, P. S. (2007). Surveillance of HIV risk and prevention behaviors of men who have sex with men: a national application of venue-based, time-space sampling. Public Health Reports, 122(Suppl. 1), 39–47.
McMahon, J. M., Tortu, S., Torres, L., Pouget, E. R., & Hamid, R. (2003). Recruitment of heterosexual couples in public health research: a study protocol. BMC Medical Research Methodology, 3, 24.
Martin, J. L., Wiley, J., & Osmond, D. (2003). Social networks and unobserved heterogeneity in risk for AIDS. Population Research and Policy Review, 22(1), 65–90.
Merriam, S. B. (2009). Qualitative research: a guide to design and implementation. San Francisco: John Wiley & Sons.
O’Hegarty, M., Pederson, L. L., Thome, S. L., Caraballo, R. S., Evans, B., Athey, L., et al. (2010). Customizing survey instruments and data collection to reach Hispanic/Latino adults in border communities in Texas. American Journal of Public Health, 100(S1), S159–S164.
Pauw, I., & Brener, L. (2003). “You are just whores: you can’t be raped”: barriers to safer sex practices among women street sex workers in Cape Town. Culture, Health & Sexuality, 5(6), 465–81.
Pennell, B-E., Levenstein, R., & Lee, H. J. (2011). Cross-Cultural Survey Guidelines. Ann Arbor, MI: University of Michigan. Retrieved from .
Pettifor, J. L. (1979). Ethical issues with special populations. Canadian Psychological Review/Psychologie Canadienne, 20(3), 148–50.
Salganik, M. J., & Heckathorn, D. D. (2004). Sampling and estimation in hidden populations using respondent-driven sampling. Sociological Methodology, 34(1), 193–239.
Schroepfer, T., Matloub, J., Creswell, P., Strickland, R., & Anderson, D. (2009). A community-specific approach to cancer research in Indian Country. Progress in Community Health Partnerships: Research, Education, and Action, 3(4), 317–25.
Shaver, F. M. (2005). Sex work research: methodological and ethical challenges. Journal of Interpersonal Violence, 20(3), 296–319.
Smith, T. W. (2003). Developing comparable questions in cross-national surveys. In Harkness, J. A., van de Vijver, F. J. R., & Mohler, P. Ph. (eds.), Cross-Cultural Survey Methods (pp. 69–92). Hoboken, NJ: John Wiley & Sons.
Spreen, M., & Zwaagstra, R. (1994). Personal network sampling, outdegree analysis and multilevel analysis: introducing the network concept in studies of hidden populations. International Sociology, 9(4), 475–91.
Stueve, A., O’Donnell, L. N., Duran, R., San Doval, A., & Blome, J. (2001). Time-space sampling in minority communities: results with young Latino men who have sex with men. American Journal of Public Health, 91(6), 922–26.
Sudman, S., Sirken, M., & Cowan, C. (1988). Sampling rare and elusive populations. Science, 240, 991–95.
Toktaş, S. (2006). The conduct of citizenship in the case of Turkey’s Jewish minority: legal status, identity, and civic virtue aspects. Comparative Studies of South Asia, Africa and the Middle East, 26(1), 121–33.
Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The Psychology of Survey Response. Cambridge: Cambridge University Press.
Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: the impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60(2), 275–304.
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859–83.
Vlahov, D., Coady, M. N., Galea, S., Ompad, D. C., & Barondess, J. A. (2007). Pandemic preparedness and hard to reach populations. American Journal of Disaster Medicine, 2(6), 281–83.
Vyavaharkar, M. V., Moneyham, L., & Corwin, S. (2008). Health care utilization: the experiences of rural HIV-positive African American women. Journal of Health Care for the Poor and Underserved, 19(1), 294–306.