Introduction
Recruitment of participants into longitudinal, observational studies that do not include any direct health benefit is a challenge [Reference Galea and Tracy1], yet such studies are critical to identifying the incidence and natural history of many pediatric health conditions. There is a growing body of evidence regarding recruitment strategies for randomized clinical trials [Reference Mapstone, Elbourne and Roberts2, Reference Treweek3], yet few evidence-based recommendations regarding effective recruitment in cohort studies. Community outreach and engagement is one commonly identified tactic [Reference Sapienza4–Reference Golding and Birmingham6]. Reviews of community-based research suggest community engagement may increase enrollment [Reference Barbara7]. The literature contains reports of outreach and engagement as a passive recruitment strategy whereby study information is made widely available throughout the community to encourage potential participants to contact study recruiters [Reference Webster, Teschke and Janssen8–Reference Promislow11]. Little is known about the effectiveness of outreach and engagement in a preparatory role for active recruitment in which potential participants are contacted by staff and invited to join a study. From 2009 to 2012, the National Children’s Study (NCS) Vanguard Study implemented outreach and engagement in support of both passive and active recruitment across the United States. In this article, we describe the effectiveness of those strategies in reaching eligible women and in supporting consent.
The NCS Vanguard Study
The Vanguard Study was a pilot study that enrolled women and their children born in 37 diverse US locations (counties or parts of counties) to evaluate the feasibility, acceptability, and cost of recruitment and operations for the NCS, a large-scale epidemiological cohort study of children and their parents [12, 13]. Data were to be collected from prepregnancy until the children reached adulthood. Ultimately, it was decided that moving forward with the NCS was not feasible as designed. The Vanguard Study was closed in December 2014 [13].
The recruitment phase of the Vanguard Study began in 2009 with the Initial Vanguard Study (IVS) (see Table 1) [Reference Baker14]. The IVS was conducted at 7 locations; field workers enumerated households in predetermined geographical areas to identify and consent eligible women. Based on early results, a decision was made to evaluate additional recruitment strategies in an Alternate Recruitment Substudy (ARS), which began November 2010. The ARS evaluated three different recruitment strategies in 30 additional locations: Direct Outreach (DO) [Reference McGovern15], Enhanced Household-Based Recruitment (EHBR) [Reference Blaisdell16], and Provider-Based Recruitment (PBR) [Reference Hale17]. The DO strategy recruited women using community outreach and engagement, including direct mailing. Interested individuals contacted study centers and were screened via an in-person or phone interview, or a self-administered questionnaire. The EHBR strategy used improved staff training and more targeted outreach and engagement to enhance the household enumeration strategy of the IVS. The PBR strategy worked with healthcare providers to identify women of childbearing age for eligibility screening and recruitment.
Overview of National Children’s Vanguard Study (NCS) recruitment groups and outreach and engagement approaches
NCS Outreach and Engagement
Recruitment at each location was conducted by a study center. Study centers were universities or research organizations, similar to academic investigative sites, under contract with the National Institutes of Health (NIH) to conduct the day-to-day operations of the study. Some centers covered more than 1 location. Each center was charged with implementing outreach and engagement to support recruitment. Although some supporting materials were centrally developed by NIH (eg, logos, web sites, pamphlets, posters, small giveaways), centers generally tailored methods and materials for localities. Outreach and engagement strategies were conducted for up to 2 years before the start of recruitment. All study centers hired local staff and had Community Advisory Boards to recommend best approaches to tailoring materials and activities. Most centers individualized the NCS logo for their location and ordered NCS-branded gear and identification badges.
Strategies generally fell into 5 categories: (1) advance letters; (2) healthcare providers; (3) community events; (4) community organizations; and (5) media (for data analysis, these categories were later divided into more specific groups of sources as reported by study participants: advance letters, healthcare providers, family/friends, other person, community organizations, and media). Except for 3 PBR locations, all centers mailed advance letters to households in predetermined geographical areas telling them about the study. Centers also conducted in-person or mailed outreach and engagement with local healthcare providers to inform them about the study, encourage them to either participate or recommend participants. All centers participated in community events (eg, local fairs, kindergarten registration) to distribute NCS information and giveaways (eg, water bottles) and, in the ARS, conduct supplemental recruitment. Centers also visited community organizations and businesses (eg, neighborhood associations, grocery stores) to raise study awareness, gain support from key formal and informal community leaders, and ask for suggestions for improving recruitment.
Media outreach efforts were both nationally and locally produced and included television (TV), radio, billboards, print, online, and social media. The NIH NCS Program Office implemented time-limited national media buys for TV, radio, and billboards in ARS locations, supplied centrally produced pamphlets and posters, and hosted a national web site with linked individualized study center sites. Centers varied in how extensive their tailored local campaigns were. All centers developed their own tailored local print materials; many placed ads in local newspapers, made regular press releases, or included mailings in local utility bills. Some ARS centers also purchased online advertisements and maintained Facebook pages, active Twitter feeds, and NCS blogs. In general, ARS locations had more widespread local campaigns than IVS locations, particularly with respect to TV, radio, online, and social media.
Materials and Methods
Sampling Frame
The original sample design for the NCS was a nationally representative, multistage, area probability sample, designed to provide estimates that could be generalized to the US population with subsets of the primary sampling units, generally equivalent to counties, selected to serve as locations; however, the 37 pilot locations of the NCS Vanguard were chosen to pretest field operations and not selected randomly. Some attempt at balancing demographic characteristics was made, but without statistical precision.
The second sampling stage was the selection of geographical segments within the primary sampling units. Segments were based on aggregations of contiguous census blocks with the measure of size being estimated annual births during the enrollment period. In combination, these secondary sampling units would yield ~250 births per year per primary sampling unit. The study centers consulted with local community representatives to get input on potential segment boundaries, so the segments would reflect coherent neighborhood groups to facilitate community outreach.
Sample
Recruitment consisted of 2 stages. First, women in the secondary sampling units who were identified by household enumeration (IVS, EHBR), self-referral (DO), or provider referral (PBR) and were 18–49 years old were asked to complete a study eligibility questionnaire, the Pregnancy Screener (see Fig. 1). Second, those women who were pregnant or actively trying to get pregnant were invited to participate. In the IVS, pregnant minors (<18 years old) were eligible if the state they resided in defined them as emancipated; minors were ineligible in the 3 ARS groups. Women who were older than 49 and pregnant were eligible in both the IVS and ARS.
Flow diagram of samples by type of recruitment. aSource: National Children’s Study [18]. bSample for this study. IVC Participant Recruitment Dataset V2.1, IVC Enrolled Women Analysis File V2.1, IVC Recruitment Analysis File V2.1, and ARS Analysis File V3.1.
Procedures
The study protocol was approved by the Eunice Kennedy Shriver National Institute of Child Health and Human Development Institutional Review Board (IRB) as the national federated IRB [Reference Slutsman and Hirschfeld19] or, depending on study center choice, a local IRB. A waiver of written documentation of consent allowed staff to obtain verbal informed consent from women for completion of the Pregnancy Screener. Women from the IVS, EHBR, and PRB locations were interviewed by trained interviewers or computer-assisted personal interviewing; women in the DO locations were either interviewed or completed a self-administered Pregnancy Screener and returned it via mail. The Pregnancy Screener was available in English and Spanish. Spanish-speaking interviewers interviewed Spanish-speaking women. If women’s primary spoken language was other than Spanish, interviews were conducted in the women’s language whenever possible.
Measures
This analysis employs 7 self-reported demographic items and 2 items about familiarity with the NCS from the initial enumeration questionnaire used in the IVS and Pregnancy Screener, used in both the IVS and ARS. The demographic items included age, race, ethnicity, primary language, marital status (married; not married—living with partner or never married; separated, divorced, or widowed), highest level of education completed (<high school diploma or general educational development; high school diploma or general educational development; some college/associate degree; bachelor’s degree; postgraduate degree), and annual family income (<$30,000; $30–$49,999; $50–$99,999; >$100,000). Education and income data were not available for the IVS. A composite race/ethnicity variable was created and included 5 groups: Hispanic; non-Hispanic White, African-American, Asian, and other. The first of the items about NCS familiarity was “Before today, had you heard about the National Children’s Study?” Response choices were: yes, no, refused, and don’t know. Those participants who responded “yes” to the first question were then asked “How did you hear about the National Children’s Study?” There were 13 possible response options for the IVS and 23 for the ARS, including options such as advance letter, healthcare provider, family member, the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) or another social agency, TV, and other. Respondents were encouraged to list all the ways that they had heard about the NCS.
Data Analysis
All data were submitted electronically by study centers to the contractors identified by the NIH Program Office and were analyzed by another private contractor using SAS/STATTM software. This analysis used the IVC Participant Recruitment Dataset V2.1, IVC Enrolled Women Analysis File V2.1, IVC Recruitment Analysis File V2.1, and ARS Analysis File V3.1 (based on April 11, 2014 Virtual Data Repository submission). Data files used were not final versions and data presented may be slightly different from other published versions. Differences in demographic variables between recruitment groups were analyzed using χ2 for categorical variables and the Kruskal-Wallis for the non-normal, continuous age variable. Logistic regression was used to analyze prediction models of hearing about the NCS and consenting for the NCS. Observations that contained missing values for either the dependent or independent variables were excluded from the regression.
Results
Our results describe the effectiveness of outreach and engagement in helping eligible women hear about the study and in promoting their consent. We address 6 research questions: (1) Did community outreach and engagement result in women having heard about the NCS when first contacted? (2) Did more women hear about the NCS in the ARS than the IVS? (3) How did women hear about the NCS? (4) Were women who had heard about the NCS more likely to consent? (5) Were certain sources of hearing about the NCS or the number of sources related to consent? (6) Were all demographic sub-populations equally likely to have heard about the NCS?
The 9844 women who completed the Pregnancy Screener and were confirmed eligible for consent are the sample for this article (see Fig. 1): 3088 from the IVS; 2786 from the DO; 2491 from the EHBR; and 1479 from the PBR recruitment groups. Table 2 displays the sample characteristics. There were significant differences between recruitment groups in all demographic characteristics (χ2=169.4−1180.9, P<0.001). However, because this was a feasibility study and the duration of participant recruitment varied across and within recruitment groups, Table 2 data are presented for descriptive purposes only, not as a basis for comparison across ARS strategies. Details about demographic characteristics of samples compared with local populations have been presented elsewhere [Reference Baker14–Reference Hale17].
Demographic characteristics of women eligible for consent for the National Children’s Study
IVS, Initial Vanguard Study; DO, Direct Outreach; EHBR, Enhanced Household-Based Recruitment; PBR, Provider-Based Recruitment; GED, general educational development.
* Including women from both the original IVS period of January 2009 to September 2010 and beyond.
How Women Heard About the Study
Overall, most women who were study eligible had already heard about the NCS before their first contact with recruiters (Table 3). ARS women had heard more than IVS women. Advance letters, media outreach, and friends, family, or other people were the most frequent ways that IVS, DO, and EHBR women had heard about the study before being approached by data collectors. Healthcare providers, as expected, and media were the most frequent for the PBR women. Community organizations were one of the least frequent ways women had heard across all groups. Most women reported that they had heard about the study from only 1 source.
Hearing about the National Children’s Study (NCS) among women eligible for consent
IVS, Initial Vanguard Study; DO, Direct Outreach; EHBR, Enhanced Household-Based Recruitment; PBR, Provider-Based Recruitment.
* Participants may select more than 1 source, so column totals may exceed 100%.
† IVS: doctor or healthcare provider; Alternate Recruitment Substudy (ARS): prenatal care provider or other healthcare provider.
‡ IVS: friends or family; ARS: other friend or family member.
§ IVS: someone else in community or community leader; ARS: other NCS participant, neighbor, co-worker, community partners, outreach event.
‖ IVS: Church, synagogue, other religious; ARS: school, WIC, other social agency, or religious organization.
¶ IVS: billboards, newspaper, television, or radio; ARS: print media, television, or radio.
Community Outreach and Engagement and Women’s Consent
After adjusting for demographics, women who had heard about the NCS (yes/no) were an estimated 1.5–3 times more likely to consent to participate than women who had not [IVS: odds ratio (OR)=1.67, 95% confidence interval (CI): 1.40, 1.98; DO: OR=2.90, 95% CI: 2.35, 3.59; EHBR: OR=1.58, 95% CI: 1.29, 1.94; PBR: OR=2.30, 95% CI: 1.61, 3.26]. This effect was strongest for DO and PBR, and weakest for IVS and EHBR. Among women who had heard, the importance of how women heard varied across recruitment groups after controlling for demographics (Table 4). In the IVS, no one category of outreach and engagement was associated with consents. In the ARS, the advance letter was not associated with DO or EHBR women’s consent. However, PBR women who heard about the study by advance letter were more likely to consent than those who heard only from other sources. Likewise, DO and EHBR women who heard via the media were more likely to consent. EHBR women who heard from a healthcare provider, friends, or family were more likely to consent. And DO and PBR women who heard from another person were more likely to consent than women who had not. Across all recruitment groups, the more sources women heard from, the greater the odds of consent, with a slight variation in the PBR group (Table 5).
Logistic regression of type of outreach on consent status among women who had heard about the National Children’s Study (NCS) (by recruitment type, controlling for demographic variables)
IVS, Initial Vanguard Study; DO, Direct Outreach; EHBR, Enhanced Household-Based Recruitment; PBR, Provider-Based Recruitment; OR, odds ratio; CI, confidence interval.
Bold value indicates that CI does not contain 1.00; P-value for the Hosmer and Lemeshow goodness-of-fit test: IVS=0.14; DO=0.96; EHBR=0.16; PBR=0.68.
* Reference category for each individual source is not hearing from that source.
† IVS: doctor or healthcare provider; Alternate Recruitment Substudy (ARS): prenatal care provider or other healthcare provider.
‡ IVS: friends or family; ARS: other friend or family member.
§ IVS: someone else in community or community leader; ARS: other NCS participant, neighbor, co-worker, community partners, outreach event.
‖ IVS: Church, synagogue, other religious; ARS: school, WIC, other social agency, or religious organization.
¶ IVS: billboards, newspaper, TV, or radio; ARS: print media, TV, or radio.
Logistic regression of number of sources heard about National Children’s Study (NCS) on consent status among women eligible for consent (by recruitment type, controlling for demographic variables)
IVS, Initial Vanguard Study; DO, Direct Outreach; EHBR, Enhanced Household-Based Recruitment; PBR, Provider-Based Recruitment; OR, odds ratio; CI, confidence interval.
Bold value indicates that CI does not contain 1.00; P-value for the Hosmer and Lemeshow goodness-of-fit test: IVS=0.95; DO=0.76; EHBR=0.04; PBR=0.23.
* Reference category is not hearing from any source.
† Six possible sources: advance letter, healthcare provider, friends/family, other person, community organization, media (see Table 3 for operational definitions).
‡ Categories 3–6 were combined to provide stable OR estimates.
Demographics and Hearing About the Study
Table 6 presents the logistic regression to assess whether demographic variables predicted whether a woman reported that she had heard about the NCS. Because of the many significant interactions between all demographic variables and recruitment type, separate logistic regression models were built for each recruitment group. In general, across all recruitment groups, women who were Hispanic, African-American, Asian, or another non-Hispanic race/ethnicity were much less likely to have heard about the NCS than non-Hispanic White women, with a few significant exceptions. There were no differences in hearing about the NCS between EHBR non-Hispanic White and Hispanic women, the group with the highest percentage (23%) of Hispanics. And DO African-American women were almost twice as likely to have heard about the NCS as non-Hispanic Whites.
Logistic regression of demographic variables on hearing about the National Children’s Study (NCS) among women eligible for consent by recruitment type
IVS, Initial Vanguard Study; DO, Direct Outreach; EHBR, Enhanced Household-Based Recruitment; PBR, Provider-Based Recruitment; OR, odds ratio; CI, confidence interval; GED, general educational development.
Bold value indicates that CI does not contain 1.00; P-value for the Hosmer and Lemeshow goodness-of-fit test: IVS=0.72; DO=0.74; EHBR=0.32; PBR=0.54.
* Referent category for logistic regression.
Age and primary language spoken at home were not associated with hearing about the NCS for any of the recruitment groups. In the EHBR and PBR groups, women who were not married were as likely to report hearing about the NCS as women who were married. In contrast, IVS and DO unmarried women were less likely than married women to have heard. Education and income data were not collected for the IVS group. However, across all 3 ARS groups, women who had bachelors or graduate degrees were more likely to have heard than high school graduates. DO women with annual incomes of $50,000 or higher were less likely than women with incomes of $30–$49,999 to report hearing about the study. Income was not associated with hearing about the NCS in the other 2 ARS groups.
Discussion
The NCS is the one of the first studies to demonstrate that, while controlling for demographic variables, effective outreach and engagement predicts more successful cohort recruitment. Effective outreach and engagement was measured by whether women had heard about the NCS. Women who had heard before having contact with recruiters were more likely to consent. Consents increased both when outreach and engagement were used as a passive recruitment strategy and women needed to self-refer (DO), and when used to supplement active recruitment by staff either through door-to-door contact (IVS, EHBR) or providers (PBR).
Tailored activities conducted for 1–2 years before the start of recruitment were associated with substantial percentages of women who reported that they had heard about the NCS at initial screening. The DO group had the highest percentage of women who had heard, not surprising given that DO recruitment was dependent on self-referral. Kaar et al. [Reference Kaar20] observed that women who said they had not heard were those whose first contact with the study was when they received the Pregnancy Screener in the mail. In addition to the DO group, the percentages of EHBR and PBR women who had heard were also substantially higher than the IVS, strongly suggesting that the more structured and extensive outreach and engagement in the ARS was most effective.
The NCS provides strong support for the importance of multifaceted outreach and engagement for successful recruitment. Across recruitment groups, the more sources women heard from, the higher the odds of consent. Depending on the recruitment strategy, different methods of outreach and engagement affected the odds of consenting differently. Word-of-mouth, hearing about the study from a healthcare provider or another person, increased the odds of consent across ARS groups. For 3 of the 4 recruitment types, those that emphasized self-referral (DO) or were household-based (IVS, EHBR), approximately one-third of the women had heard through word-of-mouth. In the PBR group, that percentage was much higher because, as would be expected, over half of the women had heard about the study from a healthcare provider. Interestingly, hearing from a provider did not increase the odds of consent in that group, but hearing from another person did.
Of the women who heard about the study, 20%–42% heard from the media—print, TV, or radio. Media outreach was much more extensive in the ARS than the IVS, with both national and local campaigns, and more widespread use of radio, TV, online and social media. Media outreach predicted consent in 2 of the 3 ARS groups, DO and EHBR. This finding is consistent with Maghera et al. [Reference Maghera9] who found that passive recruitment using free and paid media had strong effects in predicting consent when compared with active contact from a healthcare provider. In an analysis of DO recruitment, Kaar et al. [Reference Kaar20] observed that in locations with high NCS awareness (≥75% of women who had heard), higher percentages of women said that they had heard through the media. The impact of media outreach is likely underestimated here because many of the people who told others about the study (word-of-mouth) across recruitment groups quite likely heard about it through media in the first place. Manca et al. [Reference Manca10] found that nearly 3 quarters of the consented women in one of their pregnancy cohorts reported that word-of-mouth or media outreach was the primary recruitment method.
In the Vanguard Study, except for 3 PBR locations, all centers mailed advance letters to households. The letters were one of the most frequent ways that women reported that they had heard about the study. The literature regarding randomized clinical trials suggests that mass mailing may increase recruitment [Reference Tooher, Middleton and Crowther21, Reference Hinshaw, Jackson and Chen22]. However, in this study, advance mailings only predicted consent for PBR. It may be that advance mailings primed women to be more receptive when approached within a trusted setting where provider approval might be inferred, but not when women were being asked to self-refer or to make a commitment to a stranger approaching them in their homes. Indeed, Promislow et al. [Reference Promislow11] reported that only 5% of their pregnancy cohort was recruited through mailings. And a systematic review of strategies to improve recruitment to clinical trials found no effect in 2 trials of advance letters or postcards as supporting methods for active recruitment [Reference Mapstone, Elbourne and Roberts2].
Study results were mixed regarding demographics and whether women heard about the study. In general, non-Hispanic Whites were more likely than other ethnic/racial groups to have heard before being approached. However, there were 2 notable exceptions. In the EHBR group, which had the highest percentage of Hispanics, Hispanics were equally likely as non-Hispanic Whites. And in the DO group, African-Americans were twice as likely to have heard. Across all recruitment groups, women with a bachelor’s degrees or postgraduate education were more likely to have heard than women with a high school education. Our findings suggest that it is possible to tailor outreach and engagement to successfully reach ethnically and racially diverse samples, but emphasize the need to tailor approaches for diverse educational backgrounds.
This article has several limitations. First, there were significant interactions between all demographic variables and recruitment type and the demographic results should be viewed with caution. Differences between recruitment strategies could be due to differences in demographics. A comprehensive regression model containing all main and interaction effects would not be interpretable. Second, specific community outreach and engagement strategies were not quantified. Although all locations had similar stated expectations from NIH, different groups may have allocated significantly different levels of resources to outreach and engagement. Finally, the Vanguard Study was a feasibility study and the duration of recruitment varied across the locations and recruitment groups. For example, the PBR group had to negotiate access with providers before they could begin recruiting and in some locations, actual recruitment only lasted a few months. Other researchers have reported changes in patterns of cohort recruitment at 6 or 12 months [Reference Webster, Teschke and Janssen8, Reference Promislow11]. Despite this limitation, 1 strength of this study is that the final sample included nearly 10,000 study-eligible women.
Conclusion
Outreach and engagement are often some of the most time-intensive and costly aspects of study implementation, especially for longitudinal studies. This study demonstrates that multifaceted, place-sensitive outreach and engagement facilitates successful passive and active recruitment of a prebirth cohort. Effective approaches must be based on community input and a strong working knowledge of the local community. Media outreach and strategies to encourage word-of-mouth appear to be critical to successful recruitment.
Acknowledgments
This manuscript was developed by a Writing Team identified by the National Children’s Study Publications Committee for timely sharing of centrally collected NCS data. It is thus a primary NCS publication.
The Writing Team would like to thank the families and community members who participated in or contributed to the NCS Vanguard Study. Without their participation, this Study would not have been possible. We also want to thank the 40 NCS study centers, their investigators and their staff, for their contributions and many years of tireless effort.
This analysis was conducted as part of the National Children’s Study (NCS), supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and funded through its appropriation, by the Office of the Director of the National Institutes of Health (contracts HHSN275200800035C, HHSN275200800034C, HHSN275200800033C, HHSN275200800031C, HHSN275200800029C (N.E.), HHSN275200800026C, HHSN275200800025C, HHSN275200800024C, HHSN275200800018C (J.L.K), HHSN275200800017C, HHSN275200800016C, HHSN275200800015C (T.B.), HHSN275200800011C, HHSN275200800010C, HHSN275200800009C, HHSN275200800007C, HHSN275200800006C, HHSN275200800004C, HHSN275200800002C, HHSN267200700049C, HHSN267200700047C, HHSN267200700034C, HHSN267200700033C, HHSN267200700032C, HHSN267200700031C (B.B.T.) (also supported by grant number UL1TR000041), HHSN267200700030C, HHSN267200700029C, HHSN267200700028C, HHSN267200700027C, HHSN267200700025C, HHSN267200700024C, HHSN267200700023C, HHSN267200700021C, HHSN267200700020C, HHSN267200700017C, HHSN267200700015C, HHSN267200700006C).
Declaration of Interest
All authors report no conflicts of interest.
Disclaimer
The views expressed in this article are the responsibility of the authors and do not necessarily represent the position of the National Children’s Study, the National Institutes of Health, or the US Department of Health and Human Services.
Open access
