Literacy promotion for Latinos study

This ongoing clinical trial (ClinicalTrials.gov Identifier NCT04609553) tests the extent to which outreach text messages and referrals to enhance access to poverty-reducing resources, in combination with standard literacy promotion, enhances child language and social-emotional skill acquisition among Latino children from under-resourced communities. This clinical trial builds on our team’s prior community-engaged research [Reference Jimenez, Hudson, Lima and Crabtree18–Reference Jimenez, Crabtree and Veras21] that informed interventions and outcomes assessed. Specifically, local community leaders and Latino parents identified early childhood development and school readiness as prioritized goals and identified pediatric professionals as being well positioned to address these goals [Reference Jimenez, Hudson, Lima and Crabtree18,Reference Jimenez, Hudson, Lima, Mendelsohn, Pellerano and Crabtree19]. The interventions respond to barriers identified by Latino parents who participated in the pilot work. Specifically, they highlighted the need to provide reminders to engage in parenting routines like shared reading and address economic hardship that interferes with parents’ ability to engage in activities like shared reading [Reference Jimenez, Crabtree and Hudson20,Reference Jimenez, Crabtree and Veras21]. Moreover, the study assesses outcomes that parents identified as important including parenting, language development, and social-emotional skills.

Participants and setting

We recruited 630 Latino parent–infant dyads from three community health centers (CHCs) in Middlesex County, New Jersey (N.J.) (Table 1) [22]. New Brunswick and Perth Amboy, where the CHCs are located, have large Latino populations (46% and 78%), with sizeable proportions born outside the USA (77% and 91%) [Reference Perth Amboy23–25]. Two CHCs have similar annual pediatric patient volumes (∼ 5,500 pediatric patients/year), with 73% and 66% identifying as Latino, respectively. The third has higher annual pediatric patient volume (∼20,000 pediatric patients/year) and 55% identify as Latino. Many of these CHCs’ patients prefer receiving health information in Spanish. Inclusion criteria were primary caregiver of a child 6–12 months, Latino background, primary language English or Spanish, cellphone ownership, age≥18 years, willing to receive text messages, and willing to accept randomization. Exclusion criteria were children with genetic disorders or previously identified developmental delays, inability to provide informed consent, and intent to discontinue care at recruitment sites.

Table 1.

Characteristics of caregivers (N = 630)

¹ Other country of origin: Argentina, Brazil, Chile, Columbia, Costa Rica, Ecuador, El Salvador, Guatemala, Nicaragua, Peru, Panama, Puerto Rico, and Uruguay.

Overview

We incorporated quality improvement concepts, strategies, and tools from the Model for Improvement to enhance our recruitment procedures. The Associates for Process Improvement developed the Model for Improvement to improve processes and outcomes in diverse contexts, and it has been used widely in healthcare settings [16]. The model consists of two major components: (1) addressing three fundamental questions (What are we trying to accomplish?; How will we know a change is an improvement?; and, What changes can we make that will result in improvement?) and (2) implementing Plan-Do-Study-Act (PDSA) cycles to test changes. The model offers extensive strategies and tools to accelerate improvements [16]. Below, we describe how we addressed these three fundamental questions. We then provide illustrative examples of tests of change describing the factors that inhibited recruitment and the ways we used the Model for Improvement tools to help us overcome these barriers and reach our recruitment goals.

Forming the team

The interdisciplinary study team was formed over several research projects. Most members had longstanding relationships with each other, the CHCs, and the local community. The principal investigator (PI)’s research focuses on promoting health equity for Latino communities; as such, he had partnered with community members and the CHCs on other projects. The core study team meets at least weekly and includes the PI, research assistants (RAs), two research coordinators, and a data manager. The RAs conducting recruitment and enrollment were all bilingual; most had a familial connection to Spanish, were based in the area, and had connections to the university and/or community. One research coordinator who has experience running a family medicine residency research network oversaw CHC and clinician engagement. The other research coordinator, who had worked on the preliminary studies and other community-based projects, oversaw participant engagement and RA training. In addition to providing oversight of surveys and outreach techniques, he helped the RAs write a handbook outlining protocols and scripts. All RAs used this handbook during training and reviewed it regularly. Training included several mock outreach calls and study visits with real-time feedback. RAs shadowed the research coordinator during actual study visits; the coordinator then shadowed at least three of each RA’s outreach calls and study visits, which were followed by debriefing sessions. Once RAs demonstrated competency, they conducted visits independently. For the remainder of the study, RAs met one-on-one with the research coordinator weekly to review cases for quality assurance and met one-on-one with the PI at least monthly with more frequent contact via phone, text, and email.

We identified clinician champions at the three CHCs who served as important team members. For one site, two clinicians joined the research team as content experts on literacy promotion and community health, in addition to their expertise on day-to-day clinic operations. At the other two CHCs, one clinician served as study champion. The clinician champions raised awareness about the study, helped develop recruitment protocols, and met monthly with the study coordinator who oversaw CHC engagement to provide updates and feedback.

The remaining team members included experts in implementation science, health services, social science, linguistics, and statistics. Multiple members had run successful clinical trials. An advisory panel of members of community and professional organizations provided feedback at least annually.

Setting goals, establishing measures, and selecting changes

The initial proposal, which was developed before the pandemic, called for recruiting 630 parent–infant dyads in 2 years, which would require 6 enrollments/week. It took 2 years and 7 months to achieve this goal.

At the beginning of the study, we selected run charts to conduct time trend analysis and monitor progress toward our recruitment goals tracking the weekly total of clinician referrals, enrolled participants, and individuals who declined participation. We set up regular team meetings and check-ins with the CHCs to identify problems and brainstorm potential solutions in a participatory manner.

During weekly core team meetings, RAs presented the run charts, shared their observations, discussed barriers they encountered, and brainstormed potential solutions. For example, RAs observed that certain terms caused confusion (see case example “Describing the study in culturally appropriate language” below). The study coordinator overseeing CHC engagement, and an RA also met monthly with clinician champions to assess factors that influenced processes and referrals. These meetings helped the team understand patient volume changes, clinician scheduling changes, and turnover. The coordinator and RA kept detailed fieldnotes and reported updates to the PI and core team during weekly meetings. As described below, the PI eventually delivered quarterly report-backs to each CHC via Zoom on recruitment and study status to enhance clinician engagement and answer questions and to elicit recruitment challenges directly from the clinicians on the ground. Figure 1 summarizes the challenges we encountered in a cause and effect diagram [16]. Cause and effect diagrams, also known as fishbone diagrams, visually outline how issues at different levels contribute to an underlying problem, in our case, not meeting our weekly recruitment goals.

Figure 1.

Cause and effect diagram of factors related to not reaching weekly recruitment goals.

Based on the recruitment data and meetings, the team agreed on changes collaboratively. Information and opinions were gathered from participants during informal debriefs after study visits, the CHC champions, and from our other team members.

Testing and implementing changes

PDSA cycles were used to make modifications to the study processes. During weekly team meetings, the team would identify potential solutions to problems discussed, which would then be tested to determine if they were effective. In the following section, we provide specific examples. Application of QI tools allowed for data-driven improvements in study recruitment, reductions of participant burden, and integration of culturally appropriate language to facilitate the recruitment initiatives.

Adapting recruitment strategies to a virtual environment

We recruited parent–infant dyads from November 2020 to June 2023. Originally, we planned for bilingual RAs to conduct in-person recruitment in the three CHCs. However, this was not feasible at the start of the study due to physical distancing and safety guidelines. In response, we developed a virtual recruitment plan in consultation with clinician champions at the three CHCs. The protocol called for clinicians to introduce the study; study staff offered guidance on how to briefly introduce the study to parents and refer them to the study team if interested. Referrals were sent electronically to the study team using secure, encrypted email. Bilingual RAs called parents to provide information and screen for eligibility; they then set up a secure videoconference (i.e., Zoom) to conduct the informed consent process and enrollment. While most participants enrolled on Zoom, some enrolled by phone. The most common reasons were difficulty setting up the application and connecting to the meeting.

Increasing referrals from clinicians

The team piloted the virtual recruitment procedures in October 2020. Despite the pilot, we encountered challenges receiving referrals from clinicians. Figure 2 illustrates referral volumes throughout the study. After the initial spike in clinician referrals at the beginning of the study, the referrals decreased. Only certain clinicians were sending in referrals consistently and at lower rates than those needed to achieve 6 enrollments/week. Based on our previous work, we anticipated approximately 60% of individuals referred to the study would participate. Based on this estimate, we needed 10 referrals per week on average. However, by week 50 (November 1, 2021) the median number of referrals was just 2.5/week (Figure 2). The team decided that changes were needed to enhance clinician and staff engagement and tested multiple measures iteratively. These included monthly emails with recruitment statistics for each CHC and celebratory emails for holidays or events (e.g., Latino Heritage Month) that included a recruitment tip or note about the local community (e.g., the local library offers free library cards). One of the research coordinators worked directly with a CHC staff member who agreed to remind clinicians to make referrals. The PI also began to provide quarterly report-backs directly to the CHC clinicians. The frequency was identified collaboratively with the clinician champions to enhance engagement, but not increase clinician participation burden. While the research and clinical teams perceived that these changes enhanced rapport and communication, they did not have a sustained effect on increasing the number of referrals (Figure 2).

Figure 2.

Total weekly referrals.

Reducing perceived participation burden to enhance recruitment

Parents who declined to participate frequently cited limited time as the reason. Informal feedback from participants also suggested the enrollment surveys were too long (>1 hour). During meetings, the team discussed which survey questions were redundant or unnecessary. We then made changes to reduce the number of surveys and questions and tested both timing and quality of participant experience. Additionally, participants were given the choice of completing the surveys in more than one call, so that we did not take up too much of their time at once.

The RAs conducted a series of time trials in a PDSA cycle to assess survey length and determine the longest duration acceptable to participants. They noted points at which participants seemed to lose engagement. Additionally, with more practice, the RAs gained greater confidence and efficiency. These efforts resulted in decreasing the median administration time from 80 minutes from January to February 2021, to 54 minutes from October to November 2021, and finally to 41 minutes in May 2022. RAs were able to tell prospective participants that the surveys would take under an hour to complete. These changes corresponded with a gradual increase in the percentage of individuals who agreed to participate, with the median number of weekly enrollments peaking at 8 per week by March 2023 (Figure 3).

Figure 3.

Total weekly enrollments.

Adapting recruitment strategies to in-person environments

Upon relaxation of safety precautions in February 2022, our research staff was allowed onsite in two of the three CHCs as noted above. We worked with these clinics on protocols and began in-person recruitment at the first site in February 2022 and at the second site in May 2022. In-person recruitment enabled the team to reduce demands on the clinicians by explaining the study directly to participants and become a direct point of contact for the clinic staff. They were also able to describe the study to potential participants in more detail than clinicians could, establish a personalized rapport with participants, answer participant questions in real time, and in some cases, screen and consent participants onsite. This transition required modifications to IRB processes and study scripts. RAs needed flexibility to have more personalized approaches to building rapport with families but also to adapt to each CHC’s day-to-day routines to work unobtrusively alongside their staffs.

Describing the study in culturally appropriate language

Figure 2 illustrates the increase in referrals corresponding with in-person recruitment. However, with the increased number of referrals, we saw a corresponding increase in the number of individuals who declined participation (Figure 4).

Figure 4.

Total weekly declines.

In a collaborative manner, the RAs reflected on their interactions with participants with the core and larger team to identify potential reasons for an increase in the decline rate. The team reviewed and refined scripts and materials to enhance cultural and linguistic responsiveness. The introduction script was modified multiple times to eliminate unnecessary information, improve the English to Spanish translation, and clarify terminology that many parents misinterpreted. For example, many thought the word “estudio” (study) referred to a class they would need to take, or that the phrase “estudio de investigación” (research) implied biomedical procedures, like having blood drawn. Because the team could not remove these terms in accordance with IRB protocols, the team instead developed language to better explain the purpose of the research project and what would be expected of participants. The team modified the recruitment flyer multiple times during the recruitment period to address recurring questions like these, and, importantly, to add visual elements and timelines to help prospective participants understand participation expectations. The flyer was posted in patient exam rooms throughout the three CHCs, with extra copies available to hand out to patients.