Introduction
The Latinx population in the USA is the nation’s largest racial minority group, yet health disparities persist within this community [1]. It is essential that healthcare advances are made to improve this population’s health and to reduce disparities. Improvements in healthcare delivery and outcomes necessitate clinical research to advance the science of health; and clinical research requires equitable participation by different populations, based on age, sex, gender, race, and ethnicity to ensure generalizability of findings and/or to determine if different populations have different responses to or needs for health care, all with the goal of reducing health disparities. The National Institutes of Health (NIH) Revitalization Act of 1993 established guidelines to ensure inclusion of women and minority populations in clinical research to achieve this goal. Since then, while inclusion of women and certain race groups has improved, the reporting of and the inclusion of Latinx populations in clinical research in the USA continue to lag, and in fact, may have decreased since the passing of the Revitalization Act [Reference Burke, Brown, Lisabeth, Sanchez and Morgenstern2–Reference Soomro, McCarthy and Varela4].
Prior studies have examined the barriers and facilitators to participation in clinical research by Latinx populations [Reference Dreyfus, Kuri and Ferri5]. Barriers include the costs and time required for participation. Facilitators include trust in the research team, language concordance, and family support. These studies of facilitators and barriers are generally based on intervention studies designed to test specific tactics to improve recruitment or retention of underrepresented populations, including Latinx populations, while fewer studies are observational and designed to report the perceptions and attitudes of Latinx populations related to research more generally [Reference Dreyfus, Kuri and Ferri5]. Ongoing assessments and improvements in research processes and inclusivity are needed, particularly for Latinx populations, to enhance their experiences and ultimate trust in clinical research.
The Research Participants’ Perceptions Survey (RPPS) is a validated tool developed to capture key aspects throughout the experience of research participation in order to assess how research processes are being received by participants and to understand potential areas of improvement [Reference Kost, Lee, Yessis, Coller and Henderson6,Reference Yessis, Kost, Lee, Coller and Henderson7]. The validation of the RPPS included Latinx participants. The RPPS is available in English and Spanish [Reference Yessis, Kost, Lee, Coller and Henderson7], can be administered at the health system or individual study level, and survey findings can be used to tailor research programs for different populations. The RPPS is an evaluation tool that can be used to guide and evaluate efforts to improve the research participation experience, including those from populations that are underrepresented in clinical research.
The Salud Estres y Resilencia (SER) (Health Stress and Resilience) Hispano study is a longitudinal community-engaged research study that was developed in order to guide interventions that may help reduce health disparities among the Latinx community [Reference Nagy, Solorzano, Stafford, Mercado Emerson and Gonzalez-Guarda8,Reference Gonzalez-Guarda, McCabe and Nagy9]. This study was designed collaboratively by an academic research team and a community-based organization to reduce some of the known barriers of research participation for a cohort of young adult Latinx immigrants. Some of the tailored features of the SER Hispano study were to include researchers and research staff who were bicultural and bilingual as well as careful attention and effort towards the translation and acculturation of study-related materials in both English and Spanish.
Towards the end of the SER Hispano study and in collaboration with a project to help institutions streamline RPPS administration, the Empowering the Participant Voice (EPV) project [Reference Kost, Cheng and Andrews10], we administered the RPPS to SER Hispano study participants to gather information on their research experiences to help determine if the design and conduct of the study were successful in creating a positive research participation environment for its participants and to identify additional measures that could improve the experience and recruitment of the Latinx population, a population historically underrepresented in research, in future research studies. Here, we describe the administration and results of the RPPS from the SER Hispano study cohort.
Materials and methods
Research participant perceptions survey
The RPPS was developed in 2012 through a multi-institutional collaboration, using mixed methods and a participant-centered approach [Reference Kost, Lee, Yessis, Coller and Henderson6,Reference Yessis, Kost, Lee, Coller and Henderson7]. The survey assesses key aspects of the research experience as identified by participants and validated through psychometric testing. The RPPS examines the participant experience across the life of a study and was developed to collect measures that could be used to drive evidence-based improvements in research processes, participant experiences, and overall satisfaction for research participants [Reference Kost, Lee, Yessis, Coller and Henderson6,Reference Yessis, Kost, Lee, Coller and Henderson7]. After initial validation of a long version, subsequent shorter survey versions were developed and validated [Reference Kost and de Rosa11]. For the purposes of this study, we utilized the RPPS-Short survey (RPPS-S) instrument. The RPPS-S version used in the EPV project is made up of six questions that collect characteristics of the respondent (age, sex, gender, race, ethnicity, education level); two questions that assess the experience broadly, with a 0 (worst) to 10 (best) numerical scale to rate the overall experience and a Likert rating of whether they would recommend research participation to friends and family; actionable questions, including six key questions that, during validation, were shown together to account for 96% of the variance in the Overall Rating score, including questions related to informed consent, being listened to, being treated with courtesy and respect, and knowing how to reach study staff [Reference Kost and de Rosa11,Reference Kost, Lee, Yessis, Wesley, Henderson and Coller12]; additional questions related to the study experience; and a question related to future research participation. All survey versions, including the one administered as part of the EPV project, include an open text field at the end of the survey to collect additional comments from the respondent.
Description of the SER Hispano cohort study
The SER Hispano study is a longitudinal community-engaged cohort study of 391 young adult (ages 18–44 at the time of recruitment) Latinx immigrants in North Carolina whose predominant language was Spanish [Reference Nagy, Solorzano, Stafford, Mercado Emerson and Gonzalez-Guarda8,Reference Gonzalez-Guarda, McCabe and Nagy9]. The study excluded non-Spanish-speaking Latinxs, e.g. Brazilians or any indigenous Latinx populations that do not speak Spanish. The cohort was followed over two years and focused on studying the effects of acculturation stress, resilience, stress biomarkers, and physical, behavioral, and mental health outcomes [Reference Gonzalez-Guarda, McCabe and Nagy9].
Participants in this cohort study were notified to expect to receive a link to the RPPS-S survey through the quarterly SER Hispano newsletter that they received as part of study participation, either at in-person visits or via email for those who agreed to this method of contact. Survey notification in the newsletter was provided in English and Spanish. Along with the newsletter content, flyers were provided to those participants who had remaining in-person visits, alerting them of what to expect when completing the survey (e.g. examples of questions included in the survey).
Survey administration
The RPPS-S was distributed to 340 participants who agreed to be contacted for participation after the SER Hispano study procedures were completed. The survey was initially sent via email with a link to complete the survey in REDCap. The email, which was sent out in Spanish, included an invitation for all participants to complete a brief survey that would gather information about their experience participating in the study. The email invitation contained specific mention of the SER Hispano study and the name of its principal investigator to ensure that participants had a clear understanding of the research study for which they were completing the survey. Finally, the email included a link directing the participant to REDCap to complete the survey in English or Spanish. The initial email invitation to complete the survey was followed by two subsequent reminders, each a week apart, if the survey was not yet completed.
Those participants who did not complete the survey online after the initial email and reminders, and who had agreed to be contacted by phone, were subsequently contacted by phone (two attempts) to complete the survey. Administration of the survey via telephone call was conducted by a research staff member who was not involved in either the SER Hispano study or the EPV project. Responses given to the survey over the phone were entered into the REDCap database by the phone survey administrator.
Statistical analyses
We describe baseline characteristics of our cohorts of interest with frequencies. In addition, we calculate response rates to the RPPS-S questions overall and also response rates by email or telephone. The answers to survey questions are scored by the frequency of the optimal answer (Top Box scores) [Reference Yessis, Kost, Lee, Coller and Henderson7] and also by the frequencies of all responses. Responses to the main survey questions were compared between those who responded via email to those who responded by telephone using Fisher’s exact tests, assuming an alpha level of 0.01. Such a comparison was not conducted for the question related to future research participation or the free text response question.
Results
Of the 340 SER Hispano study participants who agreed to be contacted, 142 responded either by telephone or email, yielding an overall response rate of 42%. Of those that responded, 53 (37%) completed it via email and, through subsequent contact with those that did not respond via email, an additional 89 (63%) completed the survey via telephone. Of those completing the survey via the emailed REDCap link, 26 (49%) completed the survey in English; of those who completed the surveys by telephone, all chose to have the survey administered in Spanish.
Selected baseline characteristics of the SER Hispano participants at the time of the RPPS-S administration and of the SER Hispano participants who responded to the survey are described in Table 1; and demonstrated that the survey respondents were representative of the overall study cohort. Participants’ ages ranged between 18 and 54, with the majority (54%) being between 35 and 44 years of age and with the majority of participants being female (76%).
Selected baseline characteristics of Salud Estres y Resilencia (SER) Hispano and research participant perceptions survey (RPPS) participants
[-] Questions not asked in SER Hispano study, so no responses available.
Most participants identified with their country of origin, the majority being of Cuban origin (50%), followed by Other (46%). Most participants did not identify with a specific racial group and chose to only respond to the ethnicity question via both email and telephone. For those who identified as Latinx and also responded to the race question (n = 37), most (n = 32) identified as White. The question asking “What is the highest grade or level of school that you have completed?” was met with confusion from phone participants about the categorization of different educational levels. For phone participants, the survey administrator translated the question in Spanish as “How many years were you in school for?” and then converted the number of school years stated into the equivalent US educational grade level. More respondents (26%) indicated that the highest level of education achieved was high school graduate/GED or equivalent, 18% had more than a 4-year college degree, 18% had a 4-year college degree, 14% had some college education, 13% had some high school education, and 10% had 8th grade or less.
Responses to the RPPS-S questions are shown in Table 2. Participants were asked two questions to assess their overall experience in the research study, and from their responses, we found that participants generally had a very good experience, and we found no significant differences in responses via email versus telephone. When asked to use a scale to rate their overall experience in the research study, 0 being the worst possible experience and 10 being the best, 80% of participants responded with either a 9 or 10 (the defined Top Box score), with no significant difference in response by method of survey administration (P = 0.67). When answering the question “Would you recommend joining a research study to your family and friends?” most respondents (98%) selected either “definitely yes” (84%) or “probably yes” (14%), again with no significant difference in response by survey administration (P = 0.45).
Research participant perceptions survey (RPPS) responses from Salud Estres y Resilencia (SER) Hispano study participants
1 Fisher’s exact test of significance, P < 0.01.
* Considered as “Top Box” response for question.
For the actionable questions, survey respondents overall expressed positive experiences, with no significant differences in responses by survey administration for questions related to informed consent, respect, privacy, cultural and language differences. When asked if they knew how to reach the research team when participants were not on site and if they were able to reach the research team when they wanted, significantly more phone respondents gave their experiences the highest rating compared to email respondents (P<0.01 for both questions).
For the other questions related to their study experiences, there were generally no differences in responses by survey administration method. However, more participants described the demands of the study as being “moderate” or “intense” when answering by phone rather than “simple,” which was the more common answer for those responding by email (P < 0.01).
As part of the RPPS-S, respondents were asked to select what factors would most influence their participation in a future research study (Figure 1). Of the 142 respondents, 72% selected “Summary of overall research results shared with me” 69% chose “flexible schedule;” and 62% chose “results of personal lab tests shared with me or my doctor.”
Factors influencing future research participation1.
1 Responses from 142 Salud Estres y Resilencia (SER) Hispano study participants to the question: “which of the following things would be important for you in a future study? Please check all that apply.”
Finally, survey respondents had the opportunity to respond to a free text option to share anything additional about their experience in the study. There were 50 free text comments submitted. The main themes of the comments included requests to receive the results of the study (either aggregate or personal) and requests to be contacted to participate in future studies.
Lessons learned from administering survey by phone
Survey participants who were contacted by phone immediately wanted to know the duration of the survey before agreeing to participate. Surveys conducted over the phone were found to take between 5 to 7 minutes to complete. Multiple choice and check-all-that-apply questions required repetition by the survey administrator, as many participants would forget the original question that was being asked and the provided answer choices. Yes/no questions were found to require less repetition for participants and the survey administrator. Some questions were perceived as open-ended questions and gave participants room to elaborate, which led to some difficulty for the survey administrator to categorize their response with an answer choice. This was a limitation to fielding the survey by telephone, as there were no plans in place to capture open-ended responses verbatim.
Discussion
In this study, we describe the methods and results of administering the validated Research Participants’ Perceptions Survey-Short (RPPS-S) to Latinx immigrant research participants enrolled in a longitudinal cohort study, the SER Hispano study, which was specially designed to overcome many of the barriers that this population faces related to research participation. The involvement in clinical research of Latinx populations is currently low. People of Latinx ethnicity currently make up over 19% of the US population ; however, currently in the USA, only 11% of trial participants are of Latinx ethnicity, with many studies recruiting far less [Reference Dreyfus, Kuri and Ferri5]. While efforts have been made to increase Latinx representation in clinical research, including the 1993 NIH Revitalization Act, in certain measures, participation in clinical research among this population has actually decreased [Reference Burke, Brown, Lisabeth, Sanchez and Morgenstern2]. Identifying potential barriers and facilitators to clinical research participation for Latinx populations and then addressing these to encourage participation are essential for advancement of health discoveries. Recognizing the importance of research participation by Latinx populations, in its study design and conduct, the SER Hispano study attempted to address known barriers for this population; the RPPS-S survey was administered to SER Hispano study participants to help determine if these attempts were successful.
Administration of the RPPS-S to the SER Hispano Latinx immigrant cohort posed some unique considerations which had not been previously described in the use of this survey. Investigators leading the SER Hispano study anticipated that distribution of the survey via email might lead to a lower response rate compared to other populations. Prior studies of the RPPS, primarily distributed via traditional mail, describe a response rate of about 30% overall in populations of mixed race/ethnicity with about 5% of those study populations being of Latinx ethnicity [Reference Kost, Lee, Yessis, Wesley, Henderson and Coller12,Reference Kost, Lee and Yessis13]. Comparison of the administration of the survey via different methods has been previously examined at a single institution with a mixed population of races (21% of Black race; 4% of Hispanic ethnicity) [Reference Kelly-Pumarol, Henderson, Rushing, Andrews, Kost and Wagenknecht14]. That study found that response rates were also approximately 30% when distributed via traditional mail, electronic health record’s patient portal, or when administered over the phone, but that email distribution led to a lower response rate of about 15% [Reference Kelly-Pumarol, Henderson, Rushing, Andrews, Kost and Wagenknecht14]. In that study, 33 participants were of Latinx ethnicity and were sent the survey by one of the 4 methods; 7% of these participants receiving the survey by mail responded, while 30% of those that were called responded to the survey over phone [Reference Kelly-Pumarol, Henderson, Rushing, Andrews, Kost and Wagenknecht14]. In our study of the SER Hispano study cohort, we had similar response rates; email distribution of a REDCap link to the survey resulted in a 16% response rate, while telephone administration of the survey resulted in a 26% response rate.
The experiences among this research participant cohort of Latinx immigrants in the SER Hispano study were overall very positive. Of RPPS-S respondents, 80% gave a Top box score for Overall Rating of their research experience (ratings of 9 and 10), which is higher than the Overall Rating scores from participants of all races/ethnicities in other studies at our institution which was 68% (unpublished data). The SER Hispano participants also rated their Overall Experiences higher than the mean score for responses aggregated by the EPV consortium, but similar to responses from Latinx participants in the EPV consortium database (78% Top Box score) (manuscript in preparation). Moreover, 84% stated that they would “definitely” recommend research participation to friends and family, which is higher than responses given by participants of other studies at our institution overall (60%) (unpublished data). This positive response rate is also higher than response rates from the EPV consortium database overall (61%) and from the Latinx participants of the EPV consortium database (67%) (manuscript in preparation). The SER Hispano study was designed with the goal to overcome barriers that Latinx populations often face, with a research team which was comprised of study staff of diverse backgrounds, who were Spanish-speaking, with study documents which were carefully translated into Spanish, taking into account content and context, and with study procedures that were developed with and for the community that they were studying [Reference Nagy, Solorzano, Stafford, Mercado Emerson and Gonzalez-Guarda8,Reference Gonzalez-Guarda, McCabe and Nagy9]. Based on the results of the RPPS-S of the participants of the SER Hispano study, it is evident that the study design and conduct were successful in making the research experience a good one for this population; and study teams planning to work with similar populations could use similar methods.
The RPPS-S was administered in two ways to the participants of the SER Hispano study, via email as a REDCap link, with the option of completing the survey in either English or Spanish; as well as via telephone, with a Spanish-speaking staff member. The preferred language was Spanish for all participants who were contacted by phone and for almost half of the participants who responded by email, which highlights the importance of offering research-related documents in languages other than English for participants whose first language is not English. The process of translation must take into account not only literal meanings but also cultural and linguistic meanings, depending on the population being studied [Reference Nagy, Solorzano, Stafford, Mercado Emerson and Gonzalez-Guarda8]. An example of this is the question that inquired about highest grade or level of school completed which was met with confusion from phone participants, possibly due to the majority of participants’ education being attained outside of the USA. The RPPS has been translated into neutral broadcast Spanish by a team of professional translators at the University of Texas Southwestern [Reference Yessis, Kost, Lee, Coller and Henderson7] and may benefit from review when administered to different populations.
There was some concern that there might be differences in responses for this population by the method of survey administration, with a particular concern for a social desirability bias when administered via telephone [Reference Oakley, Lopez-Cevallos and Harvey15–Reference Brown17]. However, we found that for the majority of questions, there were no statistically significant differences in responses when administered via email or via telephone. With both methods, participants expressed high satisfaction with their research experience and expressed that they would encourage others to participate in research. With both methods of administration of the survey, high proportions of participants expressed being treated with courtesy and respect, being listened carefully to, and expressed that there were no language differences. However, there were some statistically significant differences in responses by method of administration for certain questions. For the question “How much did the study demand of you?,” more of those that responded by telephone expressed that the study had moderate to intense demands, while the majority of those that responded by email expressed that the study was relatively simple. And, for the questions related to being able to contact study staff when they were not at study procedures, more of the participants who answered by phone expressed that they were “always” or “usually” able to reach study staff compared to those responding via email, and more of the participants who responded by email compared to those answering by phone said that they did not need to reach the study staff when not at the research site. These questions, particularly those related to being able to reach study staff, could potentially reflect some social desirability bias, perhaps, in order to benefit the research staff.
When asked about factors that would influence their decision to participate in future studies, the most frequently chosen factor for this study population of Latinx immigrants was having the overall results of the research shared with them. Additionally, in the free text response question, participants reiterated the importance of receiving results from studies. This finding is similar to that found in studies of the administration of the RPPS survey to research participants from other academic centers, as well as from other surveys of research participants [Reference Kost, Lee, Yessis, Wesley, Henderson and Coller12,Reference Kost, Lee and Yessis13,Reference Shalowitz and Miller18] (and manuscript in preparation). While the Final Rule mandates that clinical trials do have to register and share results on clinicaltrials.gov [19], there is no current mandate that the summary of research study results has to be shared directly with the research participants in plain language or in the primary language of the participant. Sharing of summary results of any research study with their study participants would be a way to improve the research experience for all populations, including Latinx populations.
This study demonstrates that when a research study, such as the SER Hispano study, is designed thoughtfully, in collaboration with the population being studied, research participation by a Latinx population can be successful and a good experience for them; and this study demonstrates that this positive experience can be captured by the RPPS-S. There are limitations of this study, however, which include the fact that this is a single study, which limits the generalizability of the findings. Within this single study, the study participants had differences in years lived within the USA and differences in countries of origin, as well as differences in other factors which could influence their experiences, and whose impact on responses was not analyzed. In addition, within the SER Hispano study, not all participants responded to the survey, and it is possible that some participants did have negative experiences or experiences which were not captured. However, through the administration of the survey by both email and telephone, we were able to get responses from a higher proportion of participants than in prior studies of the RPPS survey, which have generally administered the survey using only one method. Finally, in our statistical reporting of differences in responses to the RPPS-S by email or telephone, we did not adjust for multiple comparisons; however, we did choose a more stringent cutoff for the P-value of significance at 0.01 to help account for this.
In this study, we describe the research experiences of participants in a cohort study of Latinx immigrants as measured by the RPPS-S survey. We found that to optimize response rate, more than one method of survey administration, including telephone administration, was helpful; and we found that responses generally did not differ by method of administration. We found that research participation of Latinx populations can be successful and can be a positive experience for them, particularly when the study is designed in ways that accommodate their cultural and language needs. Additionally, we found that sharing the results of the research with participants is highly important to them and could demonstrate the value of their participation, improve trust, and encourage future participation.
Acknowledgements
We acknowledge and appreciate the study team members, Norma Garcia Ortiz and Maralis Mercado Emerson, and participants of The SER (Salud, Estrés y Resiliencia/Health, Stress, and Resilience) Hispano Project that participated in the Research Participant Perceptions Survey.
Author contributions
SL, JR, RGG, and RC were involved with the conception and design of the work. SL, JG, AMV, and RGG were involved in the collection of data. JG, RC, AC, RGK, and RGG performed data analyses and provided expertise. SL, JR, JG, AMV, MDM, AC, RGK, RGG, and RC contributed to the interpretation of analysis and drafting and review of the manuscript, among other critical intellectual contributions. RC takes responsibility for the manuscript as a whole.
Funding statement
The SER Hispano Project is funded by funded by the National Institute on Minority Health and Health (R01MD012249, PI: Gonzalez-Guarda, R.M.). This work was also supported by a Collaborative Innovation Award from the National Center for Accelerating Translational Science U01TR003206 to the Rockefeller University, and by Clinical Translational Science Awards UL1TR001866 (Rockefeller University), UL1TR002553 (Duke University), UL1TR002243 (Vanderbilt University).
Competing interests
None.
Open access
