Background

Research participants’ experiences during study participation influence how they perceive research, whether they feel valued and respected, and whether they will enroll again or recommend participation to others [Reference Verheggen, Nieman, Reerink and Kok1–Reference Adler, Otado and Kwagyan4]. When queried, clinical trial participants say they would like to be asked for their feedback about research participation [Reference Kost, Lee and Yessis2,Reference Boyd, Sternke, Tite and Morgan5]. Academic medical centers and research organizations often routinely seek feedback about patient care, employee satisfaction, and institutional reputation, yet neglect to systematically and routinely seek feedback from their research participants.

The lack of large-scale validated outcome data about participants’ research experiences presents a barrier to translational science [6]. Persistent challenges include recruiting and retaining participants, addressing health disparities, and ensuring diverse and representative enrollment in clinical studies [Reference Smedley, Stith and Nelson7–9]. Federal agencies and others have long encouraged diverse enrollment in clinical trials, and recent changes to federal regulations have increased transparency and focus on representative enrollment [Reference Menikoff, Kaneshiro and Pritchard10,11]. However, minoritized populations, women, individuals living with health disparities and disabilities, and older adults continue to be underrepresented in clinical trials [Reference Hlávka12,Reference Flores, Frontera and Andrasik13]. Underrepresentation of specific groups arises from cultural and language barriers, potential mistrust or distrust of institutions or study staff, lack of awareness of research opportunities, and limited access to research; as a result, addressing these challenging issues requires layered and tailored participant-facing solutions [9,Reference Cunningham-Erves, Kusnoor and Villalta-Gil14–Reference Smirnoff, Wilets and Ragin20]. When members of these groups do participate in research, there is little information gleaned about their perspectives about specific experiences. Systematically evaluating which practices best facilitate and sustain equitable research participation is important for advancing research equity and reducing health disparities.

Informed consent is the keystone of ethical research [Reference Grant21–23]. Yet, achieving effective informed consent for participants is challenging given its complex requirements and forms, cultural and linguistic barriers to communication, and varying provider and study staff skills [Reference Kadam24 25,Reference Cohn and Larson26,–Reference Flory and Emanuel27]. An emerging consensus recommends a person-centered consent process, made available using multiple modalities, sensitive to individual context, and developed with community engagement [Reference Cunningham-Erves, Kusnoor and Villalta-Gil14,Reference Bona, Utecht and Kemp28–Reference Spinner and Araojo32]. To this end, plain language glossaries [Reference Baedorf Kassis, White and Bierer29,Reference Krol, Kim, Gao, Amiri-Kordestani, Beaver and Kluetz33] and multiple other tools have been developed to simplify and personalize the informed consent process [Reference Bona, Utecht and Kemp28,Reference Lawrence, Dunkel and McEver31,Reference Landi, Mimouni and Giannuzzi34]. However, there is little evidence demonstrating the broad and effective adoption of these tools or their generalizability and impact. A recent review of informed consent forms in the ClinicalTrials.gov registry found that most included language that exceeded the recommended reading level [Reference Gelinas, Morrell, Tse, Glazier, Zarin and Bierer35]. The Association for Accreditation of Human Research Protection Programs (AAHRPP), requires that organizations seek and respond to participant feedback [36], and the federal Office of Human Research Protections recently issued new training to make consent more participant-centered [37]. Although AAHRPP’s Standards (1.4, 1.4B) concern informed consent procedures, currently, there are no widely recognized outcome measures or benchmarks to evaluate or enforce whether procedures for informed consent are effective. A recent study of how IRBs measured the effectiveness of their practices found an overreliance on process measures and few outcome measures.[Reference Fernandez Lynch and Taylor38] An independent research ethics consortium recommended incorporating participant-centered measures into evaluating consent effectiveness [39]. In their 2023 report, the US Government Accountability Office recommended that the Department of Health and Human Services consider surveys of research participants as a potential measure of the effectiveness of human research protections [40]. Thus, a systematic participant-centered approach is needed to evaluate whether participant-directed practices are achieving their intended impact, that is, do they enhance trust and fairness, are they effective and respectful, and do they ensure that discoveries are developed to benefit all affected populations [Reference Stallings, Cunningham-Erves and Frazier19,Reference Smirnoff, Wilets and Ragin20].

To address the need for patient-centered outcomes measures of the clinical research enterprise, we developed the Research Participant Perception Survey (RPPS) using rigorous mixed methods and a highly participatory process to capture what participants identified as important aspects of positive or negative research experiences [Reference Kost, Lee, Yessis, Coller and Henderson41,Reference Yessis, Kost, Lee, Coller and Henderson42]. In a large-scale validation study designed to minimize participation bias, outcomes from 4960 survey responses at 15 NIH-funded research institutions revealed one-time benchmarks for research experiences and a range of scores across sites [Reference Kost, Lee and Yessis2]. A shorter RPPS version was designed to be easier to deploy. It was validated and produced higher response rates and reinforced prior findings [Reference Kost and de Rosa43,Reference Kelly-Pumarol, Henderson, Rushing, Andrews, Kost and Wagenknecht44]. Several organizations adopted the RPPS, routinely using results for internal quality improvement, but did not publish outcomes, and broader adoption was limited. In 2020–2024, through the Empowering the Participant Voice (EPV) initiative funded by the National Center for Advancing Clinical and Translational Science (NCATS), we built, tested, and freely shared tools, infrastructure, and standards to streamline institutional adoption of the RPPS for eliciting participants’ feedback. We also developed a dashboard for quick scoring, filtering outcome data by participant and study characteristics, and supporting data sharing [Reference Kost, Cheng and Andrews45,Reference Cheng, Moragas and Ellis46]. We implemented the RPPS and related infrastructure in locally customized use cases at five CTSA institutions to collect participant experience feedback and aggregate results to a multisite dashboard as previously reported [Reference Kost, Cheng and Andrews45]. Here, we report the survey results, outcomes, and benchmarks from the RPPS responses of 5024 participants across seven CTSAs, and preliminary impacts from responsive initiatives. These data build on the existing participant experience evidence base to inform local and consortium action to improve research.

Methods

The Rockefeller University Institutional Review Board ruled the analysis of deidentified survey data Exempt from review. Sites conducted local surveys under the approval or exemption of their local IRB.

The RPPS survey teams included a project PI (research faculty, human protections, or research leadership), a project manager with recruitment, outreach or evaluation expertise, and informatics specialists.

Results

From February 2, 2022 to June 15, 2024, seven CTSA sites sent the RPPS survey to 28,096 research participants and received 5045 responses.

Response and representativeness

The overall survey response rate was 18%, ranging from 12% to 53% across sites. The highest response rates came from institutions fielding targeted surveys on a study-by-study level, or in-person at point-of-care. The lowest response rate came from the site utilizing the patient portal to deliver invitations. Group-specific response rates varied by age (ranging from 7% for ages 18–34 to 24% for ages 65–74), by ethnicity (11% for Latino/x, to 41% when no ethnicity was reported), by race (9% for American Indian to 18% for White), and by gender (6% for nonbinary individuals, 16% for women, 20% for men). Of surveys started (at least one question answered), 97% were completed.

The characteristics of survey respondents, survey recipients (including nonresponders), and the US population are shown in Table 1. Compared to all survey recipients, Black individuals and participants ages 18–54 were underrepresented among survey respondents. Latino/x participants and individuals >age 65 were overrepresented compared to all survey recipients, though they are underrepresented in research overall compared to the census. Upon review of the demographics of the first ~ 2500 responses (through May 2023) [Reference Kost, Cheng and Andrews45] and in consultation with stakeholders, sites worked to increase survey awareness and expand the reach of the survey. In approximately equal cohorts from the first year compared to the subsequent year of survey fielding, the representativeness of Black participants increased from 11% to 15.6% and that of Latino/x participants increased from 3.6% to 9.8%. To a lesser extent, representativeness increased for the youngest participants, Asians, individuals with less educational attainment, and disease-affected individuals (as opposed to healthy volunteers) (Table 1). Individuals with 4 or more years or years of college education (61%) were overrepresented among survey respondents compared to the general population (38%) [52]. This reflects a disparity in research participation that has been well described by others [Reference Baquet, Commiskey, Daniel Mullins and Mishra53,Reference Scanlon, Wofford, Fair and Philippi54].

Table 1.

Characteristics of research participants who returned the Research Participant Perception Survey (RPPS) by year compared to all recipients of the RPPS survey and the US 2020 Census

M=million; *February 2022-May 2023; **May 2023- June 2024; *** include non-responders,^tAdjusted to reflect % of all adults (age 18 or older) and exclude minors.

Respondents were drawn from studies with a range of characteristics: two-thirds required a diagnosis or disease to enroll and one-third involved a drug, device, or lifestyle intervention (Table 1). Sixty percent of responses (n = 3190) included the institutional variable, “cancer center study (yes or no),” of which 49% were studies conducted in a cancer center. Consent for 20% of the respondents was conducted remotely, 47% in person, and 28% through a hybrid approach.

Research participation experiences

Topbox scores for the research experience questions with an optimal response (e.g., always), so-called “actionable” questions, are shown as mean scores for aggregated data, with site range, in Table 2. Most respondents (>90%) replied that they “always” felt treated with courtesy and respect, free from pressure to join a study, culturally respected, and had sufficient privacy. Slightly fewer participants (80–89%) gave the Topbox answer, “always,” for feeling listened to and free from pressure to stay in the study (if they said they had considered withdrawing). About three-quarters of respondents felt “always” valued as a partner in research, “always” knew how to reach the study team, and received language assistance that “completely” met their needs (if they needed it). Only two-thirds (64–65%) of respondents reported feeling “completely” prepared for the research experience by the consent form or the consent discussions or could “always” reach the team when they needed to. About two-thirds (68%) gave the overall experience the highest rating (9 or 10) or said they would “definitely” recommend research participation to others (61%).

Table 2.

Multisite Topbox scores for RPPS questions in aggregate with range across sites, and chi-squared / Fisher’s exact test (February 2022 – june 2024)

* Significant at 0.05 level;

** Significant at 0.01 level;¹Fisher’s exact test used;²One site did not have data for this question, chi-square was performed with df = 4.

Research experience scores varied across sites, sometimes by 20 percentage points or more. For 10 of the 14 actionable questions, intersite differences were statistically significant (p < .001). The lowest site scores were observed in questions that also showed the broadest intersite range, specifically for feeling completely prepared by the consent discussion for the research experience (54–74%) and always being able to reach the research team when a participant needed to (55–84%) (Table 2). One site had consistently higher scores than other sites across most of the questions. When the highest scoring site was removed from the analysis, all but two of the significant intersite findings among the remaining six sites retained their significance though the intersite differences in scores for “would recommend research to friends and family” and for “feeling like a valued partner in the research process” were no longer significant.

In a crosstab analysis, we compared the ratings of each of the actionable research experience questions to the overall experience rating of 9 or 10 (Topbox) to evaluate how top-rated experiences in other aspects of the research experience correlated with a “best” overall experience (Table 3). For all the questions, the top-rated experiences were significantly, but not exclusively, associated with conferring a top overall rating. When ratings for aspects of the research experience were lower, the overall experience fell dramatically in parallel.

Table 3.

Respondents’ overall ratings of their research experiences compared to their responses to questions about their research experiences (February 2022 –june 2024) (n = 5045)

*P-values were obtained via logistic regression models with the outcome being overall experience vs. each actionable question. Questions with “not applicable” response options did not include those who responded to the “not applicable” option.**For questions that include the response option “not applicable,” responses that are not applicable are not included in the denominator for scoring the actionable response options.

We sought to understand the characteristics of participants most closely associated with how they answered two questions: the overall experience rating and feeling like a valued partner in research. The results of the two mixed logistic regression analyses are shown in Table 4. For overall experience rating, age, gender, educational attainment, ethnicity, and the level of study demand were significantly correlated with conferring a Topbox score. Participants who were 18–34 years old, nonbinary/other gender, had higher educational attainment, or enrolled in a study with moderate or intense study demands were associated with significantly lower Topbox ratings for the overall research experience. Latino/x ethnicity was significantly associated with conferring a very positive overall rating. For the question about “Feeling like a valued partner in research,” age, gender, and study demands were significantly associated with the experience rating. Participants who were 18–34 years old, nonbinary/other gender, or in studies with moderate or intense demands were significantly less likely to select the Topbox rating for whether they always felt valued.

Table 4.

Mixed-effects logistic regression models for Topbox scores in overall rating and feeling fully prepared by the informed consent discussions

* Site is included as a random effect.

In a univariate analysis using the filter and visual analytics display of the dashboard, it appeared that individuals who had completed education at the level of a high school diploma or GED or less returned significantly lower scores (>10 percentage points) for their consent experiences. However, when controlled for the demands of the study, this finding was no longer significant, highlighting the need to examine multiple variables before acting on preliminary findings.

Timing and outcomes

Topbox scores differed significantly (p < .05) for some questions depending on survey timing. Feeling prepared by the consent discussion, receiving language assistance, and successfully reaching the study team when needed scored lower in surveys after a year of participation (long studies) compared to surveys collected postconsent or at end-of-study.

Qualitative responses

Six of the seven sites included the standard survey question asking participants to rate a list of factors for their importance when considering joining a future study, resulting in data from 90% of all respondents. The factors most frequently rated as “Very Important” were the “Return of a summary of the overall results of the study” (62.5%), followed by the “Return of personal test results” (45.2%), “A more flexible visit schedule” (28.4%), “Accessible parking” (27.6%), and “Payment/More payment” (24.7%) The “Return of a summary of the overall study results” was the top highly rated choice for participants at five of the six sites fielding this question, and second highest at the remaining site, and was among the top two choices when results were filtered by participant or study characteristics..

Open text responses

To preserve site confidentiality, open text responses were not aggregated centrally, though sites shared anecdotes with the steering committee and indicated that free text provided valuable feedback data. Typical feedback included praise for specific staff, identification of specific actionable issues for sites tracking study-specific response data, and feedback about issues useful to drive broader institutional change.

Language

Few respondents (83, 1.6%) completed the survey in Spanish, of whom 97.5% identified as Latino/x, 34.4% identified as Black or African American, 70.5% identified as White, 1.6% Asian, 4.9% Native American, 3.3% Native Hawaiian or Pacific Islander, 60% were age 65 or older, 75% identified their gender as a “woman,” 7.5% as a “man,” 2.5% nonbinary, and 15.2% said none of these terms describe me or prefer not to say. In the Spanish language cohort, 37.8% said they “always” received adequate language assistance, 41% said they had no language issues, 4.6% said their language needs were “never” met during participation, 31% had not graduated from high school, 23% had graduated high school, 23% had completed some college, and 22% had completed college or graduate education. In contrast, of participants who completed the survey in English, only 1% said their language assistance needs were “never” met and 63% had graduated from college.

Acting on findings

Sites evaluated local findings and piloted interventions or innovations to test whether they could improve specific experience scores. Leadership called this impact return on investment (ROI) (Table 5). Acting on findings, sites were able to enhance accrual and satisfaction with communication (Site A), leverage actionable participant-centered feedback to accelerate an institutional decision, and drive the design of clinical translational science (Site B), and use incentives to improve response rates (Site C). Upon reviewing their findings, the Cancer Center leadership at one site (Site C) requested a new variable to compare results among cancer center studies across sites, which the EPV steering was able to implement for existing and future data. As of September 2024, the variable describing “cancer center study (yes/no)” was available for >80% of responses in the aggregate dataset.

Table 5.

Local research experience findings, actions, and impacts

Site D has a history with the survey that predates the EPV project affording a longitudinal view of impact. In response to the finding that only 65% of participants reported “always” feeling like a valid partner in research, the institution initiated and sustained a campaign to communicate to participants that they are valued partners in the research enterprise, achieving significantly and sustainably increased scores for always feeling valued (Figure 1). Within the same time period, a study team undertook an initiative to improve the consent process for a study with a complex intervention. They engaged prior study participants to help develop a video explaining the study intervention. After implementing the video, scores rose for feeling prepared by consent, as scores for feeling valued and other experience ratings continued to rise.

Figure 1.

shows the Topbox scores for three Research Participants Perception Survey (RPPS) experience questions from 2013 to 2024 at Site D, where the RPPS has been fielded for a decade. In 2013, the site began an initiative to communicate directly to research volunteers that they were valued by researchers and the institution as partners in the research process (blue arrow). Initially communicated through brochures, pins, and banners, over time, messaging was also incorporated into institutional values through training, teaching, and policy. In 2017–2018, a research team with many RPPS respondents enlisted participants to help develop a new informed consent video and began using it in a Phase I–II study (orange arrow). In 2020–2022, the COVID pandemic disrupted many clinical operations (green arrow), including in-person consent, with full recovery of in-person activities by 2023.

Site E noticed a trend of lower scores regarding receipt of language assistance that tracked with higher participant age. In free text, several participants mentioned using hearing aids and having difficulty understanding telephone assessments or consent conversations conducted with masks on. The team consulted with their Center for Healthy Aging to design and test innovations to make written and verbal research communication more accessible for participants with hearing or vision impairments. These and other results spearheaded the creation of a new permanent Committee for Equity in Research, consisting of stakeholders to participate in survey analysis and the design of institutional responses.

Site F is leveraging its strength in paying attention to issues of literacy (and significantly higher site score regarding language assistance) and participant feedback about receiving results to design a research program for returning study results that fulfill participants’ preferences.

Conclusion

Feedback from 5045 research participants in the EPV project contributed to a growing evidence base describing the experience of research participation and enabled comparisons among subgroups and sites. Most participants felt respected and listened to; the majority say that receiving a summary of the overall study results will be very important to future decisions about research participation. There is significant room for improvement in informed consent, in conveying to participants that they are valued partners in research, and in improving the aspects of the experience that contribute to a high overall experience rating. Benchmarks from aggregated data enabled sites to compare their local RPPS findings to multisite norms. Sites that implemented actions in response to local RPPS findings were able to increase participants’ scores for feelings of being valued, increase speed of enrollment, improve communication with the study team, and design practices more responsive to participant preferences. RPPS experience data can be used to effectively drive quality improvement in clinical translational research.