(1) Changes to study materials

Before disseminating any participant-facing materials regarding the ACTIV-6 platform trial, SAC members reviewed documents and provided feedback from their diverse perspectives. This feedback was used by the core research team to iteratively refine these materials, which included recruitment fliers, websites, enrollment forms, study surveys, fact sheets and instructions on taking study medications, and lay-language summaries of trial results. This iterative process was repeated often throughout the operation of the trial. SAC members most often addressed two domains in their feedback:

(2) Recommendations on improving and maintaining patient-centeredness

Throughout the trial period, SAC recommendations emphasized a need for improving and maintaining patient-centeredness in the trial. These recommendations included viewing all study materials, actions, and requirements through the lens of a potential participant. Because SAC members often had experience with having had COVID-19 themselves, they were able to remind the rest of the study team that potential participants – by definition people in the early stages of infection with COVID-19 who were experiencing symptoms – were feeling unwell at the moment that they considered enrolling, and might be feeling anxious or scared as well. As such, it was important to consider simplicity and ease of participation in all contexts, a concept that was adopted into website designs, simplification of forms, and requests for participant information. SAC feedback of this sort resulted in more streamlined study forms and easier-to-navigate websites, as well as simplification of language. Additionally, SAC members kept the rest of the study team grounded in the sorts of actions a potential participant might take, and their likely outcomes. For example, when preparing materials related to ivermectin, an SAC member did a simple web search for “ivermectin” and found that the first result was an FDA website telling people not to take ivermectin for COVID-19. This emphasized the need for the study to address the controversy over ivermectin with trial participants.

(3) A focus on diversity in recruitment

SAC members routinely championed the need to focus on racial, ethnic, geographic, and economic diversity in the trials’ enrolled population. ACTIV-6 sought enrollment in the trial from a patient population mirroring the demographic diversity of the United States. When, at times, actual enrollment in the trial fell behind for some demographic groups, SAC members made suggestions to remedy the issue. Their routine championing of this cause inspired the NIH/NCATS and coordinating center to identify sites excelling in diverse recruitment, identify what they were doing that led to improved diversity in recruitment, and develop and present a series of webinars focused on diversity in recruitment. Study leadership also sought trial sites more likely to recruit or enroll from a diverse population. NIH/NCATS also partnered with the Community Engagement Alliance Consultative Resource (CEACR) team to provide suggestions for enrolling sites to improve diversity. Updates on participant diversity became a standard feature and discussion point of SAC meetings, and the SAC has watched diversity in trial participants (particularly with regards to members of the Hispanic/Latinx community) steadily increase as a result of the combined actions of NIH/NCATS, the trial coordinating center, and local site staff.

(4) Development of “thank you” materials and dissemination initiatives

As the trial neared its final year and prepared to close, SAC members were asked to consider how trial participants could be thanked. SAC members brainstormed a variety of ideas, including physical tokens of appreciation such as thank you cards and the production of videos tailored to study participants. Ideas for videos include continuing to focus on dissemination of results in an accessible fashion with a focus on explaining the scientific importance of null findings and the importance of diversity in recruitment for study generalizability. The videos are also each intended as a literal “Thank you” to trial participants in video format. Production of these “Thank you” items is ongoing, and at the time of writing is focused on the production of a public-facing video explaining the study results and what they mean in the current context, including explaining the importance of “null” results, which SAC members indicated might be confusing to community members.

(5) Internal evaluation of SAC

Although at the time of writing ACTIV-6 SAC activities are still ongoing, we conducted a survey evaluation of SAC members during September of 2023. Evaluation consisted of a Qualtrics-administered electronic survey utilizing the Ongoing/Long-term Engagement Questionnaire from the Public and Patient Engagement Evaluation Tool (PPEET) [19]. Of the 10 SAC members, 8 completed the survey. Full results, including all open-ended text responses, are presented in Table 2. As can be seen, overall satisfaction with Engagement in ACTIV-6 was high, with SAC members feeling that they had the opportunity to share their viewpoints and positively affect the trial. One survey respondent consistently answered every question as “Strongly Disagree,” but their corresponding open-ended text responses indicated that they were overall very pleased with participation, including the global comment “This experience has been great.” We suspect this respondent made an error in selecting their answers from the scale but have presented the data as collected. While one other member of the SAC did not feel that the study team took SAC feedback into account strongly enough, their open-ended text responses did not provide additional detail into why that was the case. Additionally, comments regarding what could have been improved centered on more communication with the rest of the study team, and occasionally different modes of communication, such as emails and in-person meetings. These responses highlight the importance of study teams incorporating stakeholders' input, and communicating which input is being incorporated in the study results and why, as advised in the following section. Overall, responses indicate that stakeholder experiences were positive and that engagement need not be flawless to be meaningful to stakeholders.

Table 2.

Full evaluation results. All open-ended text responses are presented in their entirety

Acknowledge the tension between transparency and privacy

Because the ACTIV-6 SAC operated entirely remotely and met once per month, we sought to centralize communications in a public-facing blog that SAC members could check regularly for updates. Initially conceptualized as a place to document SAC actions and share news internally the blog served as a public-facing document that chronicled the advice given by the SAC and resulting actions taken by the trial. This private-and-public-facing nature of the blog raised questions such as: Should the full names of SAC members be available on the blog, or might that constitute a breach of privacy for them? If their complete information was available, might someone from the public contact them about questions related to the trial – and if that happened, what authority and or duty might they have to respond? The SAC mutually decided to use first names only on the blog and to include photos only from SAC members who were comfortable with doing so. Additionally, because SAC members often provided feedback on documents and manuscripts that were not yet available in the public domain, it was not possible to share even highly redacted versions of the meeting minutes in such a public forum without compromising confidential trial information. As a result, blog materials that focused on the existence and general activities were incorporated into the ACTIV-6 website, and the blog itself was ultimately discontinued. When using virtual asynchronous communication in the future, password-protected blogs or remote communication platforms such as Microsoft Teams or Slack as central repositories of SAC communications would ultimately be more viable, with a publicly available blog chronicling curated SAC activities attached.

Utilize existing engagement infrastructure

The members of our SAC – a diverse group of individuals from the North East, Midwest, and South – were referred to us through the PCORnet CRNs. Researchers seeking to set up an advisory committee are encouraged to “knock” on the PCORnet Front Door and submit a request indicating that they are interested in finding patient and/or physician partners for their study. Additionally, researchers might find similar help through their local CTSA or other engagement infrastructure available at their institutions.

Create and optimize communication structures early

The SAC operated virtually, with committee members dispersed across the country. In the case of ACTIV-6, SAC facilitation is centered in Pittsburgh, PA, and the core trial team is centered in Durham, NC. Trial startup additionally followed an accelerated timeline due to the emergent nature of the pandemic. In such a dispersed environment and amid the acute phase of a global pandemic, routine communication structures between the SAC and the core trial team were not determined at the outset. Successful communication procedures developed organically over time, including the presentations of information by SAC co-chairs Ms. Florence Thicklin and Dr. Matthew McCarthy at other ACTIV-6 committee meetings, and their reporting back to the SAC. It would be beneficial for similar trials in the future to intentionally outline communication structures between the SAC and the rest of the trial team, as well as the intentions behind these communication structures, before the beginning of the SAC’s work. In particular, routine times in executive committee meetings for reporting on SAC activities should be established, as should routine attendance of study PIs or their representatives at SAC meetings. Additionally, training in the rationale for and methods of stakeholder engagement should be offered to the entire study team. Communication channels should be used to highlight which stakeholder input has been adopted to change the overall study, and clearly explain why other input was not adopted.

Engage in research reciprocity with SAC members

Research reciprocity is the concept that relationships within research should be reciprocal – i.e., that research participants should be compensated in some way for their participation [Reference von Vacano, Stodulka, Dinkelaker and Thajib20–Reference Dostilio, Brackmann, Edwards, Harrison, Kliewer and Clayton22]. Although our SAC members are not participants in the trial itself, they are advisors and collaborators on the trial and should also experience reciprocity. SAC members were financially compensated for their time. Future trials and studies should endeavor, as ACTIV-6 did, to compensate SAC members at an hourly rate equivalent to any other specialist or contractor on the grant, or as close to it as possible, although the authors allow that this may not be attainable depending on the project’s funding source. There are, however, additional ways to compensate and show appreciation to stakeholders beyond monetary compensation. For example, ACTIV-6 incorporated research reciprocity into the structure of the SAC by ensuring that the rest of the trial team was responsive to SAC questions and interests. This process began with in-depth training in patient-focused research and stakeholder engagement, which incorporated and tailored training modules developed by PCORI to provide SAC members with better information about their roles in the trial. Then, on a routine basis, the broader study team ensured that the SAC’s scientific questions were answered. For example, representatives from the NIH/NCATS explained questions regarding decisions made by the trial, and physicians on the SAC explained why null results are still exciting and useful. Finally, we engaged experts in topics that SAC members indicated were of interest. For example, following considerable interest by the SAC in improving recruitment of participants from under-represented backgrounds into the trial, the ACTIV-6 team engaged the NIH-funded CEACR team to come speak to the SAC (and the study more broadly) regarding best practices for promoting inclusive participation in clinical research in minority and underserved communities. The NIH/NCATs and DCRI teams also discussed with the SAC the challenges and timelines associated with opening new recruitment sites – particularly sites that may not have established relationships with community organizations.

In addition to the dispersed nature of the trial, the trial itself examined medical treatment for an illness that, in the United States (and across the world) was interpreted differently by different segments of the population, often in politically polarized ways. It also examined a medication, ivermectin, which some physicians and political groups were championing as a “magic bullet” to end the pandemic. Indeed, one of the SAC members’ earliest insights regarded the fact that, when conducting an internet search for “ivermectin,” the first result was a warning from the FDA not to take ivermectin for COVID-19. SAC members argued most people might reasonably conduct such an internet search upon finding out which medication(s) they could be randomized to, and therefore, addressing the reasons this drug was considered safe to take in the context of a clinical trial was thus felt to be of considerable importance for participant trust. The group felt the trial was operating in scientifically important but politically charged waters, posing challenges to communication and acceptance. Because of desire on the part of SAC members to better understand scientific communication in polarized political environments, the SAC moderators identified and invited a Professor of Communications who delivered a talk to the SAC on how to communicate in such an environment. Future trials touching on politically polarizing topics might consider crafting a communication plan around these issues at the outset and training for the study team regarding communicating in polarized environments.

Evaluate your engagement

Throughout the trial, we informally sought feedback on the thoughts and feelings of SAC members through meetings and emails aimed at gathering their thoughts on the work as it unfolded. We are planning formal summative engagement activities for the close-out of the SAC at the end of 2023, including surveys and exit interviews assessing SAC and trial team experiences. Because of the rapid start of the trial, and the sense of urgency that trial team members felt at the time, we did not include any elements of formative evaluation. In retrospect, however, we feel that studies can benefit from evaluating their engagement in an ongoing manner throughout the trial and that these evaluation activities need not be burdensome if they consist of simple surveys asking about satisfaction with engagement activities, with more extensive follow-up for surveys indicating any dissatisfaction.

Plan for on-the-ground engagement efforts

Throughout our work, we found that a centralized, remote SAC does not replace the need for local engagement work, particularly when it comes to engagement efforts that center on the recruitment of participants. For example, when the SAC suggested additional focus on diversity in recruitment, the work to actually increase diversity in recruitment on the ground had to come through community partnerships and relationships with individual sites. There is a continued need to support and expect local study teams to perform robust community engagement work on behalf of the centralized study team.