Adoption barriers and facilitators

RE-AIM defines adoption as “the proportion and representativeness of settings…that adopt a given policy or program” [Reference Glasgow, Harden and Gaglio29,Reference Glasgow, Vogt and Boles46]. In this formative pre-implementation D&I study, “adoption” emerged as a key TeamMAPPS implementation outcome. We focused our analysis of adoption on how participants framed likely barriers and facilitators at their institutions along with general anticipated adoption issues.

Within “innovation,” participants saw great benefit to adopting TeamMAPPS and described strategies to enable adoption that they planned to use. This enthusiasm reflected the characteristics of participants as individuals involved in the development, design, and implementation planning for TeamMAPPS. Participants consistently expressed that there is not a similar available intervention for science teams and spoke to the gap that TeamMAPPS can fill, emphasizing its relative advantage.

Among the most critical CFIR domains under adoption were “implementation process” and “inner setting.” Within “implementation process,” the adaptability of TeamMAPPS was seen as a major facilitator. The ability of TeamMAPPS to be delivered as a whole-team intervention, to be taken independently by individual scientists, or as part of a course was seen as a benefit. The adaptability of TeamMAPPS also appeared to influence planning processes for TeamMAPPS, particularly regarding institutional contingencies that would structure implementation. For example, CTSAs issue pilot awards, and some participants mentioned plans to incorporate TeamMAPPS into award requirements. Others described plans to incorporate TeamMAPPS into courses for trainees, as part of IPE, or as part of Team Science trainings and offerings. Participants also discussed barriers and facilitators related to assessing context and assessing needs of universities and CTSAs operating within the constraints and expectations of the health sciences, NIH, NCATS, and translational science. Several also spoke about the process of individuals reflecting on and evaluating their experiences implementing or receiving TeamMAPPS. While TeamMAPPS was perceived to be evidence-based and rooted in SciTS principles, participants noted that it would ideally be supported by additional evidence of effectiveness as implementation proceeded, through overall and site-specific evaluation.

Under the “inner setting” CFIR domain, participants discussed the importance of work infrastructure and culture at implementing institutions, particularly regarding systems of support for Team Science trainings. This was connected to the culture of institutions regarding Team Science. Some participants noted relatively limited support for trainings, while others described how their institutions valued and invested in Team Science. This fed into discussions about the relational connections that TeamMAPPS implementers could leverage at their institutions to disseminate it. TeamMAPPS’s flexibility was perceived as contributing to its learner-centeredness and deliverer-centeredness.

The second most discussed set of CFIR domains affecting adoption were “individuals” and “innovation.” Within “individuals,” the roles of implementation facilitators, leads, and team members, as well as opinion leaders and high-level organizational leaders were all framed as critical to adoption. An implementer’s ability to access and learn from the experts who developed TeamMAPPS was seen as crucial to facilitating widespread and effective adoption. However, the success of TeamMAPPS was seen to also rely on support from local leadership. Further, the flexibility of TeamMAPPS delivery was seen as a benefit to implementers and innovation recipients. Options discussed included using TeamMAPPS with students in classes, in trainee mentorship plans, with individual scientists, or with whole teams. Regarding “innovation,” TeamMAPPS was seen as being evidence-based. Implementers held the creators (source) of TeamMAPPS in high regard and believed that TeamMAPPS was highly adaptable, owing to the design of the modules and implementation support materials. These factors were seen supporting the adoption of TeamMAPPS. Adaptability also contributed to the relative advantage of TeamMAPPS compared to other available trainings.

Participants also spoke about factors that may influence TeamMAPPS adoption related to “outer setting.” That TeamMAPPS can be used to fulfill requirements for IPE and trainee activities led participants to frame TeamMAPPS as responsive to external pressures put upon implementing organizations by funders such as NIH, accreditors, and similar entities. Support from external entities, such as NIH, that finance Team Science and thereby foster local attitudes and conditions were also seen as critical to facilitating adoption.

Reach barriers and facilitators

Reach was identified as a key implementation outcome. Reach refers to the number, proportion, and representativeness of potential end-users and their reasoning for/against implementation [Reference Glasgow and Estabrooks40,Reference Glasgow, Vogt and Boles46]. Generally, interview participants focused on the kind of end-users to target and how to provide compelling rationales to encourage participation. Much discussion centered around trade-offs of focusing on specific categories of learners and how to best balance benefits and costs. The hierarchy of academic research was discussed across nearly all domains and constructs. Relative positions of power were reported to influence whether an end-user may be interested in or capable of participating in the training and integrating Team Science principles into their work. The most emphasized CFIR domains related to reach were “individuals” and “inner setting.”

When discussing “individuals,” participants focused on the importance of mid-level leaders and innovation recipients. Mid-level leaders were felt to be critically important to maximize reach, due to their influence – particularly principal investigators, because they are often members of multiple teams and can influence junior researchers. Participants weighed benefits and costs of targeting specific individuals or groups for implementation. For example, it was felt that it might be beneficial to implement TeamMAPPS with entire teams, but drawbacks included lack of time and potential difficulties in a group activity with a dysfunctional team. The benefits and drawbacks of targeting junior rather than senior researchers were also considered. On one hand, junior investigators may have more time and may be more open to changing their teamwork style. On the other, senior investigators have more influence, but also more constraints on their time and potentially more solidified teamwork styles.

The most discussed constructs related to “inner setting” were compatibility and available resources. Compatibility was explored in terms of logistics related to participation. Like discussions of adoption, time was considered a major factor influencing reach. Investigators who need or want to participate may not be able to because of competing demands. However, participants identified many potential inner setting resources that might facilitate reach. Existing educational programs, pilot funding, and research training programs were considered promising vehicles.

“Implementation process” and “innovation” were also discussed. For implementation process, tailoring strategies, adapting, and engaging constructs often co-occurred. Tailoring mode of delivery was suggested as method of making TeamMAPPS more appealing. Tailoring by problems that a particular team is addressing, team maturity (i.e., newly formed vs. longstanding), stage of the research pathway (e.g., bench vs. animal vs. human trials), and individuals’ roles in the team were suggested as potentially helpful. Various methods of adapting delivery were also suggested to increase reach, such as offering hybrid delivery, delivering specific modules, or using learning contracts. Within the “innovation” domain, the evidence base of TeamMAPPS was a major perceived benefit to facilitate reach.

“Outer setting” was also discussed, though less than other domains. Financing along with partnerships and connections were two domains of note. The CTSA network was often discussed as an important part of the implementation context. Specifically, CTSAs were thought to facilitate Team Science training through incentivizing trainees to take them.

Effectiveness barriers and facilitators

Effectiveness emerged as a key RE-AIM implementation outcome. RE-AIM defines effectiveness as “the impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes” [Reference Glasgow, Harden and Gaglio29]. Because TeamMAPPS was not yet being implemented when we conducted our interviews, discussions about effectiveness were speculative and did not refer to observed effectiveness; rather, they spoke about the potential for effectiveness. Therefore, we focus on how participants defined effectiveness contextually and how it should be measured. Effectiveness was believed to be dependent on factors related to implementation fidelity – the quality of TeamMAPPS delivery – and the success of implementation efforts regarding adoption by institutions and reach to individual learners.

Many participants noted the importance of understanding the unique learning styles and career needs of individuals and the quality of the training experience. Ideally, a well-designed intervention that has evidentiary rigor and can be tailored to specific team needs could be quite effective for enculturating a collaborative group into a high functioning research team. While participants highlighted the effectiveness and utility of TeamMAPPS in these regards, they also discussed the need for quality assurance analyses to assess the fidelity of training and “transfer of training” across different intervention recipients. This reflected participants’ general desire for consistent training and application of Team Science principles. One interviewee articulated it as a “hybrid-model” whereby core elements would be standardized, while salient, team-specific guidance could also be incorporated to maximize effectiveness. Participants also suggested that there be mechanisms and metrics to demonstrate effectiveness, ranging from traditional measures of output (i.e., publications) to pre-/post-assessments of Team Science knowledge, skills, competencies provided by the program. Some of these, such as pre-/post- tests, are built into the TeamMAPPS modules. Others, such as measuring long-term outcomes, could become goals of this D&I study or local evaluations.

The main CFIR domains that emerged for the effectiveness outcome included “implementation process” and “innovation.” Regarding implementation process, participants noted their beliefs that the effectiveness of TeamMAPPS will hinge on careful reflection regarding innovation recipients and innovation deliverers. Specific concerns included pre-/post- assessments, assessing current team environment, making material meaningful and relevant, having authentic trainers, and measuring outcomes. This aligns with other domains including assessing context and tailoring strategies to optimize impact and ensure implementation fidelity. Regarding “innovation,” one of the most notable constructs was the evidence base used to develop TeamMAPPS, which helped establish its credibility. The intervention’s effectiveness was framed as likely being positively impacted by the innovation source, or the fact that Team Science leaders created the trainings using a robust evidence base.

Secondary CFIR domains that participants discussed in relation to effectiveness included “inner setting,” “individuals,” and “outer setting.” Within inner setting, the domains relational connections, culture, and recipient centeredness were prominent. Using TeamMAPPS to foster relational connections conducive to authentic interactions, understanding other team members’ roles, and reciprocity were deemed important implementing TeamMAPPS with fidelity to maximize effectiveness. Cultural contexts such as clarifying norms and creating aligned guiding principles focused on matters related to interpersonal relationships and navigating different team cultures. A focus on training recipients can foster the aforementioned issues to create a personalized experience that meets individual and team needs. In discussions related to “individuals,” innovation recipients and innovation deliverers were the two primary constructs. Participants often discussed these constructs concordantly, regarding the ability of TeamMAPPS to facilitate effective knowledge transfer, change team norms, meet user needs, and incentivize participation. Regarding “outer setting,” the primary domains that emerged were partnerships and connections coupled with local attitudes. Participants noted that leveraging CTSA hubs and networks and the support of high-level leaders may increase participation, buy-in, and overall implementation effectiveness. Supportive local context emerged as critically important.

Potential TeamMAPPS implementation strategies

This section describes recommended implementation strategies. These were created based on findings from our interviews, recommendations from the TeamMAPPS leadership team, and findings from the CFIR + ERIC Matching Tool based on key CFIR barriers presented in Table 4 above. We exclude strategies related to “effectiveness,” as they would have been too speculative during pre-implementation. Table 6 shows potential implementation strategies, organized by the “reach” and “adoption” RE-AIM outcomes.

Table 6.

Potential implementation strategies to enhance the adoption and reach of TeamMAPPS

CFIR: Consolidated Framework for Implementation Research; CTSA: Clinical and Translational Science Award; ERIC: Expert Recommendations for Implementing Change; TeamMAPPS: Team Methods to Advance Processes and Performance in Science; TeamSTEPPS: Team Strategies and Tools to Enhance Performance and Patient Safety.

* The language of “patients” included in the CFIR + ERIC Matching Tool outputs reflects the tool’s original development for clinical interventions.

The first column displays implementation strategies recommended by the TeamMAPPS leadership team, of which this D&I study team is part. TeamMAPPS was designed with implementation strategies in mind, and there is an existing set of materials developed based on expert opinion available upon request that will be regularly updated. The second column shows recommendations based on participants’ interviews and the analysis above. The third column shows recommendations from the CFIR + ERIC Matching Tool, most of which map onto similar strategies in the first two columns. Notably, the CFIR + ERIC Matching Tool recommended many strategies that involve providing incentive structures for scientists to go through TeamMAPPS and shorting up local support. The creation of local assessment packages and tailored implementation strategies are also key recommendations.

These results provide diverse strategies to implement TeamMAPPS that could enhance reach and adoption. They were developed based on the triangulation of multiple different perspectives. CTSAs and other institutions seeking to implement TeamMAPPS should consider what strategies would work best for them based upon available resources and local needs, using our recommendations as guidelines.