Introduction
The University of Wisconsin-Madison (UW) has been recognized for its long-standing expertise in developing research mentor and mentee training programs for diverse scholars in science, technology, engineering, mathematics, and medicine [Reference Handelsman1–Reference Sorkness4]. The initial Entering Mentoring manual [Reference Handelsman1] and 9 subsequent curricular adaptations [Reference Handelsman1, Reference Pfund3, Reference Pfund, Branchaw and Handelsman5–Reference House9] were created to guide training and help mentors to provide more effective, strong mentorship, which has been subsequently linked to enhance mentee productivity, self-efficacy, career satisfaction, and prediction of academic success of scientists in training [Reference Pfund10]. Published evaluations of the Entering Mentoring workshops indicated that mentors who participated in the training gained important mentorship skills [Reference Pfund2]. In 2010, the UW Institute for Clinical and Translational Research (ICTR), the academic home of its Clinical and Translational Science Award (CTSA), partnered with investigators to adapt, expand, and implement existing research mentor training (RMT) programs to address the needs of clinical and translational scientists. The ICTR translational workforce development team subsequently built on this foundation and framework for its own biomedical research workforce development program [Reference Sorkness4], with a commitment to serve as a training hub for other CTSA hubs. This manuscript describes the dissemination efforts of ICTR and the impact on CTSA hubs nationwide.
Background
A key function of CTSA programs is to provide research education, training, and career development for the next generation of clinical and translational scientists, including the KL2 scholar program. Importantly, the initial requests for application for the CTSA award issued by the National Center for Research Resources explicitly stated that programs needed to demonstrate how mentors of scholars and trainees would be trained and evaluated. The National Center for Research Resources leadership recognized the importance of mentoring for its scholars and mentees, by convening a CTSA Education Key Function Committee “mentor working group” to identify the types of initiatives developed by CTSA hubs to support KL2 scholars. The CTSA Key Function Committee mentor working group included the Research Education and Career Development Directors from 18 CTSA hubs. This working group published a series of “white papers,” focused on the various elements of mentoring [Reference Silet, Asquith and Fleming11–Reference Meyers17].
RMT Curricula for Clinical and Translational Science Mentors
In 2010, UW received a CTSA administrative supplement to lead a 5-institution multidisciplinary team to adapt Entering Mentoring [Reference Handelsman1] for mentors of clinical and translational researchers [Reference Pfund18]. After beta-testing and final revisions, the curriculum was published as Mentor Training for Clinical and Translational Researchers [Reference Pfund3]. The curriculum was designed to engage small groups of mentors in discussions of case studies and activities to aid them in meeting specific learning objectives, established for each of the targeted competencies (maintaining effective communication; aligning expectations; assessing understanding; addressing diversity; fostering independence; and promoting professional development). The training is most often led by trained facilitators, commonly implemented in four 2-h sessions.
In the spirit of “evidence-based practice,” the Mentor Training for Clinical and Translational Researchers curriculum was evaluated in 2010–2011 using a randomized controlled trial design at 16 US academic institutions (15 CTSA hubs, 1 non-CTSA site). Mentor-mentee pairs (n=283) were recruited, with a 98% trial retention rate. Mentors were allocated to the 8-h training group (n=144) or to the control group (n=139); baseline and 6-month postrandomization interviews were conducted using the validated mentoring competency assessment [Reference Fleming19] as the primary outcome. Mentors who received training documented statistically significant self-reported skill gains and self-reported changes in mentoring behaviors [Reference Pfund20]. In addition, their mentees corroborated these positive changes in interviews.
Resources for Research Mentors and Training Efforts
With this evidence and additional support from National Institutes of Health (NIH), the UW team created a web-based legacy resource to: (1) share information on best mentoring practices and (2) develop, house, and disseminate RMT curricula. This legacy resource (https://mentoringresources.ictr.wisc.edu/) provides resources for both mentors and mentees across 3 stages of mentoring relationships. Example resources include mentoring contracts, individual development plan templates, and readings. The legacy resource also provided easy access to the Mentor Training for Clinical and Translational Researchers curriculum [Reference Pfund3], as well as 3 adapted versions for mentors in biomedical research [Reference Pfund8], clinical and behavioral research [Reference Asquith7], and community engagement research [Reference House9]. These materials are now available on the Web site for the UW Center for the Improvement of Mentored Experiences in Research (www.cimerproject.org). The Center for the Improvement of Mentored Experiences in Research site also provides facilitator training (FT) and assessment instruments.
RMT: The FT Initiative
The published trial evidence and accessible RMT curricula created a growing demand for mentor training. With support from an NIGMS R13 conference grant, the UW team promoted the dissemination of the evidence-based training using a train-the-trainer approach to prepare facilitators to implement RMT locally. Train-the-Trainer refers to a program or a course where individuals in a specific field receive training in a given subject and instruction on how to train, monitor, and facilitate other individuals in the approach [Reference Pearce21]. The goals of these FT were to: (1) increase facilitator knowledge of an RMT curriculum; (2) increase facilitator confidence in implementing the curriculum; (3) provide a safe environment to practice facilitation of curricular activities; and (4) review implementation strategies and evaluation tools [Reference Pfund22]. These events were strategically organized as intensive FT workshops at preconference sessions linked to events drawing diverse scholars including those underrepresented in the biomedical research workforce. Through these workshops, 108 facilitators were trained.
In 2014, UW investigators joined the NIH Diversity Program Consortium [23], with leadership roles in the National Research Mentoring Network (NRMN) [24], a national network helping to diversify the biomedical workforce through professional development and mentoring [Reference Sorkness25, Reference Guerrero26]. UW is home to the NRMN Mentor Training Core. The UW ICTR and NRMN Mentor Training Core teams have collaborated with CTSA hubs including those in the Big Ten Academic Alliance, Albert Einstein University, University of California at San Francisco, and others to deliver FT to enhance RMT capacity. This manuscript describes the outcomes of the FT initiative across the CTSA hubs.
Methods
FT
All data used to evaluate the FT events were collected through the UW ICTR/NRMN centralized evaluation system on both training outcomes and the facilitator’s implementation practices after initial training. All data collection was reviewed and approved by the UW-Institutional Review Board (protocols no. 2015-1330, 2016-0458, and 2017-0026).
For each FT event there was a roster of the participants that includes their names and institutions. These rosters allowed us to count the total number of FT participants (n=500), as well as to determine how many were affiliated with a CTSA hub (n=290). In addition to collecting the rosters, to ensure program quality and measure the impact of FT sessions, 2 different surveys were administered to the participants of FT events. The first survey (Facilitator Training Evaluation Survey) was administered using Qualtrics Survey Software immediately following the FT event. The purpose of this survey was to assess the quality and effectiveness of the FTs. The second survey (Implementation Survey) was emailed via an electronic survey link once a year in the fall to all individuals who had completed a FT session. This survey was also administered using Qualtrics Survey Software. The purpose of this survey was to assess the impact of FTs by querying the number and type of mentee and mentor trainings implemented by those individuals who completed a FT session.
Facilitator Training Evaluation Survey
To assess the quality and effectiveness of the FTs, the Facilitator Training Evaluation Survey measured satisfaction with the training and participant’s self-reported skill and confidence gains. To measure satisfaction, participants were asked to rate the overall workshop as well as each component using a 5-point scale ranging from poor (1) to excellent (5). In addition to facilitator demographics and satisfaction, participants were asked to rank self-reported gains in confidence for the learning objectives of the workshop, including confidence in the ability to implement training before and after the workshop. Participants retrospectively ranked confidence in their training ability preworkshop and postworkshop.Footnote 1 Confidence gains were measured on a 7-point Likert scale ranging from not at all confident (1) to extremely confident (7). These scales remained consistent across all years of the evaluation surveys. Participants were also asked if they planned to implement a training after the workshop. For this analysis we used evaluation data from 11 of the 21 FT events.Footnote 2 From these 11 FT events, 101 individuals were identified as being associated with a CTSA institution. Thus, we were able to present the evaluation results for 101 trained facilitators from CTSA hubs (35% of all trained facilitators from CTSA hubs), from across 20 different CTSA hubs. Fig. 1 illustrates the different data collected and the associated sample sizes.
Types of data collected and associated sample sizes. Evaluation surveys were used to collect participant data immediately following each facilitator training workshop. Implementation surveys were used to collect longitudinal data from participants annually. CTSA, Clinical and Translational Science Award.
Implementation Survey
To assess impact of the FTs, the implementation survey queried the number and length of trainings implemented as well as the curriculum used. This analysis used data from the fall 2016 implementation survey. Of the 290 trained facilitators that the survey was sent to 73 individuals responded from 31 different CTSA hubs,Footnote 3 allowing us to present the implementation results for 25% of the total 290 trained facilitators from CTSA hubs.
To further assess the impact of the FTs, along with the annual Implementation Survey we examined all tracked queries to the ICTR/NRMN centralized evaluation system (all events which used our system to evaluate their training), to best count the number of implementations by trained facilitators. This information allowed for the discovery of an additional 6 CTSA hubs which had a facilitator who implemented a RMT. Although this number is added to the total number of implementations, there is no other information about these implementations or the implementers, and thus they are only included in the total count.
Results
Through the growth of the partnerships described earlier, the facilitator initiative has resulted in the implementation of 21 different FT events (see Table 1). This “train-the-trainer” model created over 500 facilitators who are able to implement RMT, of whom 290 are located at CTSA hubs. Table 1 illustrates the growth of the FT initiative over 2010–2016, as well as the number of individuals from CTSA hubs trained at each workshop.
Dissemination of the facilitator training initiative
UW, University of Wisconsin; CTSA, Clinical and Translational Science Award.
Location and date of the facilitator training workshops as well as the number of participants from CTSA hubs and the total number of participants.
Quality and Effectiveness of FTs
For CTSA participants of the FTs, the average rating for satisfaction with the overall workshop was 3.28 (scale from 1 to 5) with 65% of the participants giving a rating of good or higher. Of the 78 participants from CTSA hubs who were asked if they intended to implement a training after the workshop,Footnote 4 66 of the 78 (85%) responded in the affirmative. Thus, the majority of CTSA participants of the FTs reported being satisfied with the trainings and planning to implement after the training.
Participants of the FTs from CTSA hubs reported significant confidence gains in their ability to implement trainings. Using a Wilcoxon signed-rank test, confidence before the training (retrospective) was compared with after. Table 2 shows that there were significant confidence gains for each of the 7 learning objectives of the FT. Of particular note, there are the significant gains in the participant’s confidence to implement RMT at their home institution.
Self-reported gains in confidence before and after facilitator training (FT)
Reports participant confidence gains in each of the training areas.
* Significant statistical difference in ratings before and after FT was determined by Wilcoxon signed-rank test. Confidence was rated on a 7-point Likert-scale with 1—not at all confident, 4—moderately confident, 7—extremely confident.
† This learning objective was added to the survey beginning in 2016.
We had career stage and demographic information for 49 of the 101 individuals from CTSA hubs from which we have FT evaluation data.Footnote 5 These participants represented a range of career stages including scientists/researchers (10%), training program directors (16%), deans (6%), and professors (67%).Footnote 6 Demographic information shows that 46% self-reported as female versus 25% as male.Footnote 7 The majority of this group was White (52%), a small percentage Black/African American (6%) and Asian (4%), with 2 participant’s self-reporting as Hispanic/Latino.Footnote 8
Impact of the FT Initiative
The growth of the FT initiative contributed to an increase in the number of trained facilitators across CTSA hubs and the growth of RMT. As of January 2017, there were over 290 trained facilitators spanning 48 (75%) of the 64 CTSA hubs. In addition to trained CTSA facilitators, there were 12 requests from CTSAs to launch local training efforts and 6 follow-up consultation requests. In addition, 32 of the CTSA hubs downloaded curriculum from the UW Web site. At least 15 faculty and staff members from CTSA hubs co-authored mentoring publications [Reference Asquith7, Reference Pfund10, Reference Burnham and Fleming12–Reference Huskins14, Reference Abedin16–Reference Pfund20, Reference Sorkness25] with UW team members, demonstrating ongoing collaborations to improve mentoring.
A key component to the dissemination of the FT initiative was the actual implementation of mentor training by the trained facilitators. We determined that of the 64 CTSA hubs, 37 (58%) had trained individuals who have subsequently implemented mentor training. Seventy-five percent of those who responded to the implementation survey (55/73) had implemented mentor training and 17% had not yet implemented but had future plans (12/73); only 8% reported no current plans to implement (6/73). Implementers represented a range of career stages including postdoctoral trainees (1.7%), scientists/researchers (7%), training program directors (23%), and deans (21%). The majority were professors (61%), who also self-reported another academic title. More implementers self-reported as females (54%) versus males (18%).Footnote 9 The majority of this group was White (65%), a small number Black/African American (7%), Asian (4%), and other (4%),Footnote 10 with one individual who self-reported as Hispanic/Latino.
Of the 55 individuals who implemented mentor training, the Mentor Training for Clinical and Translational Researchers (for mentors of junior faculty, postdoctoral trainees, and graduate students) curriculum was most commonly used. The Mentor Training for Biomedical Researchers curriculum (for mentors of postdoctoral trainees and graduate students) was the next most common in implementations, followed closely by Mentor Training for Clinical and Behavioral Researchers curriculum. The least common was the Entering Mentoring curriculum (for mentors of undergraduates) and Mentor Training for Community-Engaged Researchers curriculum (for mentors of junior faculty, postdoctoral trainees, and graduate students). Table 3 outlines the extent of mentor training reported by the 55 implementers. Table 3 includes both high-dose mentor trainings (those 4 h or more) and low-dose trainings (those <4 h) to provide a complete picture of implementation.
Number of research mentor trainings implemented by trained facilitators at Clinical and Translational Science Awards (CTSAs)
The number of trainings implemented from 2010 to 2016 and those which were planned for 2017.
* High-dose trainings are 4 h or more, low-dose trainings are <4 h.
Multiple mentor trainings have been implemented between 2010 and 2016, at both the low-dosage and high-dosage levels; additional trainings were planned for 2017. The extent of trainings parallels the growth of the facilitator initiative. From this sample of trained facilitators, there have been over 238 implementations of mentor training at CTSA hubs with another 70 planned for 2017. Using our centralized data collection system as a reference, we can approximate that the average mentor training session has 10 participants. Extrapolating this to our sample, there have been 2380 mentors trained by individuals at CTSA hubs, with another 700 potential participants during 2017.
Discussion
As part of their mission, CTSA hubs are charged with developing, testing, and disseminating evidence-based practices across award sites nationwide. CTSA hubs are encouraged to collaborate on educational and training initiatives, share methods and online resources, and disseminate tools and approaches. Over the past 7 years, UW has met this challenge by serving as a CTSA hub for RMT. Building upon this foundation, UW team members led the development and testing of 3 specialized RMT curricula in collaboration with other CTSA program leaders [Reference Asquith7–Reference House9].
Three elements to advance RMT have been critical: (1) using an FT model to empower others to build local capacity for mentor training; (2) tracking implementation of mentor training events across the CTSAs over time; and (3) partnering with implementation sites to evaluate the training offered at their institution. Here we report that facilitators have been trained at 75% of the CTSA hubs. These facilitators reported high satisfaction with the training, increased confidence in their ability to implement mentor training, and plans to implement mentor training at their home institutions. These findings are consistent with previous data on the national impact of the first FTs from 2010 to 2013, which stated that facilitators from both CTSA and non-CTSA hubs had satisfaction and statistically significant gains in their confidence to implement [Reference Pfund22]. With the added FTs from 2014 to 2016 and the CTSA-only cut of the data, we still see similar gains as those found earlier on. In addition, 85% of these facilitators reported plans to implement mentor training after attending the FT. These data further support that the train-the-trainer model empowers facilitators to implement mentor training at their local institutions. We acknowledge that the response rate on these surveys is low and these results should be interpreted with caution. However, we feel that these preliminary data provide a good picture of the impact of the FT initiative, even if this impact might be underestimated. Future analyses should be pursued.
To track actual implementation of mentor training, participants are surveyed annually following completion of FT. To date, participants have reported implementing over 200 training events at CTSA hubs. Due to lower than optimal survey response rate, this is likely an underestimate. These data indicate that there is an interest in and need for RMT across the CTSAs and that our FT approach aimed at building local capacity across the CTSA has had tremendous impact. Moreover, investing in regular, longitudinal tracking of capacity-building efforts is critical in assessing impact of FT approaches. High-quality training implemented by individuals engaged through FT events is essential, but requires resources and ongoing commitment. By offering customizable evaluation services, we have been able to collect data across implementations using common assessment metrics. Evaluation reports (and raw data with proper IRB approval) are returned to the host site for their own use, but their data contribute to a larger data set. This data set allows us to track quality of the trainings implemented as well as build a national data set that can be used to both iteratively improve FT events and study mentor trainings across the nation.
The success of the FT efforts coordinated by UW ICTR team members has served as a model for similar efforts now being offered through the NRMN [24]. The synergy between these 2 groups has led to 105% growth in the number of trained facilitators from CTSA hubs since the conception of NRMN. Along with providing centralized curricula and resources [27] and centralized evaluation resources [24, 27] to the trained facilitators, efforts are also underway to further support trained facilitators in their implementation efforts through programs such as the NRMN Certified Facilitator program. This program will serve as a means to recognize effective facilitation, promote dissemination of mentor and mentee training, assist with communication between facilitators, and promote information sharing and evaluation. While the hope is that trained facilitators at each CTSA will teach others to facilitate training thereby growing and sustaining their local efforts, we expect an ongoing need to continue offering FT to expand capacity at individual CTSA hubs as well as across the nation at a wide variety of institutions.
Future Directions
An important component of the dissemination of our FT initiative is the robust, ongoing collection of data. The centralized data collection allows for the quality and effectiveness of the FTs to be examined; ensuring that as the initiative grows, it maintains its integrity and furthers RMT among biomedical researchers. As more data are collected in the future, analyses can examine the trajectories, strategies, and barrier-reductions of those who attended an FT and then successfully implemented local site training. Comparisons between different sites can identify key components of success and recommendations for best practices.
A second important component of the FT initiative is to provide trained facilitators with new adaptations and enhancements to the existing curricula in response to expanded research mentorship needs. As new curricular modules are developed and tested by ICTR and NRMN team members in partnership with others around the country, they will be made available to facilitators through Web sites such as www.cimerproject.org. New modules include those focused on Research Self-Efficacy, Work-Life Integration and Motivation, Social Sciences, and Clinician Educators. In addition, leaders at UW in partnership with others have developed a parallel mentee training curriculum for career development of KL2 scholars and others. This 8-hour training has been beta-tested at 2 CTSA programs with promising results. Plans are in place for a FT workshop to help others implement this process-based curriculum.
The work at the UW CTSA in partnerships with other colleagues, including the NRMN, has established an evidence-based approach to capacity building for mentor training. This work can serve as a model for dissemination and implementation of educational interventions across the CTSA and other consortia.
Acknowledgments
This project was sponsored by the CTSA program through the NIH’s National Center for Advancing Translational Sciences, grant no. UL1TR000427 to UW ICTR and the NRMN through the National Institute of General Medical Sciences award no. U54GM119023 from the National Institutes of Health Common Fund and Office of Scientific Workforce Diversity. The authors thank CTSA partners from across the country, the NRMN Master Facilitators who have implemented many train-the-trainer events, and UW ICTR/NRMN team members. The authors also especially thank Stephanie House and Emily Utzerath for managing all of the FT events as well as Marc Drezner, Elizabeth Burnside, and Laura Hogan for reviewing this manuscript and offering helpful comments.
Disclosures
Some of the curricula noted in this manuscript has been published with MacMillian Publishers; however, these curricula are also freely available for download at www.cimerproject.org.
Open access
