Introduction
Conducting translational research requires identifying and training high-impact interdisciplinary, translational research teams [Reference Ameredes1]. Barriers to assembling such teams including attitudes, communication challenges, turf issues, and lack of leadership [Reference Pellmar and Eisenberg2] may result in overlooking key assets, misaligned resources, underutilizing talent, and underperformance of employees [Reference Cooke and Hilton3].
As in any area of work-based practice, competence in research roles is usually developed through a combination of formal training, mentorship, supervision, and experience [Reference Pellmar and Eisenberg2]. For teams to function effectively, they must also work to build trust and respect among team members, develop a shared language and vision, address power and hierarchy differences, and build a proactive approach to problem-solving and team engagement [Reference Ameredes1, Reference Bennett and Gadlin4, Reference McEwan5]. In line with the goals of the Clinical and Translational Science Awards program launched in 2006 by the National Institutes of Health’s National Center for Research Resources, research teams should be built using best practices and effective policies and procedures to advance and support clinical and translational research [Reference Weber-Main6, Reference Speicher7].
Principal investigators (PIs) have benefitted from team science skills training, which has been effective in helping them form and maintain effective collaborative, interdisciplinary, cross-translational teams, but there has been little training for other members of the translational research workforce. Research staff are typically only trained in specific research skills with minimal preparation for the process of working on a team including the interpersonal communication skills and project management skills critical to high functioning teams. In reality, research personnel mostly learn “on the job” through trial and error.
To address this lack of training opportunity, we developed the Strategic Teamwork for Effective Practice Mentor Development Program (STEP-MDP) to prepare research staff to become research team leaders (RTLs) who promote optimal team functioning. In addition, we hoped to create a community of practice (CoP) among STEP-MDP participants that would provide ongoing support after the completion of the program.
Our development objectives were to (1) establish the STEP-MDP; develop a series of experiential workshops and materials; to train RTLs in effective communication, team building, leadership skills, and coaching; (2) train a group of RTLs to become mentors within their research teams; and (3) launch a STEP CoP for continued networking, learning and mentorship among research teams within the institution.
Methods
Subjects
We recruited participants by advertising within the NYU Langone Health research network. Potential candidates self-identified or were referred by supervisors, either to remediate weaknesses or to advance skills development. They submitted resumes, information about their team role, a personal statement detailing how they planned to utilize learned skills and a letter of support from their supervisor/PI to garner support for integration of new skills into the work setting. We accepted 32 of 39 applicants for our first two training cohorts, 2 were not accepted because they were leaving the institution and the remaining were put on a waiting list. The 32 RTLs who participated in STEP-MDP held a range of different positions within research teams across the institution, bringing different perspectives and roles (see Table 1). In total, 10 participants (31%) worked at the institution 5 or more years, 7 (22%) were in their 1st year, 16% were male.
Table 1 Participant positions and engagement in leadership
STEP-MDP, strategic teamwork for effective practice mentor development program.
Intervention
Fig. 1 summarizes learning objectives and teaching strategies employed in the 3, 2-h workshops. The methods deployed in STEP-MDP were based on activities from the NYU/HHC Clinical and Translational Science Institute (CTSI) Mentor Development Program for Faculty [8] and the Mayo Clinic Wellness Coach Training Program [Reference Clark9]. Participants sharing similar roles were grouped into peer-coaching teams (PCTs) of 4 to work together during the workshops and meet between them to practice coaching skills, and troubleshoot problems. Participants were reminded by email to interact in the STEP-MDP web-based forum.
Fig. 1 Strategic teamwork for effective practice mentor development program workshop objectives and methods.
Program Evaluation
Learners were assessed both immediately after STEP-MDP and 3 months later and their supervisors’ views were elicited at 3 months.
Measures and Data Analysis
Immediate Participant Self-Assessment
We assessed participants’ perceptions and self-reported skills using a retrospective pre-post design rather than a standard pre-assessment and post-assessment because we were concerned that our participants might not be familiar enough with the content to accurately self-assess before exposure to STEP-MDP [Reference Bhanji10, Reference Howard11]. Thus, they were asked to rate their skills at the end of training and then to reflect back to before the training began and rate their skills at that time using the same survey items (retrospective pre).
Based on the STEP-MDP goals and a focused literature review we designed a 37-item survey to assess specific skills (e.g., “I can lead productive meetings”) and attitudes (e.g., “I feel empowered to help others”) in 5 domains, including Communication (4 items), Leadership (8 items), Empowerment/Motivation (12 items), Coaching (6 items), and Community (3 items). First, participants rated each item on a 4-point Likert scale, from strongly disagree to strongly agree, then they reflected back to before STEP-MDP and retrospectively rated themselves on the same items. Cronbach’s α for the survey was 0.88.
3-Month Follow-Up
Three months post workshop (1) participants completed a 7-item survey assessing use of specific leadership behaviors and the impact of this training on the participants’ professional growth and development and (2) supervisors, usually the one who wrote the letter of support, answered 8 open-ended questions exploring the impact of the training on their supervisee, specifically eliciting demonstration of coaching and leadership skills.
Mean scores were computed from the retrospective-pre post survey for each of the 5 domains for the pre- and post-time periods. These scores were compared using paired t-tests. Descriptive statistics are reported for the 3-month follow-up learner survey. Text-based responses to open-ended questions for both the learner and supervisor were content analyzed using an inductive iterative approach.
Results
Immediate Evaluation
In total, 29 participants (90%) completed the evaluation immediately after the program. Compared with retrospective pre-assessments, participants reported significant improvements in each of 5 self-assessment domains (see Fig. 2). Participants’ written comments strongly endorsed the value of the workshops for their work, emphasizing the value of the coaching skills session and the PCTs. One participant particularly valued the PCT because he had no peers in his immediate work environment. Although the only negative comments were about the barriers to using the intranet to communicate with the CoP, all but one participant reported planning to continue meeting with their PCT.
Fig. 2 Change in self-reported team leadership skills, self-efficacy, and attitudes after strategic teamwork for effective practice mentor development program.
3-Month Follow-Up
In all, 19 participants (60%) completed the online 3-month follow-up survey, 3 had left the institution and 7 did not respond. Because surveys were anonymous we could not identify data about non-responders. Most respondents reported that they used the skills taught in STEP-MDP (see Table 1). The PCTs were not strongly maintained, with barriers such as time and distance limitations and discomfort with the on-line CoP forum noted. Participants identified practicing communication skills as the most valuable aspect of the program (mentioned by 14 respondents). Exemplar responses include the following: “as someone who avoids conflict, it’s provided an easier way to ease into discussions that may include more critical feedback. Where I might previously have been nervous about the response to me saying something negative, now I’m not as reluctant to engage” and “I was able to effectively communicate my issues with a coworker and have us move past our issues.” Other prominent themes included increasing self-confidence and self-awareness, and willingness to step up in work situations. For example, “I am more motivated because I feel more secure in my abilities. I didn’t realize that some of my actions were actual techniques.” and “I have noticed a shift in my decision-making. I think more critically now about my interactions with colleagues and those I manage. Am I getting my message across effectively? Am I supporting and encouraging those I manage in the best way? How can I do better at these things?” Finally, after the intervention, 2 participants initiated conversations with supervisors that led to promotions.
Supervisor Feedback
Only 13 of the 32 supervisors (41%) responded, which is disappointing but not unexpected for busy researchers. Supervisors’ responses reinforced the learners’ reports of skills improvement. They recognized strengthened communication skills, demonstrations of more initiative, and stronger coaching and leadership skills. Exemplar quotes include: “A big difference! … He is more assertive, esp. in meetings with more senior leadership.”, “(A) The team was more willing to turn to her for guidance. (B) I noticed that she was better able to communicate with others, especially those outside the team. She provided clear and specific goals making sure that everyone was aware of what needed to be accomplished.” and “She has always been a great team member, but she has been more willing to jump in where she has identified opportunities to improved how things are done.” Two supervisors of newly hired employees noted that it was difficult to assign improvement of skills to the program.
Conclusions
To date, after having completed 2 training cycles, with a total of 32 participants, we find that the STEP-MDP is likely to increase the impact of research teams through development of staff’s mentoring and leadership skills.
Innovation/Uniqueness
Our program embraces impactful learning strategies [Reference Knowles, Holton and Swanson12] and uniquely targets team members other than PIs to build capacity for strong collaborative research. We contribute our experience to the growing literature on research team science, which asks many questions such as: What makes effective teams? and what kinds of training can impact team outcomes? [Reference Cooke and Hilton3]. Most studies in this area focus on training current PIs or future PIs to manage research teams [Reference Knowlton13]. Our work supports the importance of focusing on communication and interpersonal skills in team training, and highlights the value of training staff-level researchers in addition to PIs [Reference McEwan5]. We hope to follow-up and study the impact of the STEP-MDP on team functioning, performance, retention, and professional growth of research staff.
Some participants noted in their evaluation that they worried that skills will decrease over time without continued reinforcement. Although the purpose of creating the PCTs was to provide such ongoing peer support many participants did not continue to meet. In response, we have begun contacting participants on a monthly basis to check in with them, remind them to meet with their PCTs and to ask if they need any guidance in using their STEP-MDP skills. Similarly, while we had sought to develop a RTL CoP by creating and encouraging participants to use a community intranet page, few did. They noted that they were too busy, did not enjoy using “social-media types of systems,” or expressed concerns about privacy given that the page is public. We plan to explore this moving forward. We want to better understand the barriers to and identify strategies that promote a vital CoP for research staff because doing so is important to sustaining learning from STEP-MDP and providing expanding professional development opportunities.
Our study is limited by small sample size, missing data at 3-month follow-up, lack of a control group, short-term follow-up and no direct measure of individual or team function.
Implications/Best Practices
The number of applicants to our program suggests both a perceived need for and motivation of staff to participate in the STEP-MDP. Participants reported improved skills and sense of community with others facing similar challenges (see Fig. 2), and supervisors reported improvements in leadership behavior. We believe developing these core teamwork skills will lead to more collaborative, efficient, and innovative research and expand institutional capacity for translational research, team science and ultimately impact on health of the public.
Acknowledgements
The authors would like to acknowledge Adina Kalet, M.D. and Colleen Gillespie, Ph.D. for their critical comments in the study design and methodology, and for their editorial assistance on this manuscript. The authors acknowledge Mara McCrickard, as well, for her editorial assistance. The authors also thank Michael Pillinger, M.D. and Deborah Keeling, MS of the NYU/H&H CTSI for their support and enthusiasm for this program; the authors would also like to thank all of the staff who participated in STEP-MDP and provided critical feedback which enabled us to evaluate the outcomes of our program, as well as the faculty of NYU Langone Health who encouraged their staff to participate. Funding for this program was supported through the NYU and Health and Hospitals Clinical and Translational Science Institute (CTSI) through grant no. UL1TR001445 from the National Center for the Advancement of Translational Science (NCATS), and through philanthropic support for the Program for Medical Education Innovations and Research (PrMEIR) at NYU School of Medicine.
Disclosures
The authors have no conflicts of interest to declare.
Preliminary Research Presented at
Association for Clinical and Translational Science National Meeting. April 20, 2017. Washington, DC.
Society for General Internal Medicine National Meeting. April 21, 2017. Washington, DC.
Open access
