Introduction
Scholarly activity is an important aspect of the training and development of most medical professionals and occurs during summer electives, mandatory curricular activities, extracurricular research activities, and/or longitudinal research experiences. For entering medical students, the Association of American Medical Colleges outlines research-related competencies, such as quantitative reasoning, critical thinking, and written communication [1]. A 2015 meta-analysis reported that 72% of US medical students were interested in conducting research, while 31% were interested in a career that included research [Reference Amgad, Man Kin Tsui, Liptrott, Shash and Manalo2]. Students who participated in research activities while in medical school were 3.55 times more likely to be interested in research in their future careers [Reference Amgad, Man Kin Tsui, Liptrott, Shash and Manalo2]. The 2023 Association of American Medical Colleges Medical School Graduation Questionnaire reported that 84.4% of medical school graduates participated in a research project, 63.7% submitted a paper for publication, and 48.8% planned to participate in research during their careers [3].
For residents, the Accreditation Council for Graduate Medical Education requires participation in scholarly activity before the completion of training [4]. While most residency programmes have established guidelines for scholarly activities that align with accreditation requirements, only 37% reported having an organised, comprehensive research curriculum and only 70% taught skills important to research [Reference Alguire, Anderson, Albrecht and Poland5]. The benefits of resident research exposure are well described in the literature and include increased lifelong learning, improved patient care, increased satisfaction with training, and higher likelihood of pursuing academic careers (reviewed in [Reference Zimmerman, Alweis, Short, Wasser and Donato6– Reference Bhuiya and Makaryus8]). Nonetheless, multiple barriers exist including a lack of mentors, lack of research infrastructure, lack of trainee interest, lack of financial support, the high demand of clinical responsibilities, and the lack of research curricula.
To establish fundamental research skills and fill gaps within training curricula, we previously piloted an online, asynchronous set of modules called Research 101 to introduce medical students to various topics that are relevant to the conduct of research [Reference Blackard, Knapke and Ryan9]. Here, we evaluated the use of Research 101 for internal medicine residents.
Methods
Details on the creation and pilot study of Research 101 have been described elsewhere [Reference Blackard, Knapke and Ryan9,Reference Blackard, Knapke, Schuckman, Veevers, Hardie and Ryan10]. This research qualified as minimal risk to participants and was exempt from most of the requirements of the Federal Policy for the Protection of Human Subjects by the University of Cincinnati Institutional Review Board.
Research 101 modules were offered asynchronously through the online educational platform Canvas (Salt Lake City, UT). Each module consisted of learning objectives, a pre-module survey, assignments, and a post-module survey. For each module, participants completed 1) a pre-module survey before they reviewed any assignments associated with a module, 2) all assignments in a module, and 3) a post-module survey after each module. The pre-module survey included questions based on the learning objectives for each module with responses given on a 5-point Likert scale. For example, “I am confident in my ability to identify my skills as a mentee / trainee” – which corresponds to learning objective #1 for the Aligning Expectations module or “I am confident in my ability to describe possible barriers to an effective mentor-mentee relationship” – which corresponds to learning objective #4 for the same module. The post-module survey also included the learning objective-based questions, as well as two additional questions with yes / no / unsure as options: 1) The content of this module met my needs? and 2) Would you recommend this module to a friend if it was not a requirement?. Open-ended text fields were also included in the post-module surveys: 1) What did you like most about this module?; 2) What did you like least about this module?; and 3) If you could change one thing about this module, what would it be?
The Research Electronic Data Capture tool hosted at the University of Cincinnati was used to collect and manage all survey data [Reference Kiger and Varpio11]. For qualitative survey responses, a thematic analysis approach was utilized where data were coded to construct thematic patterns across participant responses [Reference Kiger and Varpio11]. Changes in module Likert scale scores were assessed by subtracting pre-module from post-module Likert scores, and positive differences indicated increased confidence in knowledge regarding the module content. Statistical significance in pre-post scores was evaluated using a paired t-test (SAS Version 9.4). For group comparisons, pooled p-values were reported when the test for equality of variances (Folded F) was >0.05. When the equality of variances test was <0.05, the p-value from the Satterthwaite method was reported.
In February 2023, internal medicine residents (postgraduate year 2) at One Brooklyn Health, including Brookdale Hospital Medical Center and Interfaith Hospital in Brooklyn, NY were registered for Research 101 as part of an ongoing effort to improve resident scholarly activity in an inner-city safety net hospital system. Participation was required for all 71 residents. Resident progress was encouraged by a research coordinator who sent periodic reminders to those who had not yet completed the course. Resident knowledge, confidence, and satisfaction were assessed using pre- and post-module surveys with 5-point Likert scaled questions (1 – not confident, 2 – slightly confident, 3 – moderately confident, 4 – mostly confident, and 5 – 100% confident), open-ended text responses, and a multiple-choice quiz. Survey responses, assignment responses, and quiz grades were confidential. Sixty residents responded to at least one REDCap survey, while 46 completed all surveys. Participation was required and was confirmed by completion of a final quiz consisting of 25 multiple-choice questions with one correct answer per question. Participants had access to all Research 101 content during the final quiz, feedback on incorrect responses was provided, and there was no time limit for the completion of the quiz.
Results
Table 1 shows pre-module and post-module survey results for each module and learning objective.
Pre-module and post-survey scores for research 101 modules and learning objectives. N = the number of participants responding to each post-module question
*p < 0.0001.
A small number of participants did not complete all survey questions; thus, the number of responses is not identical across all modules. Prior to starting, residents were most confident with the
Introduction to research (range 2.76 – 2.96), aligning expectations (range 2.87 – 2.90), and study design and data analysis basics (range 2.79 – 2.96) modules. Residents were least confident with the Submitting an Institutional Review Board (range 2.57 – 2.71) and presenting your summer research (range 2.61 – 2.70) modules. Post-module mean scores were significantly increased for all modules and all learning objectives compared to pre-module scores (p < 0.0001).
As shown in Table 1, “The content of this module met my needs” was endorsed highly across all modules (91.4% “yes” responses). “No” and “unsure” responses were highest for the Introduction to research module and lowest for the Conducting a literature search and Transparency, rigor, and reproducibility modules. Across all modules, “Would you recommend this module to a friend if it was not a requirement?” received 81.8% “yes” responses compared to 3.32% “no” and 14.9% “unsure” responses. “No” and “unsure” responses were highest for the Introduction to research and lowest for the Conducting a literature search modules. From a high score of 25, the median final quiz score was 23 (range: 13 – 25).
Thematic analysis of open-ended text responses from the post-module surveys identified several strengths of Research 101. Residents noted that the video assignments were high quality and concise but also provided the right level of detail for learners new to research. The examples, cases, and resources such as external websites and documents that were provided were helpful, and learners appreciated having the material organized for them in a sequential, logical manner. The qualitative data also identified several areas for improvement within Research 101 including not liking the longer video assignments, even more examples would be useful, and the website was difficult to navigate at times and required a lot of clicking.
Discussion
Research training is a highly desired component of medical education and professional development. It has been suggested that resident research may improve clinical care by fostering clinical evaluation skills, clinical reasoning, and lifelong learning [Reference Hebert, Levine, Smith and Wright12]. Early exposure to research may also increase the number of physician-scientists [Reference Hebert, Levine, Smith and Wright12–Reference Hall14]. Notably, several barriers to conducting research during residency exist, including the lack of mentors, training opportunities, curricular time, background instruction, financial support, and infrastructure, as well as the pressures of significant clinical responsibilities [Reference Alguire, Anderson, Albrecht and Poland5].
Structured research training is important for residency programmes and is included in current ACGME requirements [4]. Moreover, residency training programmes with organized programmes/curricula, including protected time for research, were associated with increased productivity [Reference Laupland, Edwards and Dhanani15]. However, existing curricula may not be available across different institutes or permit modification based on individual programmatic needs [Reference Alguire, Anderson, Albrecht and Poland5,Reference Hebert, Levine, Smith and Wright12]. A systematic review of resident research programmes found that the most commonly taught topics in a research curriculum (when present and reported) were research methods, scientific writing, and biostatistics, as well as literature searches, Institutional Review Board structure, and research ethics [Reference Zimmerman, Alweis, Short, Wasser and Donato6]. Programmes that had a research director and a structured research curriculum were associated with higher productivity.
Research 101 was developed to provide a comprehensive and structured introduction to crucial topics in research in a highly accessible and modifiable format. The online, asynchronous format facilitates a research training infrastructure that is highly flexible and enables individualized learning and/or programme-specific adjustments. Evaluating Research 101 in residents demonstrated significant learning as post-module survey scores were significantly higher than pre-module scores for all modules and all learning objectives. Final quiz scores were also quite high on average. A systematic review of resident research curricula found that evaluation methods were often rudimentary, infrequently reported obstacles encountered or modifications made to the curriculum, and rarely reported pre–post-intervention testing of learners’ knowledge [Reference Hebert, Levine, Smith and Wright12]. Thus, we consider the evaluative component of Research 101 to be a major strength.
This study has several limitations of note. First, Research 101 was piloted initially with a small number of medical students and then expanded later to include additional learner types such as residents. We recognize that distinct research educational needs are likely based on learner type and that the modules within Research 101 may require adaptation to these specific needs in subsequent iterations. Second, individuals who participated may have research interests or had prior research experiences that may contribute to selection bias amongst those completing Research 101. While not available for this iteration, the subsequent version of Research 101 now includes demographic data, as well as responses to the questions 1) have you ever published a peer-reviewed manuscript and 2) are you currently involved in the conduct of research? Third, self-efficacy/knowledge was not assessed using a standardized instrument and instead utilized a non-validated scale. Fourth, the fact that completion of Research 101 was required by the residency programme directors may have unduly influenced the findings reported here. Fifth, although Research 101 is offered asynchronously, several topics included would benefit from more direct interactions with learners. Thus, we must emphasize that Research 101 does not replace in-person interactions; rather, it provides an additional option for research education given the limited space for new content within existing resident curricula while also acknowledging the varying learning systems of learners.
Research 101 provides a comprehensive overview of research-relevant topics that complement the learning environment for residents who are preparing for or already conducting research projects. It can serve as an important addition to the research training toolkit with clear evidence of enhanced learning and confidence in research-related understanding.
Author contribution
Conception and design of the work – JTB and ST; collection or contribution of data–JTB and ST; contributions of analysis tools or expertise – JK, SS, JV, WH, RY, AK, PL, ST, PR, and JTB; conduct and interpretation of analysis–JK and PR; drafting of the manuscript – JTB; editing of the manuscript–JK, SS, JV, WH, RY, AK, PL, ST, PR, and JTB.
Funding statement
This work was supported by the University of Cincinnati Center for Clinical and Translational Science and Training through grant UL1TR001425 and short-term medical student training grants T35 DK060444 to JTB and T35 HL113229 to WH.
Competing interests
The authors have no conflicts of interest to declare.
Open access
