INTRODUCTION
Background
Scholarly activities and, in particular, research are critical to advance modern medicine.Reference Bensing 1 Although still a relatively new specialty, emergency medicine (EM) has a rich history of practice-changing evidence development and training enhancement in order to advance the field.Reference Suter 2 In medical disciplines, foundational research literacy is considered a core competence.Reference Frank, Snell and Sherbino 3 For residents, early involvement in research has been shown to enable future scholarly work.Reference Sanders, Fulginiti and Witzke 4 Scholarly projects during residency represent opportunities to develop focused research questions, to plan study protocols, to execute data collection, as well as to analyse and interpret the data. The dissemination of results represents an opportunity for residents to enhance written and oral communication skills. These experiences also impart essential critical appraisal skills and prepare those with academic inclinations to be research producers.Reference Summers, Fish, Blanda and Terndrup 5
Dual college nature of Canadian EM training
Canada is unique with two distinct training streams for EM. Each program has a different philosophy and capacity to provide resident research education, and each was considered in the recommendations development. The most significant difference is training time (i.e., 5 years in the Royal College of Physicians and Surgeons of Canada [RCPSC-EM] programs and 1 year of EM training following 2 years of family medicine residency in the College of Family Physicians of Canada [CFPC-EM] programs). Beyond the challenges for longitudinal scholarly projects, there are constraints to achieve EM clinical competence during the training time allotted for CFPC-EM residents.
Importance
Resident research is essential for developing future EM researchers and advancing the science of EM, both of which ultimately improve clinical care for patients. Given the diversity of the emergency department patient population and the challenges of providing leading edge emergency care, emergency physicians seek evidence to guide clinical decisions. Currently, for Canadian EM residency programs, the following elements of resident research are unknown: nature of research education and how it is provided, type of exposure to scholarly projects, and resulting scholarly outputs.
Objectives
We sought to characterize the current state of Canadian EM resident research and develop a series of pragmatic recommendations to promote excellence in this area.
METHODS
Design
At the direction of the Canadian Association of Emergency Physicians (CAEP) Academic Section, we assembled an expert panel and advisory committee, which included representation from RCPSC-EM and CFPC-EM programs from across the country. We systematically reviewed the existing literature on EM resident research and performed a national survey of the current state of resident research education and output among Canadian EM residency programs. Using these data, we developed preliminary recommendations via a consensus process with experts in EM education and research. These recommendations were refined from discussions at a national academic symposium held at the CAEP 2014 Conference. We obtained approval for the survey from The University of Western Ontario Research Ethics Board for Health Sciences Research Involving Human Subjects.
Systematic literature review
With the assistance of a research librarian, we developed a search strategy using predefined Medical Subject Headings (MeSH) and free text words: research$, peer review, publication$, publishing, educat$, training$, academic$, scholar$, project, education, residen$, emergency service, critical care, and trauma. Restricting language to English or French, we searched MEDLINE, Embase, and ERIC using Ovid from 1946 to November 2013. All potentially relevant citation titles, abstracts, and then full articles were independently reviewed by two authors (LC, SM), and disagreements were resolved by consensus. We included all articles that were relevant to EM resident research, consisted of descriptions or evaluations of resident research programs, or described standards for resident research. We excluded editorials, letters to the editor, and articles about a single resident research project or research programs for fellows. We categorized the resulting articles into themes and distributed them among panel members for critical appraisal and summary.
Survey
Our literature search did not encounter any survey instruments to determine the current state of EM resident research. Through an iterative process, we developed a 36-item online survey instrument designed to address four main themes arising from our literature search: 1) expectations and requirements, 2) training and assessment, 3) infrastructure and support, and 4) dissemination. Prior to distribution, we piloted the survey among panel members. We sent an email request with a survey link to program directors and assistant program directors for all RCPSC-EM and CFPC-EM programs. We excluded subspecialty programs, such as pediatric EM, prehospital medicine, ultrasound, simulation, and medical education. The link included a letter indicating that responses would be anonymized and reported in aggregate form only. Respondents received email reminders, and, in some cases, resident research coordinators and research directors were contacted to obtain responses. We analysed the resulting data using descriptive statistics reporting means, medians, and proportions.
Recommendation development
We derived the recommendations by group consensus in an iterative fashion after discussion of the literature and generated survey data. Once refined, we distributed the draft recommendations for review and feedback to the Specialty Committee of the RCPSC and the Emergency Medicine Special Interest Focused Practice Committee of the CFPC. We also distributed the draft recommendations to the CAEP Resident Section. Finally, we presented the draft recommendations for discussion at an academic symposium at the CAEP 2014 Conference, which was attended by 80 individuals, including EM program directors, researchers, administrators, attending emergency physicians, and residents. The final recommendations were subsequently revised based on the feedback obtained.
Definitions
The committee recognized early on in discussions that there was a need for clear definitions of a scholarly project and the desired levels of competence for residents completing such projects. We developed the definitions described in Boxes 1 and 2 through an iterative process, including feedback from the CAEP Academic Symposium.
Box 1 Proposed definition of a scholarly project
Scholarly project: an educational method to acquire competencies for academic inquiry via a systematic approach to define a relevant problem and design a response, with reference to previous literature and theory.
Box 2 Proposed levels of competence
1. Critical appraiser of research : This physician can demonstrate research literacy and the ability to critically appraise the evidence, incorporating this into evidence-based clinical decision making.
2. Research contributor : This physician is a skilled research consumer and participates in contributing to original research, albeit not as the primary investigator.
3. Research producer : This physician is a skilled research consumer and possesses advanced training in research methods, engaging in the process of generating original ideas, obtaining funding, executing research, and disseminating it in a scholarly fashion.
RESULTS
Systematic literature review
Our search strategy yielded 654 potentially relevant citations. After elimination of duplicate abstracts (n=170) and reports that did not satisfy inclusion criteria (n=184), we reviewed 300 citation titles and abstracts then selected 35 articles (Figure 1), which were classified into themes: 1) expectations and requirements, 2) training and assessment, 3) infrastructure and support, and 4) dissemination.
Figure 1 Flow of article selection from systematic literature review of MEDLINE, Embase, and ERIC.
Survey
We achieved a 100% response rate for our national survey of Canadian EM program directors (14 RCPSC-EM, 17 CFPC-EM). Table 1 describes the program characteristics. The majority reported requiring residents to complete a scholarly project (76% CFPC-EM and 79% RCPSC-EM).
Table 1 Characteristics of 31 responding Canadian EM residencies
* Data are from caper.ca 2012-2013.
For RCPSC-EM programs, the majority of projects involved prospective data collection (71%), educational projects (64%), quality assurance projects (64%), surveys (57%), critical appraisals of topics (57%), or health records reviews (43%) (Table 2). In contrast, the CFPC-EM program projects included critical appraisals of topics (59%); development of research protocols (47%), quality assurance projects (47%) or health records reviews (35%); prospective data collection (35%); or education projects (35%).
Table 2 Types of scholarly projects for 31 Canadian EM residencies
For both EM programs, more than half of them expected residents to prepare an abstract from their scholarly project, with the majority expecting submission locally. Very few of the CFPC-EM programs expected submission to national or international conferences (Table 3). In contrast, half of RCPSC-EM programs expected national conference submissions, and over half of these expected international venues. Just over half of the RCPSC-EM programs expected a manuscript to be prepared and submitted. Only two CFPC-EM programs expected manuscript preparation, and one expected manuscript submission.
Table 3 Program expectations for 31 Canadian EM residencies
Nearly all RCPSC-EM programs and half of CFPC-EM programs reported providing formal training to residents on how to conduct research (Table 4), typically delivered by varying instructors from epidemiologists to EM faculty to non-faculty researchers. A wide variety of resources were reportedly available to residents on how to conduct resident research. In terms of assessment, 64% of RCPSC-EM and 54% of CFPC-EM programs reported evaluating their resident scholarly projects. Less than half of the programs did so formally, and others used a hybrid approach (formal and informal). One half of RCPSC-EM and CFPC-EM programs reported performing individual assessments, and less than 20% of programs reported evaluating their overall resident research activities.
Table 4 Formal education, resources, and infrastructure for 31 Canadian EM residencies
* PhD scientist, librarian, non-clinician researchers.
** NERD block, online listing of staff interests and projects, university postgraduate resources, quality assurance resources.
† REB = Research Ethics Board.
‡ Support provided ad hoc by faculty, plans for a resident research coordinator, family medicine department resources.
We found that a majority of programs reported infrastructure in place to support resident research. Most RCPSC-EM programs (83%) reported having a resident research coordinator, as did half of CFPC-EM programs. However, the nature of support residents reportedly received for key stages of project execution was highly variable (Table 5). In most cases, funding for staffed positions to support resident research was internally derived.
Table 5 Infrastructure provided to residents
#REB = Research Ethics Board.
* Support provided ad hoc by faculty, plans for a resident research coordinator, and family medicine department resources.
Finally, in terms of dissemination of the results of resident research, we found a marked variability regarding programs’ self-report of resident abstracts and manuscripts that appeared to be independent of program size (Figures 2 and 3).
Figure 2 Number of abstracts and manuscripts reported by RCPSC-EM programs* over the past 3 years reported by the number of residents in the program, as large (>25), medium (11–25), or small (<11).
Figure 3 Number of abstracts and manuscripts reported by CFPC-EM programs* over the past 3 years reported by the number of residents in the program, as either medium (11–25) or small (<11).
Academic symposium
The importance of mentorship, clarification of the role of a research coordinator, importance of individual v. program-wide evaluations, and broadening the definition of what constitutes dissemination of scholarly work were issues raised during the academic symposium.
DISCUSSION AND RECOMMENDATIONS
We found a paucity of literature describing the current state of EM resident research in Canada. Most was more than 10 years old and US-based. Development and application of critical appraisal skills are inextricably linked to the development of research skills, and a recently published survey found that 85% of RCPS-EM programs have established evidence-based medicine curricula.Reference Bednarczyk, Pauls, Fridfinnson and Weldon 7 One example of explicit training for resident research was a mixed-methods study describing a unique 1-month mandatory rotation for junior RCPSC-EM residents, called the NERD block (Novel Education in Research and Design).Reference Abu-Laban, Jarvis-Selinger, Newton and Chung 8
Our survey highlighted marked diversity in residency programs’ self-report expectations, training, evaluation, infrastructure, and dissemination, not just between RCPSC-EM and CFPC-EM programs but also among them. The expected degree of project completeness also varied across programs. We found a lack of clarity in the goals and objectives of the scholarly project within EM residency. The research methods training that residents reportedly received was highly variable, indicating that a single consolidated resource repository would be helpful. Few residencies reported formally evaluating their resident research programs, and dissemination of resident research is inconsistent. Although a centrally mandated approach is likely undesirable, the variability across and within programs suggests that improvements are possible, which our panel believes could lead to increased academic and research productivity. Ultimately, enhancements to EM research would lead to ongoing improvements to patient care.
Expectations and requirements recommendations (Box 3)
a. Relevant literature
We sought to determine the requirements of both colleges for resident research. The RCPSC’s Objectives of Training in EM describe a scholarly project requirement: “Complete at least one scholarly project that is suitable for peer reviewed publication or presentation at a national academic meeting, as attested by the Program Director.” 9
Box 3 Recommendations
1. Expectations and Requirements Recommendations
1.1. Specific clarification is recommended around the goals and exit competencies regarding research training and assessment for emergency medicine residents from both colleges, so they are clearly aligned with the college training objectives. The desired competency outcome (i.e., critical appraiser, research contributor, and/or research producer) and an explicit definition for scholarly project should be declared.
1.2. For the RCPSC-EM programs, a range of research opportunities should be available to meet all competency outcomes, including a stream for advanced research training (e.g., MSc or PhD). The recently revised CanMEDS Scholar milestones are relevant to this process.
1.3. RCPSC programs should develop a research training curriculum for their residents matched to desired competency outcomes.
1.4. For the CFPC-EM programs, a clear path (i.e. explicit objectives and expectations) for those interested in further research training should be defined.
2. Training and Assessment Recommendations
2.1. The RCPSC should address the variability across its programs and consider the advantages of a structured research training curriculum offered early to provide residents more research opportunities.
2.2. Given the two family medicine years plus one emergency medicine year in CFPC-EM training, the CFPC should identify specific research training elements necessary to supplement the family medicine research training curriculum.
2.3. All residency programs should assess individual resident research projects as well as evaluate their resident research programs as a whole.
3. Infrastructure and Support Recommendations
3.1. All programs should strongly consider using a resident research coordinator to facilitate research ethics board application, project management, publication, and program evaluations.
3.2. Resident research programs should link to existing infrastructure within their institution (in other programs or medical departments) to assist residents with scholarly project design and statistical analysis.
3.3. All programs should dedicate research funding support toward resident research.
3.4. All programs should consider matching residents with research mentors (who can provide methodological and pragmatic support). This can also occur outside of the program and/or the institution.
4. Dissemination Recommendations
4.1. All residency programs should encourage their residents to broadly disseminate project results (whether by traditional publication or other peer-reviewed, citeable venues), including manuscript preparation by RCPSC residents and abstract presentation by all residents (at least locally, but ideally nationally).
4.2. Resident research abstracts and manuscript publications should be tracked as a metric for the evaluation of academic productivity, at least locally, but ideally nationally and/or internationally.
The CFPC-EM programs are guided by the Specific Standards for Family Medicine Residency Programs Accredited by the CFPC: The Red Book, which states:
“The acquisition of critical appraisal skills is essential. Programs might require residents to complete an academic project. For those residents who wish to pursue an academic project (research, literature review, quality improvement), the program should provide the opportunity to do so.” 10 Because we found no further literature describing the expectations and requirements for EM resident research in Canada, our panel developed and refined a proposed definition of a scholarly project at the CAEP 2014 Academic Symposium (see Box 1).
b. Relevant survey data
The majority of program directors surveyed indicated that they require scholarly projects of their residents. The types of acceptable scholarly projects were felt to be appropriate, allowing for a diversity of designs. There were some programs, particularly CFPC-EM, which restricted projects to either critical topic appraisals or generation of a research protocol. We noted variability in expectations for dissemination, with most programs requiring an abstract submission to local venues and a smaller number to national or international conferences. An even smaller number required manuscript submission. Qualitative survey responses indicated that program directors felt that the key reasons for not requiring manuscript submission included lack of an RCPSC-EM or CFPC-EM requirement and insufficient time during the training program.
Training and assessment recommendations
a. Relevant literature
We found that much of the published literature is out of date and originated from the United States. Although several papers identified proposed curricula or identified a need for resident research training, they failed to account for recent improvements in epidemiology and critical appraisal education.Reference Brautigan 11 , Reference Olson, Hamilton and Angelos 12 The existing literature suggested that, although EM resident research training is important, it is still not ideal. No study demonstrated improvements from specific resident research training or evaluation of long-term outcomes, such as academic productivity, or pursuit of an academic career.
b. Relevant survey data
Our survey highlighted both differences and similarities between RCPSC-EM and CFPC-EM residency programs. Most RCPSC-EM programs (92%) reported providing formal research education, in contrast to 50% of CFPC-EM programs. RCPSC-EM programs typically reported more extensive curricula, often including education from epidemiologists and statisticians, whereas CFPC-EM programs more frequently reported informal or optional research education. A number of CFPC-EM programs identified the 1-year training duration as a barrier. Two programs indicated that they rely, in part, on education provided during the two family medicine years. Most EM residency programs, especially the CFPC-EM, reportedly assessed resident research abilities, often via presentation at an EM research day.
c. Best practices
The RCPSC-EM residency research education initiative, NERD block, was identified as a unique program that delivers a 1-month comprehensive curriculum with four goals (obtaining research knowledge, obtaining critical appraisal skills, advancing a research project, and increasing resident/faculty collaborations).Reference Bednarczyk, Pauls, Fridfinnson and Weldon 7 The program demonstrated pre- and post-improvement of research knowledge and may merit consideration by other RCPSC programs. Academic symposium attendees supported the notion of a longitudinal curriculum that matches to goals and competencies (Box 2).
Infrastructure and support recommendations
a. Relevant literature
There was a limited number of infrastructure and support studies related to EM resident research. Program director support, faculty size, resident research awards, and overall financial support were positively associated with resident research publications.Reference Levitt, Terregino, Lopez and Celi 13 Similarly, research funding and support (specifically at least two of a dedicated lab space, support for lab research assistant, dedicated research nurse or study coordinator, statistician or assistant with a PhD) were positively associated with EM residents graduating and entering an academic career.Reference Stern, Kim and Neacy 14
b. Relevant survey data
We found that over 86% of the EM programs reported some form of infrastructure support, including 83% of RCPSC-EM and 53% of CFPC-EM programs having resident research coordinators. Some survey respondents planned to hire a resident research coordinator. Most of the staffed positions to support resident research were internally funded.
c. Academic symposium findings
Academic symposium attendees endorsed the importance of a research coordinator, and several sites attributed increased research success to a coordinator. One caveat raised was limiting the scope of the role to facilitation, because it was felt to be important that residents still maintained responsibility for the work. Attendees also stressed the importance of mentorship for scholarly projects success, and a national network of mentors was suggested.
Dissemination recommendations
a. Relevant literature
Research publication remains an important measure of academic productivity.Reference Henderson and Brestky 15 Henderson and Brestky retrospectively identified 3,551 EM articles published over a 7-year period where the academic affiliation of the first author was recorded to identify factors positively associated with the number of publications. Faculty size and program location were found to be independently associated with increased research publications. The authors suggested that programs with higher research productivity may also have more experienced research mentors and specific resources for residents and faculty.
b. Relevant survey data
Our survey results suggested that almost all Canadian EM residency programs have some infrastructure support and dedicated personnel for resident research. However, assistance for abstract submission, poster development, and manuscript preparation is markedly low in nearly half of programs. Internal funding to support resident research was reportedly available in 67% CFPC-EM and 83% RCPSC-EM programs, and was usually to offset conference travel costs. Although all RCPSC-EM programs and 94% of CFPC-EM programs hold a local EM “Research Day,” only 67% of CFPC-EM programs expected their residents to present. All of the RCPSC-EM and 63% of CFPC-EM programs reported having residents who presented abstracts over the past 3 years at national or international conferences (e.g., CAEP, Society for Academic Emergency Medicine). Over the past 3 years, the number of abstracts reported by RCPSC-EM programs ranged from 4 to 50, compared to 1 to 8 abstracts presented by CFPC-EM residents. Similarly, more RCPSC-EM programs (92%) than CFPC-EM programs (38%) reported publishing manuscripts.
c. Academic symposium findings
Academic symposium attendees suggested that 1) scholarly dissemination can be broadly defined, 2) future EM research producers should be strongly encouraged to publish during residency, and 3) expectations for dissemination should be matched to levels of competence (see Box 2).
LIMITATIONS
Although the literature search strategy we used was comprehensive, relevant articles may have been missed. The survey is potentially subject to recall bias, particularly for the number of published abstracts and manuscripts. Although the recommendations were developed from the literature, survey data, and input from stakeholders, key issues may not have emerged from academic symposium attendees or other consulted experts. The results are also limited, because subspecialties, including pediatric EM, were not included within the scope of this work.
CONCLUSION
As EM matures, the time is right to strengthen Canadian EM resident research. The future of Canadian EM research lies with our residents, and action should be taken to encourage resident research involvement and productivity. This includes 1) clear articulation by RCPSC and CFPC resident research goals, 2) clarification of the expectations and opportunities for future research critical appraisers and research producers, 3) boosting of existing training, 4) evaluations of resident research programs, 5) use of resident research coordinators, 6) linking with existing research infrastructure, 7) leveraging of internal academic funding sources to strengthen infrastructure, and 8) encouragement of the dissemination of high-quality research beyond local venues. These recommendations will assist RCPSC and CFPC, program directors, research directors, and residents to strive for excellence in Canadian EM resident research.
Acknowledgements
The authors would like to thank the advisory panel, including Tim Dalseg, Debra Eagles, and Avik Nath, as well as the librarian Alla Iansavitchen.
Competing interests: Dr. Riyad Abu-Laban, a core panel member and co-author, is also the rotation director of the 1-month “NERD Block” at the University of British Columbia research rotation discussed in the paper and lead author of the publication describing this. Dr. Abu-Laban provided input and perspective to the committee on the NERD Block rotation but was not involved in final decisions as to how this was characterized in the manuscript or the resulting final recommendations.
Dr. Jason Frank, a core panel member and co-author, was also involved in the development of the guide for The Royal College, The Royal College program directors handbook: a practical guide for leading an exceptional program (2014).
