Hostname: page-component-76d6cb85b7-jhrpq Total loading time: 0 Render date: 2026-07-15T08:51:45.778Z Has data issue: false hasContentIssue false
  • English
  • Français

Implementation of a “Flipped Classroom” for Neurosurgery Resident Education

Published online by Cambridge University Press:  18 September 2017

Fady Girgis  and
Jonathan P. Miller
Fady Girgis
Affiliation:
Department of Neurosurgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio, USA.
Jonathan P. Miller*
Affiliation:
Department of Neurosurgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio, USA.
*
Correspondence to: Jonathan P. Miller, Department of Neurological Surgery, University Hospitals Case Medical Center, 11100 Euclid Avenue, Cleveland, Ohio 44106, USA. Email: Jonathan.Miller@UHHospitals.org
Rights & Permissions [Opens in a new window]

Abstract

Introduction: Engaging residents across a multiyear training spectrum is challenging given the heterogeneity of experience and limited time available for educational activities. A “flipped classroom” model, in which residents prepare ahead of time for mentored topic discussions, has potential advantages. Methods: We implemented a curriculum consisting of topics distributed across the specialty. Weekly, each resident was randomly assigned to research a specific aspect of an assigned topic appropriate to his or her level of experience: junior residents about what characterizes each clinical entity, midlevel residents about when to intervene, and chief residents about how to administer treatment. Residents completed an anonymous survey 6 months after implementation. Board examination performance was assessed before and after implementation. Results: A total of 12 residents participated in the program. Weekly, 1.75±0.40 hours were spent in preparation, with senior residents reporting less time than junior residents. All residents indicated that the accumulation of experience across 7 years of residency was a major advantage of this program, and all preferred it to lectures. Performance on the board examination significantly increased after implementation (from 316±36 to 468±45, p<0.05). Conclusions: The flipped classroom is a viable approach to resident education and is associated with increased engagement and improved performance using validated knowledge-assessment tools.

Résumé

Mise en place du modèle de la « classe inversée » dans le cadre de la formation de résidents en neurochirurgie.Introduction: La participation de résidents à un programme pluriannuel de formation demeure d’une grande exigence étant donné l’hétérogénéité de leurs expériences et le peu de temps imparti à des activités éducatives. Le modèle de la « classe inversée » (flipped classroom), en vertu duquel des résidents préparent à l’avance des sujets de discussion, tout cela dans un contexte de mentorat, comporte des avantages potentiels. Méthodes: Nous avons mis en place un curriculum dans lequel des sujets de la spécialité étaient abordés. À chaque semaine, un résident devait explorer un aspect particulier d’un sujet lié à son niveau d’expérience : par exemple, un résident débutant, ce qui caractérise chaque entité clinique ; un résident intermédiaire, quand intervenir ; un résident chevronné, comment administrer un traitement. Les résidents participant à ce curriculum ont ensuite rempli un sondage anonyme six mois après sa mise en place. Notons enfin que le rendement des résidents, aspect dont l’évaluation incombe à un comité, a été analysé avant et après la mise en place du curriculum. Résultats: Au total, 12 résidents ont participé à ce curriculum. À chaque semaine, 1,75±0,40 heures étaient dédiées à la préparation des activités, les résidents plus chevronnés déclarant avoir besoin de moins de temps que les résidents débutants. Tous les résidents ont par ailleurs indiqué que l’expérience acquise au fil de leurs 7 années de résidence constitue un atout majeur. Ils ont aussi affirmé préférer la « classe inversée » aux exposés magistraux. En terminant, notons que leur rendement a augmenté de façon significative à la suite de la mise en place du curriculum (de 316±36 à 468±45, p<0,05). Conclusions: Le modèle de la « classe inversée » constitue donc une approche viable dans la formation des résidents car il est associé, lorsqu’utilisé avec des outils valides d’évaluation des connaissances, à une participation accrue et à un meilleur rendement de leur part.

Keywords

Small-group teachingTeaching methodsMedical education research

Information

Type
Original Articles
Information
Canadian Journal of Neurological Sciences , Volume 45 , Issue 1 , January 2018 , pp. 76 - 82
Copyright
Copyright © The Canadian Journal of Neurological Sciences Inc. 2017 

Introduction

Education of residents is confronted by several major challenges that are especially acute in lengthy surgical training programs. The volume of information that needs to be mastered is immense, and clinical obligations frequently limit the amount of time that can be devoted to education. It is often difficult to maintain a high degree of engagement in the context of fatigue and frequent clinical distractions. Unlike most other educational environments in which knowledge and experience are relatively uniform across students, residents range from interns who have just finished medical school to chief residents about to go into practice, so it can be difficult to design activities that are appropriate for all levels without being too advanced for the junior residents or too straightforward for the seniors. As a result, formal didactic resident curricula in neurosurgery are rare and generally consist of such traditional techniques as lectures and illustrative case presentations, and much of what must be learned is acquired through independent study.

Recent advances in adult learning theory have revolutionized preclinical and clinical training across a number of specialties. According to the philosophies of constructivism and active learning, adult students learn best when able to fit novel concepts into existing conceptual frameworks, allowing each individual to construct his or her own understanding of a topic based on past experience in a self-motivated manner.Reference Cornelius-White and Harbaugh 1 , Reference Gagnon and Collay 2 One strategy to accomplish this is the “flipped classroom,” conceptualized two decades ago by Mazur,Reference Mazur 3 in which students review materials ahead of time in order to maximize the value of in-classroom discussion (Figure 1).Reference Lage, Platt and Treglia 4 In contrast to traditional classroom instruction in which ideas are introduced in lecture form and subsequently applied during homework assignments, the flipped classroom involves introduction to relevant concepts during a self-directed independent study period followed by application of ideas in a facilitator-guided interactive setting. The flipped classroom has recently been applied to many educational settings at the undergraduate, graduate, and postgraduate levels,Reference Armour, Schneid and Brandl 5 - Reference Yong, Levy and Lape 17 as well as in such professional fields as pharmacy,Reference Bossaer, Panus, Stewart, Hagemeier and George 18 - Reference Koo, Demps, Farris, Bowman, Panahi and Boyle 21 nursing,Reference Betihavas, Bridgman, Kornhaber and Cross 22 medicine,Reference Belfi, Bartolotta, Giambrone, Davi and Min 23 - Reference Veeramani, Madhugiri and Chand 30 and adult instruction in the workplace,Reference Nederveld and Berge 31 and it has been shown to facilitate the development of effective teamwork and problem-solving skills and improve subsequent contextual recall of practical information. However, these principles have rarely been employed for resident educationReference Tainter, Wong, Cudemus-Deseda and Bittner 32 , Reference Young, Bailey, Guptill, Thorp and Thomas 33 and have never been studied in the context of neurosurgical resident training.

Figure 1

The flipped classroom concept. In the traditional learning environment (A), students are introduced to ideas via a lecture provided by a content expert and subsequently apply those ideas in independent study homework assignments. The flipped classroom (B) involves introduction to ideas prior to a group classroom activity where those ideas are applied in an interactive manner. Note the evolution of the faculty preceptor from the primary disseminator of information (“sage on the stage”) to become a facilitator of knowledge transfer from one resident to another (“guide on the side”).

We hypothesized that a flipped classroom model may have advantages for a surgical residency curriculum, including increased efficiency to maximize use of the limited time outside the operating room and the development of a collaborative and interactive environment to maximize resident engagement. In addition, division of labor across different degrees of complexity makes the activity appropriate for both junior and senior residents, allowing everyone to contribute based on their own level of knowledge and experience. We report the results of implementation of a novel comprehensive neurosurgery resident curriculum based on these ideas.

Methods

Design

We designed a flipped classroom curriculum for all the residents in an academic neurosurgery training program based on 40 weekly mentored discussions. Each discussion is devoted to a single neurosurgical diagnosis based on a list of topics distributed across the field of neurosurgery (see Table 2). Several days prior to each meeting, each resident is randomly assigned a specific question to study in preparation for the session based on their level of experience (see Table 1). Junior residents (postgraduate year [PGY] 1–3) are responsible for learning what each condition is: clinical presentation, differential diagnosis, radiology, pathology, etiology, and epidemiology. Senior residents (PGY 4–5), who already have a basic understanding of the disorders, study when to operate: medical and surgical options, indications and contraindications for surgery, and evidence for intervention. Chief residents nearing proficiency (PGY 6–7) are responsible for knowing how the procedures are performed: surgical technique, complications, side effects, outcome, and follow-up. Each resident is instructed to use their choice of available resources (including textbooks, journal articles, atlases, and online resources) in order to obtain a general understanding of the topic in addition to their assigned question in order to maximize the comprehensiveness of the discussion.

Implementation

The neurosurgical curriculum in place at the time of the transition to the new curriculum had not changed significantly during the seven years prior to implementation and consisted of a set of faculty-prepared lectures accompanied by case discussions. The new curriculum replaced these lectures and mentored case discussions with the new flipped classroom activity. During the in-classroom activity, the residents spent 90 minutes as a group sharing knowledge about what they had learned during preparation by individually discussing the assigned questions in order. Following the initial discussion, the group worked through a number of relevant cases to discuss clinical reasoning and medical/surgical considerations for the topic under discussion, with special emphasis on what had been learned during preparation. A neurosurgical faculty member was present and available to answer questions, but the discussion was encouraged to be resident-led, with minimal direct faculty involvement, and this allowed the residents to effectively educate each other.

Assessment

Some six months after implementation, residents were asked to complete a questionnaire to assess their impressions of the program. The assessment was anonymous except for PGY level (1–3, 4–5, or 6–7). The survey consisted of 14 statements on a 5-point Likert-type scale designed to assess efficacy, clinical utility, relevance, engagement, collaboration, and preparation strategies. Free-text entry was employed to assess the amount of time spent in preparation, what materials were used, and what they did and did not like about the program.

All residents in the program were required to take the primary examination for the American Board of Neurological Surgery. This examination comprises 375 multiple-choice questions covering information on disciplines related to the practice of neurosurgery that approximately cover the spectrum of topics addressed by the curriculum. We assessed the raw score and passage rate for the seven years prior to implementation of the program and during the first year after implementation. Formal statistical analysis was performed using Student’s t-test or analysis of variance as appropriate to compare survey results and exam performance, using a two-tailed test, with significance defined as a p value of less than 0.05. The study was reviewed and approved by the local institutional review board.

Results

Some 12 residents regularly participated in the program, including 5 junior residents (PGY 1–3), 4 senior residents (PGY 4–5), and 3 chief residents (PGY 6–7). The average time spent in pre-classroom preparation was 1.75±0.40 hours, with chief residents reporting slightly less preparation time and junior residents slightly more preparation time than the average, though this did not achieve statistical significance (Figure 2). The materials used included textbooks (11/12), journal articles from clinical neurosurgery journals (7/12), and online resources (5/12), with the majority (8/12) of residents using more than one resource. Senior residents were more likely than others to report that it was difficult to find time for preparation, although the total preparation time was highest for junior residents. The sessions were incorporated into a series of academic conferences, so that the total amount of time spent on teaching sessions did not change before or after implementation.

Figure 2

Reported time spent in pre-classroom preparation. There was a trend toward less time spent by more senior residents, but the difference did not achieve statistical significance (p=0.68). Error bars indicate standard error of the mean.

The survey results are shown in Figure 3. All respondents reported that the experience was highly worthwhile and efficient and made them feel more prepared to evaluate and treat patients with conditions that had been discussed using the new teaching format when compared with lectures. Junior residents were slightly less likely to report feeling clinically prepared but more likely to appreciate the interaction with other residents and to feel that the conversation involved equal participation. All residents indicated that the different perspectives provided by the variety of experience levels across seven years of residency was a major advantage of this program, and all indicated that they preferred it to lectures.

Figure 3

Results of 14-question survey. Each question was answered on a 5-point Likert-type scale (1=strongly disagree, 2=disagree, 3=no opinion, 4=agree, 5=strongly agree). All responses were 3 or higher and are shown according to experience level.

In the free-response part of the survey, participants praised the fact that they were “sharing ideas instead of being supplied with information” and indicated that the format was “highly engaging.” They indicated that distribution of the workload made it possible to “cover more than could be accomplished working independently,” and this led to a “sense of teamwork and camaraderie” as they learned to rely on each other. They enjoyed the “variety of having a different task” to research each week and reported that “retention and general knowledge is improving weekly” as participants became more comfortable with the process. Overall, it was found to be a “great educational experience” and “excellent for junior residents in particular.”

Analysis of the results of the written board examination revealed that the average score increased from 316±36 prior to implementation to 468±45 afterward (Figure 4A), which constituted a significant difference (p<0.05, Student’s t-test) and higher than during any of the prior seven years. Some 83% of residents achieved a passing score on the examination (Fig. 4B). While this did not represent a statistically significant difference compared with the prior year (p=0.2, Fisher’s exact test), it was more than twice the average rate across the prior seven years and the highest rate of passage in more than a decade.

Figure 4

Results of the American Board of Neurological Surgery written examination, which was taken by all participants. (A) Average score on the examination for the seven prior years compared with the year following implementation. Error bars indicate standard error of the mean; *p<0.05. (B) Proportion of residents who achieved a passing score on the board examination for the seven previous years compared with the year following implementation. The passing score is based on performance on a curve compared with all residents who take the exam nationally.

Discussion

Advantages of Flipping the Neurosurgery Classroom

According to the Flipped Learning Network, “flipped learning is a pedagogical approach in which direct instruction moves from the group learning space to the individual learning space, and the resulting group space is transformed into a dynamic, interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter.” 34 These principles have many advantages for resident education compared with traditional instruction techniques involving didactic lectures and case discussions. First, by enlisting the residents in the process of teaching, they become more engaged in the process of learning. Emphasis is placed on collaborative learning, since social interaction allows for continued feedback, deeper engagement, and inquiry-based learning, all of which facilitate active building of knowledge.Reference Cornelius-White and Harbaugh 1 , Reference Gagnon and Collay 2 The flipped learning format also resembles the real-world clinical problem-solving environment, which frequently involves consultation with peers and active discussion among team members, which often reveals a diversity of experiences and approaches.Reference Jacobsen 35 Second, by having each resident study a concept appropriate to his or her level prior to discussion in a collaborative session that involves interaction of all the residents, it is possible to leverage the extensive wealth of experience gained across seven years of resident training. Senior residents are able to impart their familiarity and proficiency in treating neurosurgical disorders to junior residents, while juniors are able to share their understanding of the basic disease processes and topics (such as pathophysiology and pharmacology) that are emphasized in medical school education but which are distant from those nearing the end of residency. Rather than being a limitation, the heterogeneity of experience levels becomes an advantage. Third, assignment to learn specific aspects of each topic encourages utilization of the many excellent neurosurgery educational resources that are available, including textbooks, journal articles, and websites. Learning also becomes personalized to each resident’s needs, preferences, accessibility, and pace,Reference Lage, Platt and Treglia 4 , Reference Mazur, Brown and Jacobsen 36 which allows for greater flexibility and control over the educational experience without requiring strict conformation to one teaching style.Reference McLoughlin and Lee 37 , 38 Finally, efficiency is increased and faculty preparation time reduced compared with preparation of didactic lectures. By shifting the primary transfer of information out of the classroom toward pre-classroom preparation, in-class time is no longer absorbed by didactic instruction, and this creates time for assimilation and application of information.Reference Mazur, Brown and Jacobsen 36 , Reference King 39

Prior Studies of Flipped Instruction

The flipped classroom has been investigated in undergraduate and graduate education in a variety of areas, including mathematics,Reference McCallum, Schultz, Sellke and Spartz 11 , Reference Yong, Levy and Lape 17 business,Reference Balan, Clark and Restall 6 , Reference McCallum, Schultz, Sellke and Spartz 11 statistics,Reference Strayer 14 , Reference Winquist and Carlson 16 chemistry,Reference Gross, Pietri, Anderson, Moyano-Camihort and Graham 8 biology,Reference Galway, Berry and Takaro 7 , Reference Jensen, Kummer and Godoy 10 and computer programming.Reference Mok 12 In each of these contexts, the approach has been associated with a more positive experience and preference over traditional techniques.Reference Galway, Berry and Takaro 7 , Reference Howitt and Pegrum 9 , Reference Moraros, Islam, Yu, Banow and Schindelka 13 , Reference Tune, Sturek and Basile 15 Studies comparing test scores between flipped and traditional formats have shown conflicting results: three reports showed improved scores with flipped teaching,Reference Gross, Pietri, Anderson, Moyano-Camihort and Graham 8 , Reference Tune, Sturek and Basile 15 , Reference Winquist and Carlson 16 two reported no difference,Reference Jensen, Kummer and Godoy 10 , Reference Yong, Levy and Lape 17 and one demonstrated worse performance, though this study reported that students had not been held accountable for pre-classroom preparation.Reference Bossaer, Panus, Stewart, Hagemeier and George 18 There is evidence that some topics might be more appropriate than others for this format. For instance, pharmacology students seem to strongly prefer a traditional lecture format, possibly because of an aversion to the preparation time required to learn mechanisms of action that might be covered more efficiently in a lecture setting.Reference Khanova, McLaughlin, Rhoney, Roth and Harris 19 - Reference Koo, Demps, Farris, Bowman, Panahi and Boyle 21 For medical school education, multiple studies have documented that flipped learning is associated with positive experiences, increased enjoyment, improved learning, and better overall satisfaction.Reference Belfi, Bartolotta, Giambrone, Davi and Min 23 - Reference Veeramani, Madhugiri and Chand 30 In addition, of those studies reporting quantitative data in the form of test results, most showed improvements in test scores in the flipped classroom groups,Reference Belfi, Bartolotta, Giambrone, Davi and Min 23 - Reference Gillispie 25 , Reference Liebert, Lin, Mazer, Bereknyei and Lau 27 , Reference O’Connor, Fried and McNulty 29 , Reference Veeramani, Madhugiri and Chand 30 while others found no difference.Reference Heitz, Prusakowski, Willis and Franck 26 , Reference Morgan, McLean, Chapman, Fitzgerald, Yousuf and Hammoud 28

In contrast to undergraduate and graduate education, studies investigating the impact of flipped learning in resident education are scarce but have generally documented good results.Reference Tainter, Wong, Cudemus-Deseda and Bittner 32 , Reference Young, Bailey, Guptill, Thorp and Thomas 33 One studyReference Tainter, Wong, Cudemus-Deseda and Bittner 32 of residents rotating through the intensive care unit demonstrated that flipped instruction was associated with significant improvement in scores of knowledge, confidence, perceived usefulness, and likelihood of skill use across several demographic subgroups and self-identified learning styles. In another study of emergency medicine residents,Reference Young, Bailey, Guptill, Thorp and Thomas 33 the vast majority of participants preferred the format to traditional lectures and felt that the flipped format added to their knowledge.

In spite of its advantages, several criticisms of the flipped learning model have been raised, including the difficulty in accessing pre-classroom preparation materials, the inability to ask questions when viewing material outside of class, and a tendency to become distracted when having to learn content in a less formal environment, as opposed to having the information given to them in a lecture.Reference Mazur, Brown and Jacobsen 36 Another criticism is that the reported positive effects may be due to the incorporation of active learning rather than the order of group discussion in relation to independent study. In a well-controlled experiment by Jensen et al,Reference Jensen, Kummer and Godoy 10 some 108 undergraduate biology students were divided into two groups, one that completed a critical-thinking assignment prior to group discussion and another that reversed this order, and no statistically significant differences were found in test scores, student attitudes, or gains in scientific reasoning ability. It is possible that there are advantages to initial introduction to principles in independent study rather than in a group environment, but this has not been formally studied.

Present Experience with Flipped Learning

Our program differs from the classically described flipped classroom in two important ways. First, we employed a strategy of division of labor to focus efforts along individually assigned questions rather than having all participants learn the same thing.Reference Lage, Platt and Treglia 4 The purpose of this was to facilitate an environment in which residents rely on each other’s expertise and are accountable for acquisition of specific knowledge. The assignment of a different question to each participant requires that everyone involved prepare and participate, so that each participant becomes responsible for the experience of the group as a whole. Second, rather than providing predefined materials to review (such as a video),Reference Lage, Platt and Treglia 4 participants were free to use their own preferred resources to answer their assigned question. The unrestricted use of available resources not only ameliorated the discomfort associated with new learning methodsReference Hutchings and Quinney 40 but also increased the breadth of potential classroom contributions to include a variety of knowledge sources and encouraged participants to share those resources that were most helpful.

Consistent with previous studies in other disciplines, we found that a flipped classroom format was strongly preferred to traditional teaching methods, that participants felt that knowledge retention was improved compared with that in the conventional environment, and that test scores significantly improved after implementation. Because neurosurgery residents include individuals with a wide variation of skills and experience, we were also able to differentiate results based on resident level, and the results of our study provide unique insight into how residents with different experience levels approach educational activities. We found that the increase in clinical confidence after each session was related to experience level, with junior residents expressing less of an impact than seniors. Conversely, junior residents were more likely to prefer the interactive environment and report satisfaction with the format. Interestingly, midlevel residents were the most likely to feel that preparation was too time-consuming, even though they spent less time on average than their more junior residents, which may be a result of different expectations across experience levels for how much time should be required for an activity like this. Residents at all levels tended not to feel intimidated in the group setting and approved of division of labor as a method for maximizing yield with minimal individual effort. Topics were thought to appropriately cover the field, and assignments were felt to be of appropriate complexity. Scores on the board examination and the passage rate dramatically improved for residents of all levels after implementation of the program, which suggests that the curriculum is associated with improved knowledge of neurosurgical principles using an objective and validated assessment tool.

Limitations

There are several important limitations to this study. First, since the analysis is retrospective and observational using historical controls, it is possible that factors other than the curriculum may have contributed to the observed improvement in test performance and learner satisfaction, although the fact that no other concurrent changes were made to the training program make this less likely. Second, the survey we designed is subjective, designed primarily to gauge attitudes about the program, and has not been validated in other contexts. Third, the sample size is very small, so that variance among individual residents may have confounded the overall results. Fourth, there is a minor trend toward improvement in board scores that preceded the study, and only one year of data after implementation is available to us, so that, even though significant, these data cannot be considered definitive. Finally, although total in-class time did not change, participants in the new curriculum were required to spend more time in independent preparation, and this might have contributed to improved performance.

Conclusions

The flipped classroom is a viable approach to resident education that is positively viewed, engaging, and associated with improvements in test performance. The benefits of the flipped method are likely due to facilitation of active learning, where students take responsibility for acquiring knowledge and are held accountable for applying that knowledge in a group setting. Although larger studies are needed to quantify the impact of this technique across other specialties, our data suggest that the approach offers a promising and pragmatic alternative to didactic resident education that may be associated with improved knowledge and enhanced board performance.

Table 1

Each resident is randomly assigned to research one of the topics for the subject being discussed that week

Table 2

Subjects in clinical neurosurgery (some 40 distinct topic areas were defined that cover the spectrum of neurosurgical diagnoses, each topic area was the basis for an individual discussion session)

Disclosures

Fady Girgis and Jonathan Miller hereby declare that they have no conflicts of interest to disclose, and that no portion of this work has been presented previously.

References

1. Cornelius-White, JHD, Harbaugh, AP. Learner-Centered Instruction: Building Relationships for Student Success. Los Angeles: Sage Publications; 2010.Google Scholar
2. Gagnon, GW, Collay, M. Constructivist Learning Design: Key Questions for Teaching to Standards. London: Corwin Press; 2006.Google Scholar
3. Mazur, E. Peer Instruction: A User’s Manual Series in Educational Innovation. Upper Saddle River, NJ: Prentice Hall; 1997.Google Scholar
4. Lage, MJ, Platt, GJ, Treglia, M. Inverting the classroom: a gateway to creating an inclusive learning environment. J Econ Educ. 2000;31(1):30-43.Google Scholar
5. Armour, C, Schneid, SD, Brandl, K. Writing on the board as students’ preferred teaching modality in a physiology course. Adv Physiol Educ. 2016;40(2):229-233.Google Scholar
6. Balan, P, Clark, M, Restall, G. Preparing students for flipped or team-based learning methods. Educ Train. 2015;57(6):639-657.Google Scholar
7. Galway, LP, Berry, B, Takaro, TK. Student perceptions and lessons learned from flipping a master’s level environmental and occupational health course [Perceptions des étudiants et leçons tirées d’une classe inversée pour un cours de maîtrise en santé environnementale et professionnelle] [in French]. Can J Learn Technol. 2015;41(2):1-16. Available at: http://files.eric.ed.gov/fulltext/EJ1064819.pdf. Accessed July 8, 2017.Google Scholar
8. Gross, D, Pietri, ES, Anderson, G, Moyano-Camihort, K, Graham, MJ. Increased preclass preparation underlies student outcome improvement in the flipped classroom. CBE Life Sci Educ. 2015;14(4):ar36. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4710397/. Accessed July 8, 2017.Google Scholar
9. Howitt, C, Pegrum, M. Implementing a flipped classroom approach in postgraduate education: an unexpected journey into pedagogical redesign. Australas J Educ Technol. 2015;31(4):458-469. Available at: https://ajet.org.au/index.php/AJET/article/viewFile/2439/1298. Accessed July 8, 2017.Google Scholar
10. Jensen, JL, Kummer, TA, Godoy, PD. Improvements from a flipped classroom may simply be the fruits of active learning. CBE Life Sci Educ. 2015;14(1):ar5. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4353080/. Accessed July 8, 2017.Google Scholar
11. McCallum, S, Schultz, J, Sellke, K, Spartz, J. An examination of the flipped classroom approach on college student academic involvement. Int J Teach Learn Higher Educ. 2015;27(1):42-55. Available at: http://www.isetl.org/ijtlhe/pdf/IJTLHE1880.pdf. Accessed July 8, 2017.Google Scholar
12. Mok, HN. Teaching tip: the flipped classroom. J Inform Syst Educ. 2014;25(1):7-11.Google Scholar
13. Moraros, J, Islam, A, Yu, S, Banow, R, Schindelka, B. Flipping for success: evaluating the effectiveness of a novel teaching approach in a graduate level setting. BMC Med Educ. 2015;15:27.Google Scholar
14. Strayer, JF. How learning in an inverted classroom influences cooperation, innovation and task orientation. Learn Environ Res. 2012;15:171-193. Available at: http://www.colorado.edu/ftep/sites/default/files/attached-files/strayer_-_inverted_classroom_influences.pdf. Accessed July 8, 2017.Google Scholar
15. Tune, JD, Sturek, M, Basile, DP. Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Adv Physiol Educ. 2013;37(4):316-320.Google Scholar
16. Winquist, JR, Carlson, KA. Flipped statistics class results: better performance than lecture over one year later. J Statistics Educ. 2014;22(3). Available at: http://ww2.amstat.org/publications/jse/v22n3/winquist.pdf. Accessed July 8, 2017.Google Scholar
17. Yong, D, Levy, R, Lape, N. Why no difference? A controlled flipped classroom study for an introductory differential equations course. PRIMUS. 2015;25(9–10):907-921.Google Scholar
18. Bossaer, JB, Panus, P, Stewart, DW, Hagemeier, NE, George, J. Student performance in a pharmacotherapy oncology module before and after flipping the classroom. Am J Pharm Educ. 2016;80(2):31. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4827582/. Accessed July 8, 2017.Google Scholar
19. Khanova, J, McLaughlin, JE, Rhoney, DH, Roth, MT, Harris, S. Student perceptions of a flipped pharmacotherapy course. Am J Pharm Educ. 2015;79(9):140. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4727361/. Accessed July 8, 2017.Google Scholar
20. Khanova, J, Roth, MT, Rodgers, JE, McLaughlin, JE. Student experiences across multiple flipped courses in a single curriculum. Med Educ. 2015;49(10):1038-1048.Google Scholar
21. Koo, CL, Demps, EL, Farris, C, Bowman, JD, Panahi, L, Boyle, P. Impact of flipped classroom design on student performance and perceptions in a pharmacotherapy course. Am J Pharm Educ. 2016;80(2):33. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4827584/. Accessed July 8, 2017.Google Scholar
22. Betihavas, V, Bridgman, H, Kornhaber, R, Cross, M. The evidence for “flipping out”: a systematic review of the flipped classroom in nursing education. Nurse Educ Today. 2016;38:15-21.Google Scholar
23. Belfi, LM, Bartolotta, RJ, Giambrone, AE, Davi, C, Min, RJ. “Flipping” the introductory clerkship in radiology: impact on medical student performance and perceptions. Acad Radiol. 2015;22(6):794-801.Google Scholar
24. Boysen-Osborn, M, Anderson, CL, Navarro, R, et al. Flipping the advanced cardiac life support classroom with team-based learning: comparison of cognitive testing performance for medical students at the University of California, Irvine, United States. J Educ Eval Health Prof. 2016;13:11. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4789594/. Accessed July 8, 2017.Google Scholar
25. Gillispie, V. Using the flipped classroom to bridge the gap to Generation Y. Ochsner J. 2016;16(1):32-36. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4795497/. Accessed July 8, 2017.Google Scholar
26. Heitz, C, Prusakowski, M, Willis, G, Franck, C. Does the concept of the “flipped classroom” extend to the emergency medicine clinical clerkship? West J Emerg Med. 2015;16(6):851-855. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4651581/. Accessed July 8, 2017.Google Scholar
27. Liebert, CA, Lin, DT, Mazer, LM, Bereknyei, S, Lau, JN. Effectiveness of the surgery core clerkship flipped classroom: a prospective cohort trial. Am J Surg. 2016;211(2):451-457.Google Scholar
28. Morgan, H, McLean, K, Chapman, C, Fitzgerald, J, Yousuf, A, Hammoud, M. The flipped classroom for medical students. Clin Teach. 2015;12(3):155-160.Google Scholar
29. O’Connor, EE, Fried, J, McNulty, N, et al. Flipping radiology education right side up. Acad Radiol. 2016;23(7):810-822.Google Scholar
30. Veeramani, R, Madhugiri, VS, Chand, P. Perception of MBBS students to “flipped classroom” approach in neuroanatomy module. Anat Cell Biol. 2015;48(2):138-143. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4488642/. Accessed July 8, 2017.Google Scholar
31. Nederveld, A, Berge, ZL. Flipped learning in the workplace. J Workplace Learn. 2015;27(2):162-172.Google Scholar
32. Tainter, CR, Wong, NL, Cudemus-Deseda, GA, Bittner, EA. The “flipped classroom” model for teaching in the intensive care unit: rationale, practical considerations, and an example of successful implementation. J Intensive Care Med. 2017;32(3):187-196.Google Scholar
33. Young, TP, Bailey, CJ, Guptill, M, Thorp, AW, Thomas, TL. The flipped classroom: a modality for mixed asynchronous and synchronous learning in a residency program. West J Emerg Med. 2014;15(7):938-944. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4251258/. Accessed July 8, 2017.Google Scholar
34. The Flipped Learning Network. What Is Flipped Learning. South Bend, IN: The Flipped Learning Network, 2016. Available at: http://www.flippedlearning.org/cms/lib07/VA01923112/Centricity/Domain/46/FLIP_handout_FNL_Web.pdf. Accessed 6/23/2016.Google Scholar
35. Jacobsen, M. Teaching in a participatory digital world. Educ Can Mag. 2015;55(3). Available at: http://galileo.org/teaching-in-a-participatory-digital-world/. July 8, 2017.Google Scholar
36. Mazur, AD, Brown, B, Jacobsen, M. Learning designs using flipped classroom instruction [Conception d’apprentissage à l’aide de l’instruction en classe inversée] [in French]. Can J Learn Technol. 2015;41(2):1-26.Google Scholar
37. McLoughlin, C, Lee, MJW. The three P’s of pedagogy for the networked society: personalization, participation, and productivity. Int J Teach Learn Higher Educ. 2008;20(1):10-27. Available at: http://files.eric.ed.gov/fulltext/EJ895221.pdf. Accessed July 8, 2017.Google Scholar
38. National Center on Universal Design for Learning. Technology CfAS. Wakefield, MA: National Center on Universal Design for Learning; 2015. Available at: http://www.udlcenter.org/aboutudl/udlguidelines_theorypractice. Accessed July 20, 2017.Google Scholar
39. King, A. From sage on the stage to guide on the side. Coll Teach. 1993;41(1):30-35. Available at: http://www.udlcenter.org/aboutudl/udlguidelines_theorypractice/. Accessed July 20, 2017.Google Scholar
40. Hutchings, M, Quinney, A. The flipped classroom, disruptive pedagogies, enabling technologies and wicked problems: responding to “The Bomb in the Basement.” Electron J e-Learn. 2015;13(2):106-119.Google Scholar
Figure 0

Figure 1 The flipped classroom concept. In the traditional learning environment (A), students are introduced to ideas via a lecture provided by a content expert and subsequently apply those ideas in independent study homework assignments. The flipped classroom (B) involves introduction to ideas prior to a group classroom activity where those ideas are applied in an interactive manner. Note the evolution of the faculty preceptor from the primary disseminator of information (“sage on the stage”) to become a facilitator of knowledge transfer from one resident to another (“guide on the side”).

Figure 1

Figure 2 Reported time spent in pre-classroom preparation. There was a trend toward less time spent by more senior residents, but the difference did not achieve statistical significance (p=0.68). Error bars indicate standard error of the mean.

Figure 2

Figure 3 Results of 14-question survey. Each question was answered on a 5-point Likert-type scale (1=strongly disagree, 2=disagree, 3=no opinion, 4=agree, 5=strongly agree). All responses were 3 or higher and are shown according to experience level.

Figure 3

Figure 4 Results of the American Board of Neurological Surgery written examination, which was taken by all participants. (A) Average score on the examination for the seven prior years compared with the year following implementation. Error bars indicate standard error of the mean; *p<0.05. (B) Proportion of residents who achieved a passing score on the board examination for the seven previous years compared with the year following implementation. The passing score is based on performance on a curve compared with all residents who take the exam nationally.

Figure 4

Table 1 Each resident is randomly assigned to research one of the topics for the subject being discussed that week

Figure 5

Table 2 Subjects in clinical neurosurgery (some 40 distinct topic areas were defined that cover the spectrum of neurosurgical diagnoses, each topic area was the basis for an individual discussion session)