Neuroanatomical localisation (NL), the process of integrating clinical signs and symptoms to identify neuroanatomical abnormalities, is a key skill in neurology and the first step in making a neurological diagnosis. Team-based learning (TBL), a student-centric active learning strategy that follows a defined sequence of three phases (independent preparation, individual and team readiness assessment, knowledge application including team discussion) ^{Reference Parmelee, Michaelsen, Cook and Hudes1} , has been employed to improve NL teaching. ^{Reference Tan, Kandiah, Chan, Umapathi, Lee and Tan2}

The coronavirus disease 2019 (COVID-19) pandemic has disrupted medical education, with educational delivery pivoting from classroom-based learning to virtual platforms. TBL is traditionally an in-person teaching format, and the effectiveness of blending technology with TBL remains unclear. ^{Reference River, Currie, Crawford, Betihavas and Randall3} Studies describing entirely virtual TBL protocols have not yet evaluated knowledge outcomes. ^{Reference Gaber, Shehata and Amin4}

This study aimed to evaluate the feasibility and effectiveness of virtual TBL in teaching NL. We hypothesised that virtual TBL is both feasible and effective in terms of learner reactions and knowledge.

The study population comprised 18 voluntary participants from a class of 108 final year medical students from the Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore. TBL is the school’s principal classroom learning and teaching strategy across all years of the five-year programme, with around 60% of curriculum time dedicated to in-person TBL in the first two preclinical years.

Ethics approval was obtained from the Nanyang Technological University Institutional Review Board (IRB-2020-06-037). All participating students were 21 years and older and provided informed consent.

The primary outcome of this post-test-only study was effectiveness of virtual TBL indicated by post-TBL learner reactions and knowledge outcomes based on the Kirkpatrick Model of Evaluation. ^{Reference Kirkpatrick5}

The Team-based Learning Student Assessment Instrument (TBL-SAI) is a validated tool to evaluate learner reactions to TBL based on ratings in three subscales: accountability, preference for lecture or TBL and student satisfaction. ^{Reference Mennenga6} Questions were added or modified to evaluate outcomes specific to virtual TBL (modified TBL-SAI). Mean scores awarded by the students on a five-point Likert scale were measured (Kirkpatrick Level 1 outcome). The Extended Matching Questionnaire (EMQ) is a validated tool developed in Singapore by two of the authors (NCKT, KT) and assesses knowledge and clinical reasoning outcomes in NL. ^{Reference Tan, Chin and Yau7} Mean EMQ scores expressed as a percentage of the total score were measured (Kirkpatrick Level 2 outcome).

A single group study was conducted with a TBL session on NL. The TBL session comprised three phases ^{Reference Parmelee, Michaelsen, Cook and Hudes1} : independent self-study with preparatory materials provided (Phase 1) followed by individual assessment (Individual Readiness Assurance Test; IRA), team assessment (Team Readiness Assurance Test; TRA) using four randomly pre-assigned teams of 4-5 students (Phase 2) and application exercises (AEs) (Phase 3) (Appendix 1 in Supplementary material).

For Phase 1, students were given preparatory materials on NL. One week later, Phases 2 and 3 were conducted virtually over Zoom as a single session over two hours. Participants attended the session remotely. They were hosted within a single Zoom session with the use of “breakout rooms” for team discussions. Phases 2 and 3 each covered 5-6 case vignettes on NL. The teams justified their answers, and contentious questions were discussed. The facilitator (PR) moderated the discussion and neurologists (NCKT, KT) clarified concepts where necessary.

The modified TBL-SAI was conducted immediately after the TBL session, whereas the EMQ was conducted one week after the TBL session as a timed close-book exercise to assess medium-term knowledge retention. Neurologists (NCKT, KT) debriefed the participants on the EMQ immediately thereafter as an additional teaching-learning activity. The TBL questions, modified TBL-SAI and EMQ were hosted on iLAMS, a secure digital platform.

Mean Likert ratings on the modified TBL-SAI and mean EMQ percentage scores were measured to evaluate virtual TBL effectiveness in terms of learner reactions and knowledge outcomes, respectively. Spearman’s non-parametric correlation coefficients were calculated to estimate the strength of the relationship between IRA scores, TRA scores, the difference between IRA and TRA scores (IRA-TRA Delta), and EMQ scores as data were non-normally distributed. Statistical analyses were performed with SPSS 26.0 with p < 0.05 considered as significant.

All 18 participants completed the virtual TBL session, modified TBL-SAI and EMQ. Mean IRA score was 87.8 (SD 2.9), and all groups attained 100.0 (SD 0) for TRA with a mean IRA-TRA Delta of 12.1 (SD 2.9). Mean EMQ score was 84.2% (SD 2.9).

Participants reported good student satisfaction scores on the modified TBL-SAI. Out of a maximum of 5.0, students gave mean ratings of 3.9 (SD 0.3) on accountability, 3.3 (SD 0.7) on preference for virtual TBL and 3.9 (SD 0.3) on satisfaction subscales with overall rating of 3.6 (SD 0.5).

There was moderate correlation between IRA and EMQ scores (ρ = 0.587, p = 0.01). There was no meaningful correlation between IRA and TRA (ρ = 1.00, p = 0.00) or IRA-TRA Delta (ρ = −1.00, p = 0.00) since all groups attained 100.0 (SD 0) for TRA. There was no correlation between IRA scores and overall TBL-SAI rating (ρ = 0.048, p = 0.850) or the accountability (ρ = 0.421, p = 0.08), preference (ρ = 0.090, p = 0.722) or satisfaction (ρ = −0.412, p = 0.08) subscales.

TBL has been used in undergraduate neurology education with one previous study demonstrating good student engagement with greater improvement in pre- and post-test scores in NL with TBL compared to passive learning. ^{Reference Tan, Kandiah, Chan, Umapathi, Lee and Tan2} Our findings suggest that virtual TBL is both feasible and effective in achieving good levels of student satisfaction and knowledge. From the modified TBL-SAI, good student satisfaction scores were obtained with the highest subscale rating of 3.9 (SD 0.3) out of 5.0 obtained on the satisfaction subscale. Virtual TBL was deemed preferable to in-person TBL with an average rating of 3.3 (SD 0.6), although participants reported better focus and content retention with in-person TBL.

While previous studies on virtual TBL only reported student satisfaction outcomes, ^{Reference Gaber, Shehata and Amin4} our study further evaluated knowledge outcomes. Mean EMQ scores obtained by our study population comprising undergraduate medical students were 84.2% (SD 2.9). In comparison, performance of junior doctors in internal medicine residency training ranged from 81.8% (SD 12.1) to 83.0% (SD 1.6) on the same EMQ previously. ^{Reference Tan, Chin and Yau7,Reference Loh, Rotgans, Tan and Tan8} While the results may be due to high intrinsic ability of the participants, it is unlikely for undergraduate medical students to have comparable pre-test knowledge in a complex topic such as NL compared to postgraduate residents, and high mean EMQ scores may reflect true knowledge gain from the TBL session.

However, there are other possible reasons for the high EMQ mean scores. First, the student participants were tested immediately after the TBL, whereas the residents were tested after 6–12 weeks of clinical work and may have been less engaged in the content. Furthermore, the EMQ questions may be pitched at the level of medical students and lacked the complexity to discriminate between medical students and doctors. More studies are also required to investigate whether higher EMQ scores translate into better clinical performance.

In our study, IRA scores correlated with EMQ scores whereas the other TBL scores (TRA, IRA-TRA Delta) did not. This suggests that the largest determinant of knowledge outcome is the preparatory phase of TBL (Phase 1). There are some possible explanations. Schmidt et al. previously hypothesised “knowledge reconstitution” as the psychological basis of TBL ^{Reference Schmidt, Rotgans, Rajalingam and Low-Beer9} ; independent consolidation during Phase 1 may be the most crucial to facilitate knowledge reconstitution during the subsequent Phases 2 and 3 of TBL and translated to knowledge outcomes. Another plausible explanation is the change in the delivery platform. The independent preparatory phase of TBL (Phase 1) remained constant despite the shift to virtual TBL. On the other hand, pertinent group discussion components of TRA and AE (Phases 2 and 3) were conducted via Zoom breakout rooms. This may have resulted in a diminished learning value from Phases 2 and 3, and in the process, highlighting effects of Phase 1.

Few authors have investigated if all elements of TBL contribute towards learning to the same extent. For example, Gopalan and colleagues found that omitting the IRA negatively impacted TRA performance. ^{Reference Gopalan, Fox and Gaebelein10} More of such studies are required to systematically evaluate the extent of learning in each phase, the synergistic effects between phases and how that impacts performance.

This study had some limitations. First, results may be affected by selection bias as students who volunteered may have had greater interest and exposure to neurology. Second, the study was conducted in a single medical school on one neurology topic. Due to the need for social distancing during the COVID-19 pandemic, participants could not be randomised to an in-person TBL to act as controls and study design was limited to a single group study with a small sample size. Further larger studies are required to assess the generalisability of the findings and corroborate them with an experimental study design. However, this study provides a basis for virtual TBL as a feasible teaching method with potential for multi-centre virtual TBLs across different locations and institutions. This has applicability outside of the social distancing requirements during the COVID-19 pandemic for medical schools where students are deployed across different campuses or training hospitals.

In conclusion, this study demonstrates that virtual TBL is feasible for teaching neurolocalisation. Virtual TBL achieves good student satisfaction and knowledge outcomes, with the preparatory phase of TBL appearing to be the largest contributor. Future studies to further evaluate knowledge outcomes of virtual TBL and the extent of learning in each TBL phase will be beneficial.