Virtual patient encounters have traditionally been useful for patients with geographic or mobility limitations. ^{Reference Yeroushalmi, Maloni, Costello and Wallin1} Lu et al. reported reliability in cognitive, stroke and movement disorder assessments conducted through video. ^{Reference Lu, Veet, Aljundi, Whitaker, Smith and Smith2} Multiple sclerosis (MS) patients can benefit from virtual care, since this condition requires consistent follow-up and monitoring of treatment. ^{Reference Soelberg Sorensen, Giovannoni, Montalban, Thalheim, Zaratin and Comi3} D’Haeseleer et al. found most MS patients to be satisfied with telecommunication. ^{Reference D’Haeseleer, Eelen, Sadeghi, D’Hooghe, Van Schependom and Nagels4} During the peak of the COVID-19 pandemic, virtual consultations became crucial to maintain continuity of patient care amid infectious restrictions. ^{Reference Portnoy, Waller and Elliott5,Reference O’Sullivan and Schneider6,Reference Al Hussona, Maher and Chan7}

Despite the benefits of virtual neurological assessments, there are potential concerns regarding their effectiveness. The neurological exam is unique in the practice of neurology, consisting of hands-on strength, reflex and sensory testing to assist in the localization of neurological lesions. ^{Reference Stanley, Hall and Mullally8,Reference Nicholl and Appleton9,Reference Zasler10}

Potential disadvantages of virtual neurological exams include limited patient compliance, reliance on patients’ technological capabilities and limited evidence regarding their ability to monitor disease progression. ^{Reference Yeroushalmi, Maloni, Costello and Wallin1,Reference D’Haeseleer, Eelen, Sadeghi, D’Hooghe, Van Schependom and Nagels4,Reference Bashshur, Doarn, Frenk, Kvedar, Shannon and Woolliscroft11} Although some studies demonstrate high agreement between video and in-person assessment with the Expanded Disability Scale in MS patients, the sample sizes are small. ^{Reference Lu, Veet, Aljundi, Whitaker, Smith and Smith2} Furthermore, there is some evidence that telemedicine may be less accurate for new outpatient neurological consultations; however, there is limited data on follow-up assessments. ^{Reference Watila, Duncan and Mackay12} The question has been raised regarding the accuracy of the virtual neurological exams compared to in-person evaluations. The COVID-19 pandemic presented a unique opportunity to investigate the reliability of virtual neurological assessment compared to in-person evaluation, primarily in MS patients as well as those with other neurological disorders.

Patient charts were retrospectively reviewed at Sunnybrook Health Sciences Centre tertiary outpatient neurological clinic in Toronto, Ontario, Canada, between March 23, 2020, and March 23, 2021, during the peak of the COVID-19 pandemic. Patients included in the study had at least one virtual appointment followed by one subsequent in-person assessment to allow comparison of clinical impression across various neurological conditions. In-person assessments occurred within days when there was a concerning virtual assessment or within one year for routine annual visits when COVID-19 restrictions were lifted. The same physician conducted all the assessments using the standard video examination as explained by Al Hussona et al. ^{Reference Al Hussona, Maher and Chan7}

If the physician’s clinical impression of stability remained unchanged from the virtual to the in-person assessment, this was considered to be “No Change.” “Clinical disparities (DISP)” were cases where there was a mismatch between what was interpreted virtually versus what was seen in person. This was defined as either (a) patients reporting progression virtually but showing no significant changes on in-person exam with an alternative explanation for complaints or (b) patients reporting stability virtually, but significant changes were found on in-person exam. Patient encounters were also considered to be “Clarified” when the in-person exam confirmed virtually reported findings and helped to clarify management – specifically, when the virtual impression suggested progression of disease and the in-person assessment confirmed this impression.

A total of 1484 patients were reviewed: 230 patients were excluded with only one appointment, no virtual appointments or no in-person assessment. Thus, 1036 patients were analyzed. The average time between virtual and in-person assessments was one year for routine follow-ups, except in 35 cases where clinical concerns were raised and patients were seen within a few days.

Table 1 outlines the neurological conditions reviewed in this analysis, in which the majority of patients had MS (69.0%, n = 715). The remaining patients had other inflammatory disorders, seizure disorders, headache disorders, movement disorders, cognitive complaints and other neurological conditions.

Table 1.

Prevalence of neurological disease in patient cohort. The percentage of patients with neurological disease, the percentage of telephone and video consultations for each disease and the percentage of those patients with clinical disparities (DISP)

Figure 1 displays the number of virtual consultations included. The majority of patients (n = 748 or 72.2%) had one or more virtual consultations through telephone, and the remaining (n = 288 or 27.8%) were through video. From virtual to in person, there was “No Change” in clinical impression for most patients (n = 758 or 73.2%), that is, stable virtually and stable in person. Patients with suspicion of a change in clinical status virtually were confirmed on in-person exam in 114 out of 748 telephone consultations (15%) and 21 out of 288 video consultations (7.3%), as seen in the “Clarified” category. Otherwise stated, 114 out of 218 (52.3%) telephone disparities and 21 out of 60 (35%) video disparities were “Clarified.” In total, consistent clinical impressions between virtual and in-person assessments were obtained 86.2% of the time. Imaging assisted in clinical decisions in some cases, but the clinical impression was the driving factor in decision-making.

Figure 1.

Flow chart of neurological virtual assessment outcomes in retrospective study. Patients are categorized into initial video versus telephone consultations. These consultations are further grouped into: clinical disparities (DISP), mismatch between virtual and in-person impression; Clarified, virtual impression suggested disease progression and in-person confirmed; and No Change, virtual impression suggested stability and in-person confirmed. The percentages represent the proportion of the total number of patients.

There was an overall “DISP” of 13.8% (n = 143) from virtual to in-person assessments. A similar proportion of incongruent conclusions were seen with telephone (104 of 748 telephone assessments) and video consultations (39 out of 288 video assessments) at 13.9% and 13.5%, respectively. Figure 1 shows the DISP in relation to the total number of consultations: 10.0% with telephone (104 out of 1036) and 3.76% DISP with video (39 out of 1036). Additionally, 3.38% (n = 35) of patients stated they were rapidly deteriorating virtually but were found to be stable in their neurological condition after in-person assessment, with alternative explanations identified such as functional or non-neurological causes. Of the DISP cases, 2.32% (n = 24) of all patients stated they were stable with the consultation virtually, but significant changes were seen with the in-person exam in their one-year follow-up.

In this study of a cohort of patients in a single center, 86.2% of virtual assessments aligned with subsequent in-person assessments, suggesting high reliability and accuracy. Hophing et al. reported similar findings in a general neurology population, with 86% specificity and 56% sensitivity for virtual assessments compared to in-person evaluations. ^{Reference Hophing, Tse and Naimer14}

While our study demonstrates the clear benefits of virtual assessments, it also highlights potential limitations. In 13.8% of assessments, there was clinical disparity (DISP), in which the in-person neurological exam led to a change in clinical opinion, for both initial telephone and video appointments. This underscores the value of the in-person neurological exam in specific situations, as supported by the existing literature. ^{Reference Stanley, Hall and Mullally8,Reference Nicholl and Appleton9} The majority of the disparities were in cases of patients reporting worsening of their clinical status virtually, but in-person examination revealed an alternative explanation for their symptoms such as underlying infection or non-neurological causes, averting a change in medical treatment. These results align with those of Hophing et al., who found that in-person exams changed the localization and diagnosis in approximately 15% of the cases.Reference Hophing, Tse and Naimer ¹⁴ Importantly, 2.32% (n = 24) of patients in our study reported clinical stability virtually, yet significant changes were identified with the in-person neurological examination, leading to a change in management. These findings suggest that subtle physical examination findings, such as weaknesses, abnormal reflexes or sensory changes, may be missed without direct patient contact. Patients were not brought in earlier as they did not endorse worsening symptoms and continued to endorse stability during the in-person assessment,

Unexpectedly, both video and telephone appointments showed similar rates of clinical discrepancy, specifically 13.5% and 13.9%, respectively. This contrasts with Rush et al., who suggested video assessments confer more reliability than telephone. ^{Reference Rush, Howlett, Munro and Burton13} One possible explanation for this finding is that patients with technological limitations may have struggled with video assessments, reducing their effectiveness. However, given the relatively small sample sizes, further study is warranted to compare the accuracy of the different modalities of virtual care.

The majority of cases with clinical DISP in our study were observed with MS patients and were less frequent with migraine, headache, seizure and epilepsy patients. This may reflect the varying role of the neurological examination across different pathologies. In MS, progression is often monitored by physical findings, in addition to their history, cognitive and neuro-imaging findings. In contrast, other neurological pathologies such as seizures and migraines may be more reliably assessed through patient history, allowing for more accurate virtual monitoring, though exceptions can certainly exist. Regardless, much larger cohorts of different diseases will be needed in further studies to see if the disparity exists, given the limited sample size of the non-MS population in our study.

The strength of the study is the inclusion of a relatively large cohort of patients with predominantly MS pathology, contributing valuable insights into the utility of virtual medicine in MS care. Additionally, the same physician conducted the virtual and in-person assessments of all patients, reducing inter-rater variability. Important limitations of the study include the limited neurological examination during telephone assessments (solely history with mental status, language and speech assessment), along with the length of time up to a year between virtual and in-person visits, especially in this MS-predominant population where progression may have reasonably occurred between assessments. The study’s retrospective design is a further limitation.

Despite these limitations, this study supports virtual medicine for patient management in outpatient neurology, especially during the pandemic. This retrospective study demonstrated that virtual assessments are accurate and reliable in most clinical settings. However, acknowledging its limitations is prudent, and in-person neurological examinations can address gaps in virtual care. Further research is needed to examine its reliability across a broader range of neurological conditions in a prospective fashion.