CLINICIAN’S CAPSULE
What is known about the topic?
Canadian guidelines recommend that suspected transient ischemic attack (TIA) patients receive brain imaging in the emergency department; yet, high requisition rates for non-cerebrovascular patients exist.
What did this study ask?
What is the hypothetical impact that a clinical decision support tool (CDST) would have on computed tomography angiography (CTA) requisitions for suspected TIA patients.
What did this study find?
CDST use would have resulted in more TIA patients receiving CTA, while imaging fewer non-cerebrovascular patients.
Why does this study matter to clinicians?
A CDST could assist clinicians in applying the Canadian stroke guidelines as pragmatically as possible.
INTRODUCTION
The management of transient ischemic attack (TIA)/minor-stroke in the emergency department (ED) is challenging.Reference Prabhakaran, Silver and Warrior 1 The risk of a recurrent stroke after TIA is greatest during the first 24 hours immediately following the event,Reference Johnston, Rothwell and Nguyen-Huynh 2 – Reference Coutts, Modi and Patel 5 with approximately 50% of recurrent strokes occurring during this period.Reference Chandratheva, Mehta and Geraghty 4 The risk of a recurrent stroke after TIA, therefore, constitutes a medical emergency requiring urgent intervention to maximize positive outcomes. Early recognition of the condition and initiating appropriate investigations and treatments are key to ensuring positive outcomes. The Canadian stroke best practice guidelines recommend that all patients with motor/speech deficits suspected of TIA who present to the ED less than 48 hours after symptom onset receive vascular imaging.Reference Coutts, Wein and Lindsay 6 , Reference Casaubon, Boulanger and Blacquiere 7
Suspected TIA patients with carotid stenosis, large vessel occlusion, and intracranial atherosclerosis are at greatest risk of stroke recurrence.Reference Purroy, Montaner and Molina 8 Previous studies have demonstrated a high prevalence of these etiologies among TIA patients.Reference Sheehan, Kyne and Kelly 9 , Reference Smith, Lev and English 10 Several guidelinesReference Coutts, Wein and Lindsay 6 , Reference Casaubon, Boulanger and Blacquiere 7 , Reference Jauch, Saver and Adams 11 , Reference Easton, Saver and Albers 12 recommend vascular imaging of suspected TIA patients in the ED for the early detection of these high-risk conditions, specifically carotid stenosis (Evidence Level A). Approximately 2% of TIA patients receive carotid endarterectomy (CEA),Reference Rappeport, Simonsen, Christensen and Boysen 13 with the number needed to treat to prevent one recurrent stroke equal to 5 when performed within 2 weeks of the initial event.Reference Rothwell, Eliasziw and Gutnikov 14 Computed tomography angiography (CTA) is recognized as a preferential vascular imaging investigation (Evidence Level C) to conduct when TIA/minor stroke is clinically suspected.Reference Coutts, Wein and Lindsay 6 , Reference Casaubon, Boulanger and Blacquiere 7 , Reference Graham, Menon and Coutts 15 , 16
A challenge in adhering to the guidelines is the high prevalence of low-risk, non-cerebrovascular conditions (e.g., migraine; see Table 1) that mimic stroke in clinical presentations (i.e., stroke mimics), and which can account for 40% to 60% of patients referred to fast-track TIA units.Reference Prabhakaran, Silver and Warrior 1 , Reference Ferro, Falcão and Rodrigues 17 – Reference Fonseca and Canhão 20 Unnecessarily performing CTA on stroke-mimic patients represents an inappropriate use of radiological resources and limits access to higher-risk patients. Further, to identify patients at highest risk of stroke recurrence, CTA investigations are recommended to be conducted in the ED prior to discharge. However, limited radiological resources means that the CTA decision for some patients must be delayed, with patients sent to outpatient TIA units. These patients represent an opportunity to improve TIA management.Reference Coutts, Modi and Patel 5 Finally, previous research has suggested that vascular imaging is frequently underutilized within Canadian EDs to assess suspected TIA patients.Reference Hosier, Phillips and Doucette 21 As such, present-day ED practices do not currently align with best practice recommendations. The gap between evidence-based guidelines and physician practice is widespread across medical specialties, and clinical decision support tools (CDSTs) have been suggested as a way of reducing this gap.Reference Bates, Kuperman and Wang 22 , Reference Sim, Gorman and Greenes 23
Distribution of ED, SRAU, combined ED + SRAU, and CDST CTA requisitions for the 10 most frequent stroke-mimic diagnoses (N=865)
CDST=clinical decision support tool; ED=emergency department; NYD=not yet diagnosed; Other=non-typical mimic diagnosis; SRAU=Stroke Rapid Assessment Unit;
TGA=transient global amnesia.
With our study, we hope to bridge the gap between stroke best practice recommendations and practicing ED physicians. Our group has developed an electronic CDST to assist in decision-making for CTA imaging of suspected TIA patients in ED settings. Our tool is based upon a logistic regression model (clinical classifier) that we previously developed and validated to identify TIA patients in the ED,Reference Bibok, Penn and Lesperance 24 – Reference Bibok, Votova and Balshaw 26 and incorporates the Canadian stroke best practice guidelines.Reference Coutts, Wein and Lindsay 6 This decision support simplifies adherence to the guidelines, which contain numerous details and conditional logic. Our decision support tool thus assists ED physicians with decision-making by making the best pragmatic use of the guidelines and evidence-based care in an end-user informed format.
In the present retrospective study, we aim to determine the hypothetical impact that the use of our CDST would have had on CTA requisitions at the institutional level if it had been available for use by ED physicians. We hypothesize that use of the CDST would have resulted in more TIA patients and fewer stroke-mimic patients receiving CTA imaging relative to the combined baseline CTA requisition patterns in three urban hospitals.
METHODS
Participants
The Stroke Rapid Assessment Unit (SRAU), Victoria, BC, Canada, is a specialized outpatient stroke unit servicing Vancouver Island (population: 799,400). The SRAU receives referrals from EDs, family practice, and specialists (e.g., ophthalmologists). The referral form for the unit is known as the ACVS (acute cerebrovascular syndrome) Assessment Form 27 - Reference Kidwell and Warach 29 and has been in use since November 2014. We specifically developed the assessment form to capture all of the data elements required by our clinical classifier, as well as to improve referral triage within our unit.Reference Bibok, Votova and Balshaw 26 Referral forms are entered verbatim by unit staff into the SRAU electronic medical record (EMR) system.
Neurological evaluation and diagnosis
Patients received standard of care treatment by unit neurologists upon arrival at the SRAU. Radiological investigations frequently include magnetic resonance imaging (MRI) and CTA. Unit neurologists diagnose patients on the basis of both clinical and radiological findings, the combination of which has been argued to represent the gold standard in stroke diagnosis.Reference Whiteley, Tseng and Sandercock 30 , Reference Whiteley, Wardlaw and Dennis 31 Possible diagnostic categories include “TIA/minor stroke,” “stroke-mimic conditions,” “not yet diagnosed” (NYD), and “hemorrhagic stroke.”
TIA and minor stroke are grouped together because they both represent the lower end of the brain ischemia continuum,Reference Albers 28 , Reference Kidwell and Warach 29 with TIA defined as a “brief episode of neurologic dysfunction caused by focal brain or retinal ischemia, with clinical symptoms typically lasting less than one hour, and without evidence of acute infarction…[t]he corollary is that persistent clinical signs or characteristic imaging abnormalities define infarction — that is, stroke,”Reference Albers, Caplan and Easton 32 (p1715) with minor stroke defined as a stroke with a National Institutes of Health Stroke Scale (NIHSS) of<4.Reference Fischer, Baumgartner and Arnold 33
Stroke-mimic conditions are diagnosed when possible, with the most frequently identified being migraine, vestibulopathy, syncope, and neuropathy (see Table 1). Less common stroke-mimic diagnoses are classified as “Other” and include a variety of less common conditions, including congenital strabismus, and medication side effects.
A small portion of referred patients are not seen at the SRAU (i.e., “no shows”). Reasons for non-attendance include 1) patient refused appointment, 2) patient admitted to hospital with recurrent TIA/stroke, 3) patient seen by other physician/specialist, 4) inappropriate referral, 5) patient seen by inpatient neurology, and 6) death.
For analysis purposes we created a patient classification variable that indicated either patients’ SRAU diagnoses (e.g., TIA, stroke mimic, NYD, hemorrhagic stroke) or reason for unit non-attendance.
Clinical decision support tool
The CDST that we have developed was informed by data from a health informatics research study conducted to determine the operational requirements for the design of an electronic decision support tool to triage suspected TIA patients in the ED.Reference Lau, Partridge and Penn 34 Activities involved in this study included ED workflow analysis and ED physician and nurse focus groups. A complete description of the CDST and clinical classifier can be found in the supplement.
Variables
Clinical variables consisted of the data elements from the ACVS Assessment Form. 27 Variables represented by checkboxes were treated as binary variables, with unchecked items interpreted as the absence of the given data element (0=absent; 1=present). Blood pressure was measured in millimeter of mercury (mm Hg), and patient age was recorded in years. Patient sex was treated as a binary variable (0=female; 1=male). CTA status (0=not done; 1=done), date of CTA investigation, and date of TIA unit arrival were also extracted from the SRAU EMR. CEA status was extracted from the regional hospital electronic health record system.
A variable representing institutional level CTAs (ED + SRAU) was also derived. The institutional level encompasses the entire patient care pathway for TIA management and includes both frontline, referring ED physicians and TIA unit staff and neurologists. We will use the term, hospital clinicians, henceforth, to refer to ED physicians and TIA unit neurologists who can requisition CTA for suspected TIA patients.
From the referral form information, we derived a binary variable to indicate the absence or presence of characteristic TIA symptoms (0=absent; 1=present), for example, language disturbance, speech disturbance, and face droop. Details are provided in the supplement.
Sample
The study sample consisted of ED patients consecutively referred from three urban EDs to the SRAU between January 2015 and December 2016. The three EDs (Victoria General Hospital, Royal Jubilee Hospital [Victoria, BC], Nanaimo Regional General Hospital) all have CTA imaging availability on-site. Upon extraction from the SRAU EMR, the sample consisted of 1,679 ED referred patients. Referral systolic and diastolic blood pressure readings were missing for 104 of these patients, with an additional 5 patients missing only referral diastolic blood pressure. Missing blood pressure values were imputed using mean substitution as described in the supplement (mean systolic=142 mm Hg; mean diastolic=78 mm Hg). Table 2 displays the demographic characteristics of the data set prior to corrections for missing blood pressure.
Demographics stratified by diagnosis
* t and chi-square homogeneity test.
† Variable was coded as present if patients’ referral data indicated any of the following symptoms: a) unilateral limb weakness; b) unilateral limb numbness; c) language disturbance (i.e., aphasia); d) speech disturbance (i.e., dysarthria); e) face droop; f) visual field deficits; g) unsteadiness (i.e., ataxia); h) diplopia; i) “curtain” descending over field of vision (i.e., amaurosis fugax); and j) vision loss.
‡ All medications referring to those that patients were currently taking or were prescribed in the ED at the time of their assessment in the ED;
§ Any statin medication at any dose.
¶ Values prior to mean substitution correction for missing blood pressure values.
ASA=acetylsalicylic acid; CT=computed tomography; CTA=computed tomography angiography; ED=emergency department; MRI=magnetic resonance imaging; Other=all patient classifications not Mimic or TIA.
Ethics statement
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Study approval (reference number H2009-114) was granted by the Health Research Ethic Board of the Vancouver Island Health Authority, Victoria, BC, Canada.
Statistical analysis
Our retrospective analysis compares the allocation of CTA requisitions by hospital clinicians among true TIA and stroke-mimic patients referred from the ED to our fast-track TIA unit, with the allocation that would have occurred had the CDST been available in the ED.
To compare the performance of our CDST for CTA requisition with hospital clinicians’ CTA requisition baseline, we first applied the logistic regression model constituting the CDST (see the supplement for model coefficients) to calculate the estimated probability of TIA/minor stroke for each patient in the data set. Estimated probabilities were then filtered by the presence of characteristic TIA/minor-stroke symptoms: If the characteristic TIA symptoms variable was coded as absent, then the estimated probability was set to zero; otherwise, the estimated probability remained unchanged. The estimated probabilities were then dichotomized using two pre-established cutpoints (≥0.516 for flagging potential TIA cases, and ≥0.662 for recommending CTA). A discussion of the derivation and rationale for these cutpoints can be found in the supplement.
To evaluate the effect of the CDST on the number of imaging requisition, we compared the total number of CTA orders by hospital clinicians with those of the CDST for all consecutively referred ED patients, regardless of diagnosis. To evaluate the performance of the CDST’s recommendations on CTA requisitions, we restricted the sample to only those patients who received a diagnosis of TIA or stroke mimic at the SRAU. We calculated the diagnostic odds ratio (DOR)Reference Glas, Lijmer and Prins 35 of hospital clinicians and the CDST with respect to ordering CTA for TIA patients. We compared the sensitivity, specificity, and accuracy of hospital clinicians CTA orders with those of the CDST using McNemar tests.Reference McNemar 36 The diagnostic performance of the CDST (i.e., flagging possible TIA cases) is also presented, although no comparisons with hospital clinicians are possible because the referring physicians by definition will have 100% sensitivity for TIA patients in this sample of referred patients.
Analyses were completed using the ROCR (v1.0.7),Reference Sing, Sander, Beerenwinkel and Lengauer 37 pROC (v1.12.1),Reference Robin, Turck and Hainard 38 Hmisc (v4.1.1),Reference Harrell and Dupont 39 and rms (v5.1.2)Reference Harrell 40 libraries in the R statistical language (v3.4.4). 41
RESULTS
Table 3 summarizes the frequencies of patient classifications for the 1,679 patients referred to the SRAU. Of the patients referred to the SRAU, 1,537 (91.5%) attended the unit and were assessed by unit neurologists; 142 referred patients did not attend the unit (i.e., “no show”). Of “no show” patients, the reason for unit non-attendance was unknown for 2 patients.
Distribution of ED, SRAU, combined ED + SRAU, and CDST requisitions for CT, MRI, and CTA by patient classification
CDST=clinical decision support tool; CT=computed tomography; CTA=computed tomography angiography; ED=emergency department; MRI=magnetic resonance imaging; NYD=not yet diagnosed; SRAU=Stroke Rapid Assessment Unit.
A total of 23 patients were admitted to the hospital with recurrent TIA/stroke prior to their SRAU appointment. The mean time to TIA/stroke recurrence from the initial ED visit to hospital admission was 35.5 hours (range, 5.8-91.2); 10 (43.5%) of recurrent events occurred within 24 hours. Of recurrent TIA/strokes, 7 (30.4%) resulted in persistent deficits.
A total of 23 patients (3% of TIA patients) received CEA in relation to their condition (none with recurrent stroke). Of the patients (N=1,537) who attended the unit, 1,487 (96.7%) received a definite diagnosis (i.e., TIA or stroke mimic). A total of 3 patients were diagnosed with a hemorrhagic stroke at the SRAU. Neurologist diagnoses were unknown for 16 patients. The median time from ED referral to unit arrival was 3.2 days (interquartile range 1.6-5.4 days).
Table 3 summarizes the number of CTA requisitions by ED physicians, the SRAU, hospital clinicians (ED +SRAU), and the CDST, stratified by patient classification. Out of the 1,679 ED referrals, hospital clinicians ordered 954 CTAs. Over the 2-year period, mean of CTA requisitions per day by ED physicians was 0.91 (range, 0-5). The total volume of CTAs ordered over the 2-year period remained relatively constant between hospital clinicians and the CDST (954 v. 977). Overall, the CDST would have ordered 23 additional CTAs over the 2-year period and increased the number of imaged-TIA patients by 89 (10.1%), while imaging 98 (16.1%) fewer stroke-mimic patients. For the 2-year period, the mean of CTA requisitions per day by the CDST would have been 1.34 (range, 0-7).
Of the 23 recurrent TIA/stroke patients admitted to the hospital before arriving at the SRAU, 10 (43.5%) received CTA during their initial ED visit. The CDST would have requisitioned CTA for 18 (78.3%) of the 23 recurrent TIA/stroke patients; 9 of the CDST requisitions overlapped with the 10 CTA orders that took place during the initial ED visit. Of the 7 recurrent patients with persistent deficits, 4 received CTA during their initial ED visit; the CDST would have requisitioned CTA for all such cases.
Of the 23 CEA patients, 7 (30.4%) received CTA during their initial ED visit, and 11 (47.8%) during their SRAU visit. The CDST would have requisitioned CTA for 22 (95.7%) of the 23 CEA patients.
Table 1 summarizes the number of CTAs for diagnosed mimic patients, stratified by the 10 most frequent stroke-mimic diagnoses. Of the stroke-mimic subdiagnoses, the CDST would have requisitioned fewer CTAs for migraine and vestibulopathy patients. These two stroke-mimic diagnoses represent the most frequently occurring stroke-mimic diagnoses encountered at the SRAU.
Restricting the sample to only those patients with a diagnosis of TIA or stroke mimic, the CDST demonstrated a sensitivity of 83% (95% CI, 80%, 85%), specificity of 48% (95% CI, 44%, 52%), and diagnostic accuracy of 69% (95% CI, 66%, 71%), when using the diagnostic threshold (≥0.516) to identify TIA patients. The DOR of the CDST at this threshold was 4.48 (95% CI, 3.54, 5.68).
When the second threshold for determining CTA requisition (≥0.662) was used, the CDST demonstrated a sensitivity of 74% (95% CI, 71%, 76%), specificity of 62% (95% CI, 58%, 66%), and diagnostic accuracy of 69% (95% CI, 66%, 71%). The DOR of the CDST at this threshold was 4.54 (95% CI, 3.64, 5.67).
Figure 1 displays the discriminant predictive performance of hospital clinicians (ED +SRAU) and the CDST for requisition of CTA imaging for patients referred from the ED and diagnosed at the SRAU as either TIA or stroke mimic (N=1,487). Hospital clinicians CTA requisitions had a sensitivity of 63% (95% CI, 60%, 67%), specificity of 46% (95% CI, 42%, 50%), and diagnostic accuracy of 56% (95% CI, 54%, 59%). The DOR of hospital clinicians was 1.47 (95% CI, 1.19, 1.82).
Discriminant performance of hospital clinicians v. CDST for TIA and stroke-mimic patients referred from the ED and diagnosed at the SRAU. ACC=diagnostic accuracy; CDST=clinical decision support tool; Clin.=hospital clinicians; Sen=sensitivity; Spe=specificity.
The CDST was found to have a significantly higher sensitivity (74% v. 63%), specificity (62% v. 46%), and diagnostic accuracy (69% v. 56%) than hospital clinicians’ baseline CTA ordering (each, p<0.001). The DOR of the CDST was significantly greater than that of hospital clinicians (4.54 v. 1.47), as indicated by non-overlapping confidence intervals.
DISCUSSION
The results of our study suggest that use of the CDST could have had a beneficial impact on CTA requisitions at the institutional level. Critically, the use of our CDST would have increased the number of TIA patients (particularly, recurrent stroke and CEA candidate patients) receiving CTA imaging before ED discharge (rather than later at TIA units; median time from referral to unit arrival 3.2 days), which is in keeping with the Canadian stroke best practice guidelines. Earlier detection and treatment of high-risk TIA patients may improve outcomes by reducing the number of recurrent strokes occurring within 24 hours.Reference Coutts, Modi and Patel 5 Use of our CDST in the ED would have increased the mean number of ED CTAs per day for suspected TIA patients by 0.42, while keeping the institutional number of CTAs ordered relatively constant. As such, our CDST has the potential to improve the effectiveness of existing CTA usage without overutilization of the resource above current levels.
We anticipate our tool being particularly helpful in support of medical decision-making, regarding transport/referral from rural/remote communities to larger centres for brain imaging. In these situations, the decision to transport/refer will be based solely on the patient’s presenting history. Focus group discussion and demonstration of the CDST in rural settings, thus far, have been strongly received and supportive; further study is planned.
A limitation of the current study is that we have no way of ascertaining the number of TIA patients missed by ED physicians and, therefore, not referred to the TIA unit. Our results, therefore, should not be interpreted as making any statement regarding the diagnostic accuracy of referring ED physicians.
An additional limitation of the current study is its retrospective design. As such, it represents a best-case scenario, assuming full adoption of the tool by front-line physicians. As well, the study is based on referral data, and it is likely that physicians report positive TIA symptoms with greater frequency than less referral-relevant mimic symptoms (e.g., confusion). This would decrease specificity of the CDST to identify stroke-mimic patients. Interactive use of the tool may change physician reporting behaviour, by increasing the clinical relevance of mimic symptoms for decision-making. A prospective evaluation of the tool will be required to determine its clinical utility and rate of adoption by ED physicians, and to determine tool performance in real-world settings.
CONCLUSION
Our CDST has the potential to increase the institutional effectiveness of CTA requisition among clinically suspected TIA/minor-stroke patients. Specifically, a greater number of TIA patients would receive CTA imaging, with that imaging occurring before ED discharge rather than later at outpatient TIA units. In this way, use of our CDST would permit healthcare institutions to provide greater patient care and financial value out of each CTA conducted for suspected TIA patients, all the while improving institutional adherence to the Canadian stroke best practices guidelines. Further prospective validation of the CDST is necessary, however, before it can be adopted into clinical practice.
Financial support: This work was supported by the Heart and Stroke Foundation (A.M.P., grant number PG-08-0415) and Genome British Columbia, Genome Alberta, and Genome Canada (A.M.P, grant number 4125-Penn).
Competing interests: None declared.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/cem.2018.449
