Setting

Ontario is Canada’s most populous province with currently 15 million residents. Residents of Ontario have access to a single-payer government healthcare system with universal access. The Ontario Ministry of Health tracks healthcare utilization using administrative data. These data thus reflect care for the full population, and they have been previously validated for secondary analysis for research and quality improvement purposes. ^{Reference Juurlink, Preyra and Croxford11,Reference Yu, Holodinsky and Zerna12} In Ontario, administrative data are housed at ICES (previously Institute for Clinical Evaluative Sciences), an independent, nonprofit health services research institute that is a Prescribed Entity under Ontario’s Personal Health Information Protection Act.

Establishing a Canadian stroke registry

A Canadian national stroke registry was first established in 2001 under the name of the Registry of the Canadian Stroke Network. It was funded by a research grant from the Canadian Stroke Network, one of Canada’s Networks of Centres of Excellence, with data collection in Ontario funded by the Ontario Ministry of Health and Long-Term Care. It involved primary data collection in consenting patients with acute stroke or transient ischemic attack (TIA) seen in the emergency department or admitted to 20 academic and comprehensive stroke centers across Canada (later expanded to 24 centers), with 6-month follow-up interviews to collect data on post-stroke functional status and quality of life (Phases 1 and 2, see Table 1). However, rural and nonacademic hospitals were excluded, and obtaining patient consent proved to be costly, impracticable and therefore prone to important selection biases.^{Reference Tu, Willison and Silver13} In 2003, the registry changed to a model where a more representative sample of hospitals was included, chart abstraction was performed with a waiver of consent, and linkages to administrative data were performed to provide information on deaths, readmissions and other aspects of health services utilization. For reasons of feasibility, data collection was then limited to the provinces of Ontario and Nova Scotia (Phases 3 and 4, see Table 1). In 2011, the Canadian Stroke Network ended, ownership of the registry was transferred to ICES and the name was changed to the Ontario Stroke Registry.

Table 1. Phases in the evolution of the Ontario Stroke Registry (OSR), originally known as the Registry of the Canadian Stroke Network

* Data collection was also performed on 46,275 patients seen at 40 stroke secondary prevention clinics in 2006–2012; details not shown here. CSC = comprehensive stroke centers; ED = emergency department; TIA = transient ischemic attack.

Patient inclusion criteria, case ascertainment and sampling

From 2001 to 2002, data were collected on all consecutive patients with ischemic stroke, subarachnoid hemorrhage, intracerebral hemorrhage or TIA seen at comprehensive stroke centers, whether hospitalized or seen in an emergency department without admission and including those with in-hospital stroke. After 2003, data collection was expanded to include a periodic audit on a simple random sample of patients seen at all (N ∼ 150) acute care hospitals in the province of Ontario (Table 1). This sampling strategy was adopted because it was not feasible to collect data on all of the estimated annual 25,000 patients with stroke or TIA. Initially, patients were identified through daily review of patient lists with prospective data collection, but subsequently, as the sampling strategy became more population-based, case lists were generated using validated International Statistical Classification of Diseases, 10th Revision, Canadian codes (ICD-10-CM) codes I63 and I64 for ischemic stroke, I61 for intracerebral hemorrhage, I60 for subarachnoid hemorrhage and G45 (excluding G45.4) for TIA. ¹⁴ Trained research coordinators confirmed the final diagnosis using retrospective chart review.

Variables

The registry collected information on over 400 data elements including baseline demographics, comorbid conditions, pre-stroke functional status, stroke presentation (including the timing of stroke symptom recognition, hospital arrival, neuroimaging and interventions), stroke type and severity, results of neuroimaging, in-hospital processes of care, complications and mortality, length of stay and discharge destination and functional status at discharge; full case report forms available online. ¹⁵ From 2001 to 2008, data were also collected on variables such as ethnicity, language fluency and social support. Data elements were selected after wide consultation with clinicians, persons with lived experience, health authorities and other stakeholders and included information needed to evaluate the quality of stroke care delivery based on best practice guidelines. Duplicate chart abstraction on a random sample of 10% of patients showed excellent inter-rater agreement (kappa > 0.8) for key variables such as age, sex, stroke type and use of thrombolysis.

Follow-up data

From 2001 to 2002, 6-month follow-up interviews were performed to obtain information on survival, functional status based on the Stroke Impact Scale-16 and quality of life based on the Health Utilities Index Mark 2/3. ^{Reference Duncan, Lai, Bode, Perera and DeRosa16,Reference Furlong, Feeny, Torrance and Barr17} From 2003 onward, follow-up interviews were no longer performed, both for feasibility and due to bias arising from cohort attrition and missing data. Instead, data were deterministically linked using encoded unique patient identifiers to population-based administrative, survey and laboratory databases for information on sociodemographic factors, medication use, healthcare encounters, deaths and other outcomes. This allowed for long-term indefinite follow-up.

Data privacy and ethics

When the registry was established in 2001, the overall project was approved by the Research Ethics Board at Sunnybrook Health Sciences Centre in Toronto, and informed consent was required from patients or proxies for participation. However, despite concerted efforts by study personnel, under 50% of eligible patients consented to participate, and those who were enrolled were substantially different from those who were not, limiting the registry’s ability to monitor the quality of stroke care delivery at participating sites. ^{Reference Tu, Willison and Silver13} In 2003, the registry was relaunched with a waiver of consent for chart abstraction, and the Canadian Stroke Network was designated as a “prescribed person” under section 39(1)(c) of Ontario’s Personal Health Information Protection Act, which permits health information custodians to disclose personal health information without consent for the purposes of facilitating or improving the provision of stroke care in the province. In 2013, with the conclusion of the Canadian Stroke Network, ownership of the registry was transferred to ICES, itself a prescribed entity. With each data collection cycle, data sharing agreements between the registry team and the individual hospitals were established.

Extension of the ontario stroke registry for SMART-EVT

Due to lack of funding, there was a hiatus in registry data collection between 2013 and 2019. However, in 2015, EVT became a standard of care treatment for patients with ischemic stroke and large vessel occlusion. ^{Reference Goyal, Menon and van Zwam18,Reference Casaubon, Boulanger and Blacquiere19} Several initiatives were established to track processes of care and outcomes among patients treated with EVT, such as the Alberta provincial QuICR registry – Quality Improvement and Clinical Research ^{Reference Kamal, Jeerakathil and Stang20} and its subsequent national extension Optimizing Patient Treatment In Major Ischemic Stroke with EVT (OPTIMISE). ^{Reference Sajobi, Singh and Almekhlafi21} However, there was almost no information on the patients who were not screened for large vessel occlusion or did not undergo EVT even when an occlusion was found. Ontario’s stroke systems of care were not optimized for the provision of EVT. Only 51% of the population living in rural Ontario regions have access to a computed tomography angiography scan within 30 minutes of driving time and 32% are within 60 minutes of an EVT-capable hospital. ^{Reference Kapral, Hall and Gozdyra22} It was imperative to resume the registry activities to evaluate whether patients with stroke were appropriately screened for large vessel occlusion and whether there was equitable access to this treatment. In 2020, the Canadian Institutes of Health Research funded the research project “SMART-EVT: Stroke Metrics for quAlity, Reporting, and Translation in the implementation of EndoVascular Thrombectomy,” which involved new data collection through the Ontario Stroke Registry (phase 5) with a focus on the evaluation of EVT delivery during the fiscal years 2019/20 and 2022/23. We anticipate that the project will provide novel and much-needed information on the frequency of screening for large vessel occlusion and treatment with EVT, the reasons for potential variations in screening and treatment and the outcomes of treatment in routine clinical practice and in important patient subgroups.

Cohort creation for SMART-EVT

For phase 5 of the registry, we again used passive surveillance with administrative data linkage to generate the case list for data collection. To avoid double-counting inter-hospital transfers as multiple events, we created unique episodes of care using ICES’ standard definition: (1) any admissions within 6 hours of the previous discharge, (2) any admissions within 12 hours of the previous discharge where discharge codes indicate transfer between two acute care hospitals and (3) any admissions within 48 hours of the previous discharge where the “institution from” and “institution to” numbers match. We excluded patients aged < 18 and > 105 years, those with invalid healthcare card numbers, admission date before January 1, 2019, elective admission without preceding emergency department visit and stroke events occurring while already hospitalized for another condition, and for patients who had more than one stroke episode during the fiscal year, we only included the first event. We retained admissions for ischemic stroke or intracerebral hemorrhage to hospitals with ≥ 70 annual admissions. Patients diagnosed with TIA or those discharged from the emergency department were not included as they would be unlikely to be EVT candidates. As shown in Figure 1, there were a total of 19,009 hospitalizations for stroke or TIA in Ontario during fiscal year 2019/20. After applying the exclusion criteria, we created a case list of 13,828 unique patients admitted with ischemic stroke or intracerebral hemorrhage across 54 hospitals. This sampling strategy included 93.3% of all eligible episodes.

Figure 1. Cohort creation flowchart for the SMART-EVT project (Ontario Stroke Registry phase 5).

Privacy impact assessment, data sharing agreements and security

Data collection was approved by the ICES Chief Privacy Officer through a Privacy Impact Assessment obviating the need for approval by individual hospital research ethics boards. Data sharing agreements between ICES and each of the 54 hospitals were obtained. ICES shared the hospital-specific case list containing patient identifiers and the date of admission of interest with the health data record manager of each hospital. In hospitals with paper chart systems, the health data record team retrieved the appropriate paper chart for trained abstractors to review. In hospitals with electronic health records, abstractors were given access to the electronic system. To maximize data collection on EVT-related variables, when patients were cared for at multiple hospitals in one episode of care, abstractors were sent to the hospital with the highest level of stroke care, that is, comprehensive stroke centers with EVT and thrombolysis capability, followed by primary stroke centers with thrombolysis capability and finally non-designated centers. Data were entered on an electronic clinical report form using an encrypted research laptop by the abstractor through REDCap (Research Electronic Data Capture), a secure web application with two-factor authentication and end-to-end encryption. Abstractors only had access to the patient identifiers of the specific hospital case list they were working on. Once data were downloaded at ICES for analysis, data were de-identified and variables with fewer than five counts were suppressed to reduce re-identification risk. Encoded patient-level unique identifiers were retained at ICES to allow for deterministic linkage to population-based administrative data.

Data collection and quality monitoring

Chart abstractors reviewed charts to collect information on important patient characteristics and stroke process measures, investigation results, treatments and outcomes at the time of discharge that would otherwise not be obtainable from administrative data (e.g., patient baseline functional status, time of last known well, stroke severity, whether neurovascular imaging was performed, whether an intracranial occlusion was present and among those with large vessel occlusion, whether EVT was performed). The full case report form is in Supplemental Table 1 and also publicly available on the ICES website. ¹⁵

We took several steps to ensure high data quality. First, a REDCap database was programmed to manage internal quality checks and to force entry for key variables to minimize missing data. Second, most abstractors already had knowledge of stroke or health data records as they held day-time jobs as stroke clinical or research coordinators or worked in hospital medical health records departments. Third, all abstractors, regardless of their baseline knowledge of stroke, underwent a 3-hour training session with the principal investigator (PI) followed by full data abstraction of 10 test charts, which were compared to the gold standard answers entered by the PI. Each abstractor received subsequent one-on-one feedback based on their answers. Abstractors were compensated for their time spent on training and testing. Abstractors could communicate with the PI by email, phone or text messages for clarification of specific challenging cases during their work, and monthly frequently asked questions were sent to all abstractors so all could learn from each other’s questions. Finally, we downloaded data at regular intervals to perform logic checks and to ensure missing data and the use of “unable to be determined” answers were kept to a minimum. Data downloads for quality checks occurred monthly initially and quarterly later.

Results and impact

Data from the registry have led to several hundred peer-reviewed manuscripts and reports to date. ¹⁵ These data have played a key role in informing clinicians, researchers, administrators and knowledge users to impact policy, such as the development of regional stroke systems of care, the implementation of thrombolysis and the expansion of stroke units. In Ontario, we showed that establishing stroke systems of care was associated with increased use of neuroimaging, thrombolysis, admission to stroke units and decline in mortality. ^{Reference Kapral, Fang and Silver23,Reference Kapral, Laupacis and Phillips24} Registry data were also used to conduct novel methodology work validating administrative data to reduce the burden of primary data collection and improve the sustainability of the registry, including deriving and validating case definitions for stroke case ascertainment, evaluating the quality of stroke data collected by the Canadian Institute for Health Information, validating the home-time outcome metric and developing the Passive Surveillance Stroke Severity (PaSSV) indicator for baseline stroke severity. ^{15,Reference Hall, Mondor, Porter, Fang and Kapral25–Reference Joundi, King and Stang30} The process of waiver of patient consent in the later phases of the registry and its positive impact on the representativeness of the registry population contributed to shaping the design of other stroke registries. ^{Reference Cadilhac, Kim and Lannin10,Reference Ingelfinger and Drazen31,Reference Reeves, Fonarow, Smith, Sheth, Messe and Schwamm32} Dedicated studies to validate EVT-related metrics on processes of care and outcomes and intracranial vascular imaging are still lacking and will be feasible with the newest data collection cycle.