In the decades since the inception of critical care, advances in medical technology and in the understanding of pathogenesis and pathophysiology of critically ill patients have transpired, leading to unavoidable increase in the costs of providing medical care. In Canada, use of intensive care units (ICU) is increasing faster than acute care hospitalizations (1). The daily cost of each ICU hospitalization is up to three times the cost of a general hospital ward, due to the increased resource use in personnel, equipment, and medications (1). Patients on mechanical ventilation (MV), a defining, life-sustaining therapy in critically ill patients, accrue significantly greater costs than nonventilated patients (Reference Dasta, McLaughlin, Mody and Piech2). The incremental cost of MV was estimated at $1,522 USD per patient day shown by a 2005 US study (Reference Dasta, McLaughlin, Mody and Piech2). The already extraordinary costs of ICU care requiring MV are expected to increase in the next few years by 2.3 percent annually, outpacing population growth (Reference Needham, Bronskill and Calinawan3).
Across Canada, there are 286 hospitals with intensive care units capable of providing invasive MV, totalling 3,170 beds designed for this purpose (Reference Fowler, Abdelmalik and Wood4). From 2013–14, 33 percent of Canadian ICU patients received MV, up from 28 percent in 2007–08 which is in line with rates of 19–39 percent reported internationally (1). In addition to the equipment and personnel costs related to ventilation, prolonged ventilation is also associated with increased length of stay (LOS), and complications including greater risk of nosocomial infection, pneumothorax, ventilator-associated pneumonia, and mortality (Reference Vincent5). Thus, hastening the weaning and liberation (i.e., extubation) from MV are critically important.
Extubation remains a high stakes clinical decision, as extubation failure and reintubation has detrimental effects on patient outcomes and cost (6;7). Extubation failure is typically defined as the need for reintubation within 48–72 hour (Reference Cheng, Cheng, Chen, Hsing and Sung8) although it is occasionally defined up to 1 week (Reference Thille, Richard and Brochard9). Despite advances in critical care, failed extubation occurs at a rate of approximately 10–20 percent (6;9–11). Previous studies across Europe, Asia and the United States on extubation failure found significant increases in morbidity (6–8;12–14), LOS (6;7;13;15;16), ICU stay (6–8;12–14;16), and costs (6;8;15;16). Complications include the need for tracheostomy, transfer to long-term care or development of infection. There is also an associated increase in mortality (6–8;12;15;16) with rates of 25–50 percent (9), independent of underlying disease severity (17;18).
The Extubation Advisor (EA) was designed as a novel extubation decision support tool to assist in the assessment of extubation decision-making and prediction of extubation failure risk, with the aim of reducing the rate of failure. EA utilized a predictive model based on respiratory rate variability (RRV) (19). The EA thus has the potential to reduce adverse outcomes and costs by optimizing the timing and enabling both earlier and safer extubation. Although several studies have examined the economics of MV in the ICU, the literature is sparse on the effectiveness and cost-effectiveness of the EA tool. This early economic analysis is therefore conducted to estimate the minimum reduction in failed extubation needed to make the EA tool economically attractive. This information can help developers refine the tool to meet the needs of health care systems.
Methods
We performed an early return on investment (ROI) analysis to estimate the minimum reduction in extubation failure rates that an EA decision tool needed to achieve to make the tool cost-effective from the hospital’s perspective. We used a Markov model (Figure 1) to combine evidence regarding the existing standard of care, that is, without the implementation of the EA tool, and compare it to a hypothetical care pathway that incorporates the tool. This information can inform the tool’s value for money in its intended context and the associated risk, which helps guide further research and development (20). We estimated the cost of EA tool as the sum of maintenance costs, upfront training and material costs. We defined the effectiveness of the EA tool as the reduction in failed extubation in ICU patients as extubation failure has been shown to be associated with morbidity and mortality (6–8;12–14), prolonged hospital and ICU LOS (6;7;13;15;16), and costs (6;8;15;16) in ICU patients.
Setting and Population
Our analysis focused on intubated patients in two ICUs at the Ottawa Hospital (TOH) in Ottawa, ON, Canada, from November 2015 to November 2019. TOH is one of Canada’s largest teaching hospitals, with 1,271 hospital beds and 56,029 patient admissions annually.
Data Sources
Risk of Failed Extubations
We determined whether an extubated patient was or was not successfully extubated from the ICU database from TOH Data Warehouse, a relational database containing the operational information of each of TOH’s campuses. Failed extubation was defined as the patient being reintubated within 48 hour of being extubated. All patient data was anonymized prior to extraction for analysis. Our study was based on 434 patients who had complete baseline and hospital cost data; of these, eighty-two patients had daily records indicating intubation status from which we could assess whether or not their extubation was successful.
Hospital Costs
In this study, hospital costs for each inpatient encounter were identified within the case-costing system of TOH Data Warehouse, a standardized case-costing methodology developed by the Ontario Case Costing Initiative (21) based on the Canadian Institute for Health Information Management Information Systems guidelines (22). The case-costing system links financial, clinical, and patient activity information stored within the Data Warehouse to define intermediate products, such as nursing time, medications, and laboratory tests. The data collected by TOH is broken down by day, to allow us to observe the time to given events based on fee codes we identified as being related to extubation and intubation. The total hospital costs were equal to the sum of the direct and indirect hospital costs for each intermediate product used during an encounter for each patient.
Costs of the EA Tool
The annual institutional cost of the EA tool was calculated as the sum of annual personnel costs and annualized costs of infrastructure. Personnel costs included costs of staff time for coordination, technical support, and opportunity costs of attending training and entering patient data into the EA tool. Infrastructure costs included annual licensing costs for the EA, costs of a virtual server and server storage. Costs were estimated by multiplying the type and frequency of resource use with their unit costs. Data on resource use, that is, staff time on various activities, requirements for virtual server and storage was provided by the project staff. Information on annual licensing costs was provided by the tool developer. Staff salaries were extracted from administrative documents. Additionally, cost per patient for implementing the EA application was estimated by dividing the annual institutional cost by the average number of ventilated patients in a year at the hospital, which was obtained from the TOH data warehouse. All costs were presented in 2020 Canadian Dollars (C$).
Analysis
Based on TOH data, we developed a Markov model (Figure 1) that followed patients over time, where we could observe transitions by intubation status, discharge status, and mortality according to daily transition probabilities. The model was run as Monte Carlo simulations using 5,000 iterations, in which all parameters were varied according to their variance and distribution derived from the data set. Table 1 presents the model input parameters used for the decision model. We reported both the average trajectories as well as 95 percent confidence intervals (CI) based on the observed variance in outcomes in the ICU population over the 5-year retrospective data period. We estimated the annual institutional ROI for the EA tool based on the following formula:
$$ \mathrm{ROI}=\frac{\begin{array}{c}\left(\left|\mathrm{Annual}\ \mathrm{hospital}\ \mathrm{cost}\ \mathrm{savings}\ \mathrm{attributable}\ \mathrm{to}\ \mathrm{the}\ \mathrm{EA}\ \mathrm{to}\mathrm{ol}\right|\right)-\\ {}\left(\mathrm{Annual}\ \mathrm{cost}\ \mathrm{of}\ \mathrm{the}\hskip0.5em \mathrm{EA}\hskip0.5em \mathrm{tool}\right)\end{array}}{\left(\mathrm{Annual}\ \mathrm{cost}\ \mathrm{of}\hskip0.5em \mathrm{EA}\hskip0.5em \mathrm{tool}\right)}. $$
Schematic diagram for a decision analytical (Markov) model.
Model Input Parameters
ICU, intensive care unit.
If ROI is greater than 0, the EA tool is deemed to offer good ROI. We used the simulation model to estimate the minimum percent change in failed extubation is necessary to make the ROI greater than 0. Annual hospital cost savings attributable to the EA tool was estimated from hospital costs associated with failed extubations derived from a generalized linear model with an identity distribution and a log-link function. The model adjusted for patient age, sex, and Charlson comorbidity index.
We performed a series of scenario analysis to assess the robustness of our study findings. In the first scenario, we replaced the total hospital costs with direct variable hospital costs, which include items, such as fluids and medications that could be saved if the hospital beds are not used. Another scenario used alternative assumptions for the EA’s licensing fees. We estimated the annual institutional cost of EA and its ROI by varying the EA’s licensing costs between C$2,500 (US$1,965) and C$12,500 (US$9,823) per bed per year.
Results
A descriptive analysis revealed that the average LOS in the ICU for an intubated patient was 10.70 days (95 percent CI: 8.9–12.4), during which time they costed the hospital, on average, a total of C$54,647 (US$42,943) (95 percent CI: 46,171, 63,122; US$36,282, US$49,603). We also found that 13 percent of intubated ICU patients had a failed extubation. Results from the regression analysis showed that failed extubation patients were associated with double the LOS (an additional 8.98 days, 95 percent CI: 6.15, 11.80) and similarly scaled added total cost C$43,464 (US$34,155) (95 percent CI: 28,242, 58,686; US$22,193, US$46,117). Of the total cost for patients with a failed extubation, direct variable costs accounted for 70.44 percent, followed by indirect variable costs (13.52 percent). Fixed costs accounted for 16.04 percent of total costs. Failed extubation patients were also nearly twice as likely to die in hospital (5.30 percent absolute value increase) even after adjusting for age, sex, and comorbidity index.
Figure 2 shows patient transitions under the standard of care derived from our decision model, starting from the day on which patients are intubated and admitted in the ICU. This model parameterizes the risk of failed extubation as a daily hazard function, where 13 percent of patients will fail their extubation. Parameterizing failure risk allowed us to change this single parameter to consider alternate scenarios in which we could set the new average failure rate to any value and see how it would change the average patient outcomes, all else being equal.
Patient transitions between health states under the standard of care of patients from the index intubation.
Our model results match the actual data, estimating the average length of ICU stay for an intubated patient at 10.4 (95 percent CI: 9.7, 11.0) days and a total cost of C$60,037 (US$47,178) (95 percent CI: 47,911, 75,197; US$37,649, US$59,091). The model estimated that, on average, a 1 percent reduction in failed extubation rates could save the hospital C$289 (US$227) per intubated patient (95 percent CI 197, 459) (Table 2). At TOH, the hospital sees approximately 2,500 intubated ICU patients per year, it is therefore estimated that the hospital would save C$723,124 (US$568,246) per year per percent reduction in failed extubations.
Base Case Results
EA, Extubation Advisor; ROI, return on investment.
The annual institutional cost for the EA application was C$164,221 (US$129,048) or C$65.69 (US$51.62) per patient (Supplementary Table 1); of these, 57 percent (C$93,488; US$73,465) accounted for the infrastructure cost and 43 percent (C$70,734; US$55,584) was spent on personnel cost. Even at the lowest cost level (C$2,500/bed/year; US$1,965/bed/year), the software licensing cost was found to be the largest cost driver, accounting for 55 percent of the total costs of EA tool, as training and set up costs are low. Supplementary Table 1 presents a breakdown of the mean annual institutional cost for EA by resource category.
Our model estimated that at the current annual price of C$164,221 (US$129,048), the EA tool must reduce the failed extubation by at least 0.24 percent (95 percent CI: .14, .41) to make the tool a good investment from TOH’s perspective, that is, savings due to the EA tool is greater than its cost. If we only considered hospital direct variable costs, the EA tool must reduce the extubation failure rate by at least 0.34 percent (95 percent CI: .12, .39) to be cost-effective (Table 3, Scenario Analysis A). In addition, the scenario analysis that varied the annual licensing costs for EA indicated that the minimum threshold for the effectiveness of EA increased with higher licensing costs (Table 3, Scenario Analysis B). Varying the costs associated with the EA tool implementation of the base case values had minimal impact on the minimum percentage change required to make the EA tool economically attractive.
Results from Scenario Analyses for Licensing Costs
a The minimum reduction of extubation failure rate above which the savings due to the EA tool is greater than its cost.
EA, Extubation Advisor.
Discussion
Our study demonstrates that in a Canadian tertiary care hospital network, the attributable cost of a single failed extubation at C$43,464 (US$34,155) is significant, nearly doubling the average cost of hospitalization compared to a successfully extubated patient. In comparison, reports from the United States have shown costs of similarly large magnitude. Seymour et al. (Reference Seymour, Martinez, Christie and Fuchs6) showed added costs of failed extubation totalling US$33,926 in a sixteen ICU bed community hospital, while Menon et al. (Reference Menon, Joffe and Deem16) work in an eighty-eight ICU bed Level 1 Trauma Center had even higher incremental hospital charges totalling US$126,400 (17). Pronovost et al. (Reference Pronovost, Garrett and Dorman15) showed that reintubation increased total hospital costs by approximately 20 percent or US$9,510 (1996 dollars) in abdominal aortic surgery patients. Although no comparable Canadian data was found, these values underscore the significant potential of clinical decision-making tools such as the EA tool for reducing hospital costs if the tool is able to reduce extubation failure rates. This economic benefit is expected to be larger if the EA tool’s impact on ICU patients and their families, such as time missed from work, was considered.
This research was conducted at a large tertiary-care hospital in Canada, and the results showed that the breakeven effectiveness required from EA is a 0.24 percent absolute reduction in extubation failure assuming a cost of C$2,500/bed/year (US$1,965/bed/year). The average extubation failure rate at our institution is 13 percent, which was within the ranges of 10–20 percent reported in others (6;9–11), highlighting how a miniscule absolute reduction of 0.24 percent (proportionally < 5 percent) will cover the costs of C$165,000 (US$129,660) on an annual basis. The breakeven effectiveness was 0.34 percent when a smaller fraction of hospital (direct variable) costs were considered. The breakeven effectiveness required from EA ranged between 0.24 and 0.76 percent absolute reduction in failure rates when licensing costs for EA were varied between $2,500 (US$1,965) and C$12,500 (US$9,823) per bed per year. This indicates that even at the highest cost of C$12,500 (US$9,823) per bed, positive ROI is achieved if extubation failure is reduced by at least 0.76 percent.
Based on the analysis of the EA and the WAVE study (Reference Seely, Bravi and Herry19), the EA tool was safe, and it is likely that the potential for the EA to reduce the rate of extubation failure would exceed the threshold required for cost-effectiveness, although this remains to be proven in a future planned interventional trial. The WAVE study concluded that a predictive model using RRV during the last spontaneous breathing trial (SBT) provided optimal accuracy of prediction in all patients, with improved accuracy when combined with clinical impression or rapid shallow breathing index (RSBI), factors which were integrated into the EA tool.
Although the values reported in this paper pertain to the context of similar sized tertiary care Canadian ICUs, our study findings can be scaled according to the size of the facility, with larger or busier centers expected to break even at a lower percent reduction in extubation failure and benefit from an even greater effect on cost reduction. In addition, the opportunity cost of data entry, currently estimated at 10 min per SBT, is expected to decrease with time and experience as respiratory therapists integrate the EA’s standardized SBT assessment into their regular workflow.
Limitations
The main limitation of this study was the limited ICU data. Of 434 patients for which we had complete time-to-event information, only 82 patients could be used to assess failed extubation and associated mortality rates. The hospital data was well described when patients needed to be reintubated the day (or more) after their extubation, but both ICU data input and shift data did not adequately reflect changes within a single day. Data indicating an extubation or reintubation attempts were not coded. Since most failed extubations and subsequent reintubation happen within 15–22 hour (16;23;24), this meant we were likely missing a significant amount of data. Consequently, we linked the TOH with the WAVE trial data to determine failed extubation events (19). Due to data limitation, we could not perform a time to event analysis and estimate how mortality rates might vary by days from intubation. We therefore assumed a constant daily mortality rate in the model. Because this assumption was applied to the EA and usual care scenarios, it was unlikely to affect the estimated break-even-point of the EA tool. Last, it is clear that the impact of an extubation clinical decision support tool on clinician decision making may be variable between clinicians and centers. While the EA tool incorporates measures obtained during the SBT that are important to the prediction of extubation outcomes such as respiratory rate (RR), tidal volume (TV), RSBI (=RR/TV) (25), RRV (19;26;27), and cough strength, among others (25;28;29), techniques used to conduct and assess SBTs vary between institutions (30;31). The variable impact of clinical decision support tools on decision making remain to be studied. Regardless, this study serves to identify minimal thresholds of reduction in overall extubation failure rates to make a tool cost effective.
Conclusion
With rising healthcare costs, there is an increasing need for clinical decision support tools to inform care in an evidence-based and cost-effective fashion. Our early economic modeling suggests that the EA is one such tool, which has high economic feasibility as it could reduce the hospital cost of MV by providing earlier extubation and reduce the rate of extubation failure and reintubation events.
Abbreviations
- EA
-
Extubation Advisor
- MV
-
mechanical ventilation
- ROI
-
return on investment
- RRV
-
respiratory rate variability
- RSBI
-
rapid shallow breathing index
- SBT
-
spontaneous breathing trial
- TOH
-
The Ottawa Hospital
Supplementary Material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462322000472.
Conflicts of Interest
A.J.E.S. is Founder and Board Chair of Therapeutic Monitoring Systems (TMS). TMS is developing and commercializing waveform-based variability-derived clinical decision support tools (including Extubation Advisor) in order to improve care for patients at risk for or with existing critical illness. Both A.J.E.S. and C.L.H. are coinventors on patents related to physiological waveform assessment and variability analysis. All remaining authors have disclosed that they do not have any potential conflicts of interest.
Funding Statement
This work was supported by a peer-reviewed grant from The Ottawa Hospital Academic Medical Organization (TOHAMO) Innovation Fund 2016–17, awarded March 2017.
Author Contributions
A.J.E.S. and K.T. conceived the study. K.Z. performed a review of the literature. K.Z., S.K., and K.T. collected and analyzed the data and wrote the initial draft of the manuscript. S.K. and K.T. performed the economic analysis. All authors contributed to reading, revising, and approving the final manuscript.
Data Availability Statement
The data sets generated during and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Ethical Standards
This study was approved by the Ottawa Health Science Network Research Ethics Board.
Open access
