Skip to main content Accessibility help
×
×
Home

Real-Time, Automated Detection of Ventilator-Associated Events: Avoiding Missed Detections, Misclassifications, and False Detections Due to Human Error

  • Erica S. Shenoy (a1) (a2) (a3) (a4) (a5), Eric S. Rosenthal (a3) (a6) (a4) (a5), Yu-Ping Shao (a6) (a4), Siddharth Biswal (a7), Manohar Ghanta (a6) (a4), Erin E. Ryan (a1), Dolores Suslak (a1), Nancy Swanson (a1), Valdery Moura Junior (a6) (a4), David C. Hooper (a1) (a2) (a3) and M. Brandon Westover (a3) (a6) (a4) (a5)...

Abstract

OBJECTIVE

To validate a system to detect ventilator associated events (VAEs) autonomously and in real time.

DESIGN

Retrospective review of ventilated patients using a secure informatics platform to identify VAEs (ie, automated surveillance) compared to surveillance by infection control (IC) staff (ie, manual surveillance), including development and validation cohorts.

SETTING

The Massachusetts General Hospital, a tertiary-care academic health center, during January–March 2015 (development cohort) and January–March 2016 (validation cohort).

PATIENTS

Ventilated patients in 4 intensive care units.

METHODS

The automated process included (1) analysis of physiologic data to detect increases in positive end-expiratory pressure (PEEP) and fraction of inspired oxygen (FiO2); (2) querying the electronic health record (EHR) for leukopenia or leukocytosis and antibiotic initiation data; and (3) retrieval and interpretation of microbiology reports. The cohorts were evaluated as follows: (1) manual surveillance by IC staff with independent chart review; (2) automated surveillance detection of ventilator-associated condition (VAC), infection-related ventilator-associated complication (IVAC), and possible VAP (PVAP); (3) senior IC staff adjudicated manual surveillance–automated surveillance discordance. Outcomes included sensitivity, specificity, positive predictive value (PPV), and manual surveillance detection errors. Errors detected during the development cohort resulted in algorithm updates applied to the validation cohort.

RESULTS

In the development cohort, there were 1,325 admissions, 479 ventilated patients, 2,539 ventilator days, and 47 VAEs. In the validation cohort, there were 1,234 admissions, 431 ventilated patients, 2,604 ventilator days, and 56 VAEs. With manual surveillance, in the development cohort, sensitivity was 40%, specificity was 98%, and PPV was 70%. In the validation cohort, sensitivity was 71%, specificity was 98%, and PPV was 87%. With automated surveillance, in the development cohort, sensitivity was 100%, specificity was 100%, and PPV was 100%. In the validation cohort, sensitivity was 85%, specificity was 99%, and PPV was 100%. Manual surveillance detection errors included missed detections, misclassifications, and false detections.

CONCLUSIONS

Manual surveillance is vulnerable to human error. Automated surveillance is more accurate and more efficient for VAE surveillance.

Infect Control Hosp Epidemiol 2018;826–833

Copyright

Corresponding author

Address correspondence to Erica S. Shenoy, MD, PhD, Massachusetts General Hospital, 55 Fruit Street, Bulfinch 334, Boston, MA 02114 (eshenoy@mgh.harvard.edu).

Footnotes

Hide All

PREVIOUS PRESENTATION. This work was presented at ID Week 2017 (abstract no. 2151) on October 7, 2017, in San Diego, California.

a

Senior authors with equal contribution.

Footnotes

References

Hide All
1. Magill, SS, Klompas, M, Balk, R, et al. Developing a new, national approach to surveillance for ventilator-associated events: executive summary. Clin Infect Dis 2013;57:17421746.
2. Klompas, M. Ventilator-associated events surveillance: a patient safety opportunity. Curr Opin Crit Care 2013;19:424431.
3. Klompas, M, Magill, S, Robicsek, A, et al. Objective surveillance definitions for ventilator-associated pneumonia. Crit Care Med 2012;40:31543161.
4. Mann, T, Ellsworth, J, Huda, N, et al. Building and validating a computerized algorithm for surveillance of ventilator-associated events. Infect Control Hosp Epidemiol 2015;36:9991003.
5. Klein Klouwenberg, PM, van Mourik, MS, Ong, DS, et al. Electronic implementation of a novel surveillance paradigm for ventilator-associated events. Feasibility and validation. Am J Respir Crit Care Med 2014;189:947955.
6. Stevens, JP, Silva, G, Gillis, J, et al. Automated surveillance for ventilator-associated events. Chest 2014;146:16121618.
7. Nuckchady, D, Heckman, MG, Diehl, NN, et al. Assessment of an automated surveillance system for detection of initial ventilator-associated events. Am J Infect Control 2015;43:11191121.
8. National Healthcare Safety Network. Ventilator-associated events (VAE). Centers for Disease Control and Prevention website. https://www.cdc.gov/nhsn/pdfs/pscmanual/10-vae_final.pdf. Published 2017. Accessed April 9, 2018.
9. Ventilator-associated event calculator (version 4.0). Materials for enrolled facilities. Centers for Disease Control and Prevention website. https://www.cdc.gov/nhsn/vae-calculator/index.html. Published August 4, 2016. Accessed October 10, 2017.
10. Halpin, H, Shortell, SM, Milstein, A, Vanneman, M. Hospital adoption of automated surveillance technology and the implementation of infection prevention and control programs. Am J Infect Control 2011;39:270276.
11. Gastmeier, P, Behnke, M. Electronic surveillance and using administrative data to identify healthcare associated infections. Curr Opin Infect Dis 2016;29:394399.
12. Puhto, T, Syrjala, H. Incidence of healthcare-associated infections in a tertiary care hospital: results from a three-year period of electronic surveillance. J Hosp Infect 2015;90:4651.
13. de Bruin, JS, Seeling, W, Schuh, C. Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review. J Am Med Inform Assoc 2014;21:942951.
14. Grota, PG, Stone, PW, Jordan, S, Pogorzelska, M, Larson, E. Electronic surveillance systems in infection prevention: organizational support, program characteristics, and user satisfaction. Am J Infect Control 2010;38:509514.
15. Furuno, JP, Schweizer, ML, McGregor, JC, Perencevich, EN. Economics of infection control surveillance technology: cost-effective or just cost? Am J Infect Control 2008;36(3 Suppl):S12S17.
16. Sips, ME, Bonten, MJM, van Mourik, MSM. Automated surveillance of healthcare-associated infections: state of the art. Curr Opin Infect Dis 2017;30:425431.
17. Klompas, M, Yokoe, DS. Automated surveillance of health care-associated infections. Clin Infect Dis 2009;48:12681275.
18. Resetar, E, McMullen, KM, Russo, AJ, Doherty, JA, Gase, KA, Woeltje, KF. Development, implementation and use of electronic surveillance for ventilator-associated events (VAE) in adults. AMIA Annu Symp Proc 2014:10101017.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Infection Control & Hospital Epidemiology
  • ISSN: 0899-823X
  • EISSN: 1559-6834
  • URL: /core/journals/infection-control-and-hospital-epidemiology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed