Hostname: page-component-76fb5796d-9pm4c Total loading time: 0 Render date: 2024-04-25T09:55:08.365Z Has data issue: false hasContentIssue false
  • English
  • Français

Electronic surveillance criteria for non–ventilator-associated hospital-acquired pneumonia: Assessment of reliability and validity

Published online by Cambridge University Press:  15 March 2023

Sarah E. Stern Open the ORCID record for Sarah E. Stern [Opens in a new window] ,
Matthew A. Christensen Open the ORCID record for Matthew A. Christensen [Opens in a new window] ,
McKenna R. Nevers ,
Jian Ying ,
Caroline McKenna ,
Shannon Munro ,
Chanu Rhee Open the ORCID record for Chanu Rhee [Opens in a new window] ,
Matthew H. Samore ,
Michael Klompas Open the ORCID record for Michael Klompas [Opens in a new window]  and
Barbara E. Jones
Sarah E. Stern*
Affiliation:
Division of Pulmonary & Critical Care Medicine, University of Utah, Salt Lake City, Utah
Matthew A. Christensen
Affiliation:
Division of Allergy, Pulmonary, & Critical Care Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
McKenna R. Nevers
Affiliation:
Division of Epidemiology, University of Utah, Salt Lake City, Utah
Jian Ying
Affiliation:
Division of Epidemiology, University of Utah, Salt Lake City, Utah
Caroline McKenna
Affiliation:
Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts
Shannon Munro
Affiliation:
Department of Veterans’ Affairs Medical Center, Salem, Virginia
Chanu Rhee
Affiliation:
Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts Department of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
Matthew H. Samore
Affiliation:
Division of Epidemiology, University of Utah, Salt Lake City, Utah VA Salt Lake City Health Care System, Salt Lake City, Utah
Michael Klompas
Affiliation:
Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts Brigham and Women’s Hospital, Boston, Massachusetts
Barbara E. Jones
Affiliation:
Division of Pulmonary & Critical Care Medicine, University of Utah, Salt Lake City, Utah VA Salt Lake City Health Care System, Salt Lake City, Utah
*
Author for correspondence: Sarah Stern, MD, Division of Pulmonary & Critical Care Medicine, University of Utah, 701 Wintrobe, 26 North 1900 East, Salt Lake City, Utah 84132. E-mail: sarah.stern16@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

Surveillance of non–ventilator-associated hospital-acquired pneumonia (NV-HAP) is complicated by subjectivity and variability in diagnosing pneumonia. We compared a fully automatable surveillance definition using routine electronic health record data to manual determinations of NV-HAP according to surveillance criteria and clinical diagnoses.

Methods:

We retrospectively applied an electronic surveillance definition for NV-HAP to all adults admitted to Veterans’ Affairs (VA) hospitals from January 1, 2015, to November 30, 2020. We randomly selected 250 hospitalizations meeting NV-HAP surveillance criteria for independent review by 2 clinicians and calculated the percent of hospitalizations with (1) clinical deterioration, (2) CDC National Healthcare Safety Network (CDC-NHSN) criteria, (3) NV-HAP according to a reviewer, (4) NV-HAP according to a treating clinician, (5) pneumonia diagnosis in discharge summary; and (6) discharge diagnosis codes for HAP. We assessed interrater reliability by calculating simple agreement and the Cohen κ (kappa).

Results:

Among 3.1 million hospitalizations, 14,023 met NV-HAP electronic surveillance criteria. Among reviewed cases, 98% had a confirmed clinical deterioration; 67% met CDC-NHSN criteria; 71% had NV-HAP according to a reviewer; 60% had NV-HAP according to a treating clinician; 49% had a discharge summary diagnosis of pneumonia; and 82% had NV-HAP according to any definition according to at least 1 reviewer. Only 8% had diagnosis codes for HAP. Interrater agreement was 75% (κ = 0.50) for CDC-NHSN criteria and 78% (κ = 0.55) for reviewer diagnosis of NV-HAP.

Conclusions:

Electronic NV-HAP surveillance criteria correlated moderately with existing manual surveillance criteria. Reviewer variability for all manual assessments was high. Electronic surveillance using clinical data may therefore allow for more consistent and efficient surveillance with similar accuracy compared to manual assessments or diagnosis codes.

Type
Original Article
Information
Infection Control & Hospital Epidemiology , Volume 44 , Issue 11 , November 2023 , pp. 1769 - 1775
Check for updates
Creative Commons
To the extent this is a work of the US Government, it is not subject to copyright protection within the United States. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America.
Copyright
© Veterans Health Administration and the Author(s), 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

PREVIOUS PRESENTATION: Components of this study were previously presented as an oral abstract presentation at the Infectious Diseases Society of America (IDSA) annual meeting IDWeek on September 29–October 3, 2021, held virtually. Parts of this study were also presented as a poster presentation at the American Thoracic 2021 International Conference on May 14–19, 2021, in San Diego, California.

References

Giuliano, KK, Baker, D, Quinn, B. The epidemiology of nonventilator hospital-acquired pneumonia in the United States. Am J Infect Control 2018;46:322–327.10.1016/j.ajic.2017.09.005CrossRefGoogle Scholar
Baker, D, Quinn, B. Hospital-Acquired Pneumonia Prevention Initiative-2: incidence of nonventilator hospital-acquired pneumonia in the United States. Am J Infect Control 2018;46:2–7.10.1016/j.ajic.2017.08.036CrossRefGoogle ScholarPubMed
Micek, ST, Chew, B, Hampton, N, Kollef, MH. A case–control study assessing the impact of nonventilated hospital-acquired pneumonia on patient outcomes. Chest 2016;150:1008–1014.10.1016/j.chest.2016.04.009CrossRefGoogle ScholarPubMed
Horan, TC, Andrus, M, Dudeck, MA. CDC/NHSN surveillance definition of healthcare-associated infection and criteria for specific types of infections in the acute-care setting. Am J Infect Control 2008;36:309–332.10.1016/j.ajic.2008.03.002CrossRefGoogle ScholarPubMed
van Mourik, MS, van Duijn, PJ, Moons, KG, Bonten, MJ, Lee, GM. Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review. BMJ Open 2015;5:e008424.10.1136/bmjopen-2015-008424CrossRefGoogle ScholarPubMed
Ji, W, McKenna, C, Ochoa, A, et al. Development and assessment of objective surveillance definitions for nonventilator hospital-acquired pneumonia. JAMA Netw Open 2019;2:e1913674.10.1001/jamanetworkopen.2019.13674CrossRefGoogle ScholarPubMed
Ramirez Batlle, H, Klompas, M, Program, CDCPE. Accuracy and reliability of electronic versus CDC surveillance criteria for non-ventilator hospital-acquired pneumonia. Infect Control Hosp Epidemiol 2020;41:219–221.Google Scholar
Klompas, M. Interobserver variability in ventilator-associated pneumonia surveillance. Am J Infect Control 2010;38:237–239.10.1016/j.ajic.2009.10.003CrossRefGoogle ScholarPubMed
Tejerina, E, Esteban, A, Fernandez-Segoviano, P, et al. Accuracy of clinical definitions of ventilator-associated pneumonia: comparison with autopsy findings. J Crit Care 2010;25:62–68.10.1016/j.jcrc.2009.05.008CrossRefGoogle ScholarPubMed
Kerlin, MP, Trick, WE, Anderson, DJ, et al. Interrater reliability of surveillance for ventilator-associated events and pneumonia. Infect Control Hosp Epidemiol 2017;38:172–178.10.1017/ice.2016.262CrossRefGoogle ScholarPubMed
VHA facility quality and safety report-fiscal year 2012 data. Veterans’ Health Administration website. https://www.va.gov/HEALTH/docs/VHA_Quality_and_Safety_Report_2013.pdf. Published 2013. Accessed December 18, 2021.Google Scholar
Charlson, ME, Pompei, P, Ales, KL, MacKenzie, CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373–383.10.1016/0021-9681(87)90171-8CrossRefGoogle ScholarPubMed
VA Informatics and Computing Infrastructure (VINCI). US Department of Vterans’ Affairs website. https://www.hsrd.research.va.gov/for_researchers/vinci/. Accessed November 16, 2021.Google Scholar
Carey, E, Blankenhorn, R, Chen, P, Munro, S. Non–ventilator-associated hospital-acquired pneumonia incidence and health outcomes among US veterans from 2016–2020. Am J Infect Control 2022;50:116–119.10.1016/j.ajic.2021.06.001CrossRefGoogle ScholarPubMed
National Healthcare Safety Network (NHSN) patient safety component manual. US Centers for Disease Control and Prevention website. https://www.cdc.gov/nhsn/pdfs/pscmanual/pcsmanual_current.pdf. Updated 2023. Accessed February 6, 2023.Google Scholar
Byrt, T, Bishop, J, Bias, Carlin JB., prevalence and kappa. J Clin Epidemiol 1993;46:423–429.10.1016/0895-4356(93)90018-VCrossRefGoogle ScholarPubMed
Conway, JR, Lex, A, Gehlenborg, N. UpSetR: an R package for the visualization of intersecting sets and their properties. Bioinformatics 2017;33:2938–2940.10.1093/bioinformatics/btx364CrossRefGoogle Scholar
RStudio: integrated development environment for R. R Studio website. https://www.rstudio.com/categories/integrated-development-environment/. Published 2021. Accessed February 6, 2022.Google Scholar
See, I, Chang, J, Gualandi, N, et al. Clinical correlates of surveillance events detected by National Healthcare Safety Network pneumonia and lower respiratory infection definitions—Pennsylvania, 2011–2012. Infect Control Hosp Epidemiol 2016;37:818–824.10.1017/ice.2016.74CrossRefGoogle Scholar
Wolfensberger, A, Meier, AH, Kuster, SP, Mehra, T, Meier, MT, Sax, H. Should International Classification of Diseases codes be used to survey hospital-acquired pneumonia? J Hosp Infect 2018;99:81–84.10.1016/j.jhin.2018.01.017CrossRefGoogle ScholarPubMed
Wolfensberger, A, Jakob, W, Faes Hesse, M, et al. Development and validation of a semi-automated surveillance system-lowering the fruit for non–ventilator-associated hospital-acquired pneumonia (nvHAP) prevention. Clin Microbiol Infect 2019;25:1428.10.1016/j.cmi.2019.03.019CrossRefGoogle ScholarPubMed
Naidech, AM, Liebling, SM, Duran, IM, Moore, MJ, Wunderink, RG, Zembower, TR. Reliability of the validated clinical diagnosis of pneumonia on validated outcomes after intracranial hemorrhage. J Crit Care 2012;27:527.10.1016/j.jcrc.2011.11.009CrossRefGoogle ScholarPubMed
Melbye, H, Dale, K. Interobserver variability in the radiographic diagnosis of adult outpatient pneumonia. Acta Radiol 1992;33:79–81.Google ScholarPubMed
Vassar, M, Holzmann, M. The retrospective chart review: important methodological considerations. J Educ Eval Health Prof 2013;10:12.10.3352/jeehp.2013.10.12CrossRefGoogle ScholarPubMed
Schreiber, M, Krauss, D, Blake, B, Boone, E, Almonte, R. Balancing value and burden: the Centers for Medicare & Medicaid Services electronic Clinical Quality Measure (eCQM) strategy project. J Am Med Inform Assoc 2021;28:2475–2482.10.1093/jamia/ocab013CrossRefGoogle ScholarPubMed
Jones, BE, Haroldsen, C, Madaras-Kelly, K, et al. In data we trust? Comparison of electronic versus manual abstraction of antimicrobial prescribing quality metrics for hospitalized veterans with pneumonia. Med Care 2018;56:626–633.10.1097/MLR.0000000000000916CrossRefGoogle ScholarPubMed
Magill, SS, O’Leary, E, Janelle, SJ, et al. Changes in prevalence of healthcare-associated infections in US hospitals. N Engl J Med 2018;379:1732–1744.10.1056/NEJMoa1801550CrossRefGoogle Scholar
Kazaure, HS, Martin, M, Yoon, JK, Wren, SM. Long-term results of a postoperative pneumonia prevention program for the inpatient surgical ward. JAMA Surg 2014;149:914–918.10.1001/jamasurg.2014.1216CrossRefGoogle ScholarPubMed
Munro, SC, Baker, D, Giuliano, KK, et al. Nonventilator hospital-acquired pneumonia: a call to action. Infect Control Hosp Epidemiol 2021;42:991–996.10.1017/ice.2021.239CrossRefGoogle ScholarPubMed
Wren, SM, Martin, M, Yoon, JK, Bech, F. Postoperative pneumonia—prevention program for the inpatient surgical ward. J Am Coll Surg 2010;210:491–495.10.1016/j.jamcollsurg.2010.01.009CrossRefGoogle ScholarPubMed
Munro, S, Baker, D. Reducing missed oral-care opportunities to prevent non–ventilator-associated hospital-acquired pneumonia at the Department of Veterans’ Affairs. Appl Nurs Res 2018;44:48–53.10.1016/j.apnr.2018.09.004CrossRefGoogle ScholarPubMed
Lacerna, CC, Patey, D, Block, L, et al. A successful program preventing nonventilator hospital-acquired pneumonia in a large hospital system. Infect Control Hosp Epidemiol 2020;41:547–552.10.1017/ice.2019.368CrossRefGoogle Scholar
Supplementary material: File

Stern et al. supplementary material

Appendix

Download Stern et al. supplementary material(File)
File 128.7 KB