Introduction
Hospital-acquired pneumonia (HAP) is the most common healthcare-associated infection and is associated with high morbidity, mortality, and health care costs. Reference Magill, O’Leary and Janelle1 Non-ventilator HAP (NV-HAP) accounts for the majority of HAP, Reference Giuliano, Baker and Quinn2 but prevention efforts have historically focused on ventilator-associated pneumonia. NV-HAP is thought to be attributable to aspiration of oral microorganisms into the lungs, although the factors governing pathogenesis are poorly understood. Oral care, consisting of toothbrushing and denture cleaning, is a component of basic hospital care that may reduce NV-HAP risk and is listed as a recommended strategy for prevention. Reference Giuliano, Penoyer, Middleton and Baker3,Reference Klompas, Branson and Cawcutt4 However, there are several barriers. Reference Li, Yao and Yang5–Reference Lewis, Harvey, Hogan and Kitson7
After showing promise in reducing NV-HAP diagnoses in pilot sites and committing to the importance of oral care, the Veterans Affairs (VA) Administration implemented an oral care improvement initiative on a rolling basis starting in 2016. Reference Munro, Phillips, Hasselbeck, Lucatorto, Hehr and Ochylski8,Reference Munro and Baker9 By July 2021, the initiative had been successfully adopted at every VA Medical Center in the nation as standard of care for all hospitalized patients and long-term residents and was recognized by the 2020 Gears of Government President’s Award for Innovation. Reference Marketplace10
Establishing the effectiveness of this initiative to prevent pneumonia, however, has been challenging, in part due to the lack of a reliable measure of NV-HAP. Reference Munro, Baker and Giuliano11 Surveillance definitions of NV-HAP include many subjective and ambiguous criteria that are difficult to apply consistently. 12 Diagnosis codes have demonstrated poor sensitivity, specificity, and coding variability. Reference Wolfensberger, Meier, Kuster, Mehra, Meier and Sax13 Emerging evidence suggests that an electronic definition for NV-HAP based on changes in clinical parameters (oxygenation, temperature, white blood cell count, orders for chest imaging, and antibiotic administrations) can identify NV-HAP with higher accuracy than diagnosis codes and similar accuracy to traditional surveillance methods with less burden. Reference Stern, Christensen and Nevers14–Reference Ji, McKenna and Ochoa16 In this study, we used both the electronic surveillance definition and diagnostic coding of NV-HAP to assess the impact of the oral care initiative on NV-HAP.
Methods
Settings and participants
The VA is the largest integrated healthcare system in the United States caring for 9 million Veterans nationwide. 17 Clincial data were accessed through the Veterans Informatics and Computing Infrastructure (VINCI), a computing environment that stores clinical data for research purposes. 18 We included only facilities that began implementing the oral care initiative at medical-surgical units at least one year before the COVID-19 pandemic; 17 facilities met these criteria. We extracted data for the period starting one year before the first facility began implementation through one year following the last facility’s implementation date, corresponding to January 10, 2015, through December 31, 2019 (Figure 1). The study was approved by VA and the University of Utah Institutional Review Boards.
Study population.
Measurements
Implementation of an oral care initiative
The VA’s oral care initiative aimed to standardize twice daily oral care for all patients hospitalized on medical-surgical and long-term care units through a well-planned, nurse-initiated implementation that followed evidence-based implementation principles informed by research. Reference Stevenson, Munro, Klocko and Sayre19 Full details have previously been described. 20 In brief, the implementation contained 6 steps: (1) Prepare foundation (identify clinical champions, determine implementation approach, develop charter that outlines implementation, and engage stakeholders); (2) Obtain and organize supplies, including toothbrushes, toothpaste, and denture cleansing supplies; (3) Standardize templates, tools, and patient education materials; (4) Customize and conduct nursing staff trainings; (5) Implement; and (6) Monitor. Reference Klompas, Branson and Cawcutt4 We identified and verified the dates that each facility started and completed rolling out the initiative on medical-surgical wards.
Patient factors
We extracted patient demographics, comorbidities, vital signs, supplementary oxygen devices, laboratory tests, and discharge diagnosis codes. We defined and summarized comorbid conditions (Table 1) using the Charlson method applied to International Classification of Disease (ICD)-10-CM 21 codes via software developed by the Agency for Healthcare Research and Quality. Reference Davies, Geppert, McClellan, McDonald, Romano and Shojania22,Reference Charlson, Pompei, Ales and MacKenzie23
Patient characteristics, risk of NV-HAP, and outcomes among (1) all patients hospitalized to 17 hospitals during January 1, 2015—December 31, 2019, and (2) patients hospitalized 1 year before the hospital start of oral care initiative, and (3) patients hospitalized 1 year after the start of oral care initiative
a Included in adjusted analysis model.
b Median (IQR); n (%).
Laboratory values and vital signs within 2 calendar days prior to the hospitalization and on the first day of hospitalization were used and classified as “normal” or “abnormal” based on pre-defined cut-offs indicated in Table 1. Missing laboratories and vital signs were imputed as normal. We identified admitting specialty service and ward type (medical/surgical ward or intensive care unit). We also extracted facility characteristics. Thirty-day mortality was assessed using a vital status file maintained by the VA that includes data on deaths occurring both inside and outside of VA facilities. Reference Arnold24
Electronic clinical-data NV-HAP definition
The electronic clinical data definition was designed to identify non-ventilated patients with new respiratory deterioration (≥2 days of decreased oxygen saturation or increase in supplemental oxygen after ≥2 days of stable or improving oxygenation) and concurrent fever or leukocytosis, performance of chest imaging, and the administration of new antibiotics for at least 3 days. Reference Stern, Christensen and Nevers14,Reference Ji, McKenna and Ochoa16 Complete details and SAS code describing the definition are publicly available at https://github.com/caramckenna/NVHAP/tree/main/.
Diagnostic coding-based NV-HAP definition
We extracted a diagnostic coding-based NV-HAP definition previously used by the VA quality improvement initiative to track changes in NV-HAP risk. Reference Carey, Blankenhorn, Chen and Munro25 NV-HAP was defined as the presence of a primary or secondary discharge diagnosis code for pneumonia not present on admission (B95.3, B96.0, J13, J15.X, J16.X, J17.X, J18.X, J84.111, J84.116, J84.117, J84.2, J85.1, and J85.2).
Statistical analysis
Among the entire population of hospitalizations at the 17 facilities during the study period, we calculated the percentage of hospitalizations with NV-HAP using both the electronic surveillance criteria and the diagnostic-coding-based criteria. For each hospitalization, only the first electronic surveillance event was counted.
Temporal trends in NV-HAP, 2015–2019
We examined trends in each of the following outcomes:
-
-
a) The electronic clinical data criteria;
-
b) The diagnostic-coding based definition;
-
We fit separate models for each outcome assuming a linear trend in days since the beginning of the study (10/01/2015). For NV-HAP, we compared the two definitions using a logistic regression model that included main effects for each definition, as well as an interaction term between definition and time; the interaction term was used to assess for difference in trends between the two definitions. The model was fit using generalized estimating equations (GEEs) to account for clustering by hospitalization. Logistic regression was used to model mortality, and a Hurdle model (described below) Reference Mullahy26 was used for length-of-stay. To visualize trends over time, we plotted the estimated percentages for NV-HAP and mortality and means for length-of-stay along with the observed percentages/means by quarter.
Effect of oral care implementation on NV-HAP (1-year pre- versus 1-year post-implementation)
Unadjusted analysis
For each site, we calculated crude differences in the following outcomes 1-year post-implementation versus 1-year pre-implementation: the percentage of hospitalizations with ≥1 NV-HAP event according to the electronic surveillance definition and according to the diagnostic coding definition; the percentage of hospitalizations that culminated in death within 30 days of admission; and mean length-of-stay.
Adjusted analysis
For each definition, we sought to compare, on a risk difference scale, the counterfactual risk of hospitalization with an NV-HAP event for the year post-implementation if, contrary to fact, there had been no implementation, to the actual risk of hospitalization with an NV-HAP event for the year post-implementation. Our approach accounts for site-level variability in risk differences and the possibility that the intervention could have a direct effect on preventing NV-HAP, and/or an indirect effect on NV-HAP through changing length of stay.
We estimated the counterfactual risk as follows. Using data one year before implementation from the 17 sites, we fit models for (1) the conditional distribution of length-of-stay given site, calendar year and week of hospitalization, patient characteristics, and health status variables available through day of admission and (2) the conditional probability of NV-HAP given length-of-stay and the same factors included in the length-of-stay model. Specific patient variables used are listed in Table 1. A hurdle model Reference Mullahy26 with a negative binomial count component was specified for (1), and a logistic regression model was specified for (2). The estimated fits of these models were used to compute, for each patient admitted to one of the 17 sites during the post-implementation period and every possible level of length-of-stay (l = 1,2, …), (a) the counterfactual probability of length-of-stay level l and (b) the counterfactual probability of NV-HAP given length-of-stay level l. All these quantities were combined, using the law of total probability, to estimate, for each patient, the counterfactual probability of NV-HAP. These estimated probabilities were then averaged over patients at each site to create site-specific counterfactual estimated risks. For each site, we computed the actual risk of NV-HAP events per hospitalization during the post-implementation period. Each site’s risk difference was then estimated as the difference between the site-specific estimated counterfactual and actual risks.
Pooled analyses
For both the unadjusted and adjusted analyses, the variance-covariance matrix of the site-specific risk differences was estimated using the delta method. Reference Casella and Berger27 A pooled estimate of site-specific effects was computed using meta-analysis with inverse-variance weighting of site-specific estimates. Computation of the standard error of the pooled estimate accounted for the correlation of the site-specific estimates.
Secondary analyses
We used the same approach to estimate the effect of implementation on 30-day mortality and hospital length-of-stay using logistic regression and hurdle models, respectively. We also conducted subgroup analyses restricted to patients admitted only to non-ICU wards (the primary target of implementation), with implementation dates shifted to 3 months prior to the listed start date (to account for anticipatory effects) as well as shifted to the listed completion dates of the implementation (to account for delayed effects). All statistical analyses were performed using RStudio (version 1.4; RStudio, PBC, Boston, MA, 2021).
Results
Descriptive characteristics
There were 333,257 patient hospitalizations during the study period. Of these, 156,120 occurred within 1 year before or after their facility’s implementation start (Figure 1). Median age was 69 (interquartile range 61, 75), 95% were male, and 16% were admitted to intensive care units. Median time to completing roll-out of the implementation at all medical/surgical wards was 8 months (intra-quartile range: 6–12 months). Patient characteristics and illness severity (Table 1) were similar between pre- and post-implementation periods as well the national VA population. Reference Stern, Christensen and Nevers14
Temporal trends in NV-HAP, mortality, and length of stay (2015–2019)
Among 333,257 hospitalizations, 1,922 (0.58%) met NV-HAP clinical surveillance criteria, and 2,386 (0.72%) met diagnostic coding criteria. Trends in NV-HAP across the study period are shown in Figure 2. Using the electronic surveillance definition, the percentage of hospitalizations with NV-HAP was 0.62% in October 2015 and 0.54% in Dec 2019 (estimated difference –0.084% [95% CI: –0.17%, 0.0056%]; estimated log odds change per year –0.034% [95% CI: –0.071%, 0.0023%]). Using the diagnostic coding definition, NV-HAP decreased from 1.0% to 0.48%. (estimated difference –0.53% [95% CI: –0.63%, –0.43%]; estimated log odds change per year –0.18, [95% CI: –0.21, –0.15]). The difference in the estimated log-odds change per year measured by electronic surveillance coding versus diagnostic coding was 0.14 (95% CI: 0.097, 0.19).
30-day mortality among all hospitalizations was 4.9%, which fluctuated by season and year without an appreciable trend over time (estimated probability of 30-day mortality 5.02% in 2015 to 4.95% in 2019 (estimated difference –0.066% [95% CI: –0.32%, 0.19%] (Figure 3A). The overall estimated mean length-of-stay was 6.1 days. This fluctuated by season with a net increase across the study period from 5.98 days to 6.21 days (estimated difference 0.23 days [95% CI: 0.16, 0.30]) (Figure 3B).
Trends in NV-HAP risk by electronic surveillance versus claims-based definition.
Trends in 30-day mortality and hospital length of stay.
Effect of oral care implementation on NV-HAP (1-year pre- vs post-implementation)
Among the 156,120 hospitalizations in the pre-post analysis, 77,300 occurred within the year pre-implementation and 78,296 in the year post-implementation. Changes in NV-HAP, mortality and length-of-stay before and after each facility’s implementation dates are shown in eFigures 2–4. Using the electronic clinical surveillance definition, 427 (0.55%) had NV-HAP in the year before implementation versus 437 (0.56%) in the year after implementation. Using the diagnostic coding definition, 580 (0.75%) had NV-HAP in the year before implementation versus 520 (0.66%) in the year after.
Figure 4 (electronic surveillance) and Figure 5 (diagnostic coding) present the unadjusted (left panels) and adjusted (right panels) site-specific and overall estimated risk differences and confidence intervals. The overall adjusted risk differences were –0.078% (95% CI: –0.25%, 0.091%) and –0.021% (95% CI: –0.18%, 0.14%) for the electronic surveillance and diagnostic coding criteria, respectively. Thus, there was no evidence of an effect of the oral care initiative on NV-HAP using either definition. There was also no detectable effect on 30-day mortality (Figure 6), with an adjusted risk difference of 0.20% (95% CI: –0.49%, 0.89%). The adjusted mean difference in length-of-stay (eFigure 1) was –0.13 days (95% CI: –0.24 days, –0.02 days).
Effect of oral care implementation on NV-HAP risk defined by electronic clinical criteria. Panel A. Unadjusted analysis: differences in percentage of NV-HAP events 1 year post versus 1 year prior to implementation among 17 facilities undergoing oral care implementation. Panel B. Adjusted analysis: differences in observed percentage of NV-HAP events post implementation versus predicted probability after accounting for patient characteristics.
Effect of oral care implementation on NV-HAP defined by diagnostic coding. Panel A. Unadjusted analysis: differences in proportion of NV-HAP diagnoses 1 year post versus 1 year prior to implementation among 17 facilities undergoing oral care implementation. Panel B. Adjusted analysis: Differences in observed proportion of NV-HAP diagnoses post-implementation versus predicted probability after accounting for patient characteristics.
Effect of oral care implementation on 30-day mortality. Panel A. Unadjusted analysis: differences in proportion of NV-HAP events 1 year post versus 1 year prior to implementation among 17 facilities undergoing oral care implementation. Panel B. Adjusted analysis: Differences in observed proportion of 30-day mortality events post implementation versus predicted probability after accounting for patient characteristics.
The results were similar for all secondary analyses (Supplement eFigures 5–10). For the subgroup of patients admitted to non-ICU wards, NV-HAP rates did not change appreciably using either the electronic surveillance or diagnostic coding criteria. When the implementation date was shifted 3 months earlier than the formal start date or to 3 months after the completion date, there was no detectable effect on NV-HAP or mortality, but a small effect on length-of-stay.
Discussion
In a large population of patients hospitalized at 17 VA hospitals that implemented an oral care initiative on at least one medical surgical unit between 2015 and 2019, there was a temporal decrease in NV-HAP defined by diagnostic coding but not when using electronic clinical criteria. There was no detectable difference in NV-HAP rates in the year before vs after implementation using either diagnostic code criteria or electronic clinical criteria, no detectable effect on 30-day mortality, and a small effect on length-of-stay.
The temporal decrease in NV-HAP defined by diagnostic coding is consistent with a previous system-wide evaluation of the VA that found a 32% decrease in NV-HAP diagnoses between 2015 and 2020. Reference Carey, Blankenhorn, Chen and Munro25 However, we did not observe such a dramatic trend using electronic clinical criteria for NV-HAP. Since a large CDC point-prevalence study Reference Magill, Edwards and Bamberg28 documented the high burden of HAP between 2011 and 2015, multiple efforts to measure and reduce NV-HAP were launched, including an updated guideline for prevention. Reference Klompas, Branson and Cawcutt4 Other studies have reported inconsistent trends during this period: Baker et al reported a temporal increase in NV-HAP-associated costs for Medicaid beneficiaries using diagnosis codes between 2015 and 2019, Reference Giuliano, Baker and Thakkar-Samtani29 and a study using the Medicare Patient Monitoring System reported a decrease in post-operative pneumonia, which could suggest that efforts to prevent NV-HAP may be having an effect. Reference Metersky, Wang, Klompas, Eckenrode, Mathew and Krumholz30 Our results were limited to 17 facilities; so, a larger analysis may show a clearer system-wide improvement across the entire VA. The difference in trends using diagnosis codes versus electronic clinical criteria may reflect the subjectivity and variable usage of codes by different hospitals. Reference Wolfensberger, Meier, Kuster, Mehra, Meier and Sax13,Reference van Mourik, van Duijn, Moons, Bonten and Lee31 However, the initiative did not appear to have a detectable effect on NV-HAP diagnoses at the facility level, which suggests that implementation of the initiative did not directly result in any local coding shifts.
Our failure to identify an effect of oral care implementation on NV-HAP builds on existing evidence on oral care implementation, which has been inconsistent. One cluster randomized trial reported reduction in NV-HAP with an oral care protocol, although the difference was significant for surgical but not medical patients. Three prior pre-post implementation studies reported decreases in NV-HAP diagnoses with oral care initiatives in acute settings, Reference Giuliano, Penoyer, Middleton and Baker3,Reference Warren, Medei, Wood and Schutte32–Reference Munro, Haile-Mariam, Greenwell, Demirci, Farooqi and Vasudeva35 and one non-randomized comparative study did not. Reference McNally, Krisciunas, Langmore, Crimlisk, Pisegna and Massaro36 Two clinical trials of nursing home patients demonstrated negative results. Reference Juthani-Mehta, Van Ness and McGloin37,Reference Zimmerman, Sloane and Ward38
There are several potential explanations for our findings. First, improvements in NV-HAP from the initiative could have evaded our measurement strategies and observational study design. While we evaluated the initiative using two definitions of NV-HAP, both are imperfect measures of rare events, and pneumonia remains difficult to accurately assess due to its lack of gold standard criteria. However, the lack of substantial reductions in mortality or length-of-stay—two objective measures that should be impacted by reductions in NV-HAP—make it less likely that we missed a large positive finding due to measurement error. Our retrospective analysis of observational data allows for the possibility of residual confounding, despite the breadth of variables included in our adjusted analyses.
Second, the initiative as we captured it may not have been effective at increasing oral care, either due to low fidelity of the initiative, complexity of the implementation, or high baseline adherence to oral care in the studied hospitals. We were unable to directly assess oral care provided in study hospitals, making it difficult to differentiate these possibilities. Oral care implementation was targeted in a rolling fashion to all medical-surgical units, while our primary analysis evaluated all patients hospitalized in all units within each facility (most of which were medical-surgical units) which could have missed isolated decreases in NV-HAP within target units. Our primary analysis was also based upon a discrete start date per hospital, while implementations were phased, with a median time to completing roll-out of 8 months; so, the effect of the initiative may have lagged the evaluation period for some hospitals. However, secondary analyses that evaluated only non-ICU patients and that shifted implementation dates to account for anticipatory or delayed effects of phased implementation yielded similar results. The oral care initiative was also conducted in parallel with national efforts to raise awareness of NV-HAP and the importance of oral care throughout the VA system. Reference Carey, Blankenhorn, Chen and Munro25 It is thus possible that, VA’s national commitment to NV-HAP prevention in 2016 catalyzed nurses to improve oral care prior to the initiative. This may explain the system-level temporal trends but lack of facility-level effect.
Third, enhanced oral care alone may not be sufficient to prevent NV-HAP. The pathogenesis of pneumonia is complex, with multiple contributing factors including the burden of oral flora, the frequency and quantity of aspiration, host airway clearance capacity, and host immunity. Targeting multiple processes simultaneously including patient mobility, dysphagia management, respiratory therapy, minimizing sedation, and delirium prevention may be necessary to prevent this condition. Reference Klompas, Branson and Cawcutt4,Reference Munro, Baker and Giuliano11 A recent clinical trial by Wolfensberger et al demonstrated a correlation between effective implementation of such an intervention and a reduction of NV-HAP, highlighting the importance of multimodal strategies. Reference Wolfensberger, Clack and von Felten39
Our results do not reduce the importance of oral care to patient comfort, oral health, and hospital quality. Its benefits extend well beyond pneumonia prevention alone. Maintaining oral health is a fundamental component of physical and personal well-being and impacts nutrition, communication, comfort, and quality of life, Reference Petersen and Ogawa40 yet poor oral health is common among hospitalized patients. Reference Shiraishi, Wakabayashi and Yoshimura41 Promoting oral health during hospitalizations not only reduces hospital complications but also promotes oral health upon discharge. Reference Antonacci, Ahmed, Lennox, Rigby and Coronini-Cronberg42 Despite its importance, oral care is difficult to prioritize without standardization and support. Reference van Noort, Witteman, den Hertog-Voortman, Everaars, Vermeulen and Huisman-de Waal43 Oral care initiatives such as the one implemented across the VA system are thus essential components of high-quality hospital care in addition to pneumonia prevention efforts.
Our analysis adds to existing evidence regarding the complexity of NV-HAP development, prevention, and measurement that challenge the evaluation of strategies for prevention. Our study suggests that initiatives targeting oral care alone may not be sufficient to reduce NV-HAP rates. These results stress the need to evaluate multimodal, complex interventions, and use objective clinical measures to advance our understanding of NV-HAP pathogenesis and prevention. Large studies that evaluate the impact of multiple prevention strategies on multiple outcomes—including objective definitions of NV-HAP as well as length of stay, mortality, microbiologic findings, and key patient-centered outcomes—are needed to advance understanding of the pathogenesis of NV-HAP and identify which strategies prevent it.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2025.54
Acknowledgments
This work was supported by an interagency agreement from the U.S. Centers for Disease Control and Prevention [IAA # 24FED2400006CKD]. B.J. is supported by the VA HSR&D Informatics, Decision-Enhancement, and Analytic Sciences (IDEAS) Center of Innovation [CIN 13-414]. M.K. reports royalties from UpToDate, Inc. (authoring chapters on pneumonia) and grants from CDC and AHRQ. All remaining authors report no conflicts of interest relevant to this article.
Disclaimer
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the Department of Veterans Affairs.
Open access
