Treated, hospital-onset Clostridiodes difficile infection: An evaluation of predictors and feasibility of benchmarking comparing 2 risk-adjusted models among 265 hospitals

Objectives: To evaluate the incidence of a candidate definition of healthcare facility-onset, treated Clostridioides difficile (CD) infection (cHT-CDI) and to identify variables and best model fit of a risk-adjusted cHT-CDI metric using extractable electronic heath data. Methods: We analyzed 9,134,276 admissions from 265 hospitals during 2015–2020. The cHT-CDI events were defined based on the first positive laboratory final identification of CD after day 3 of hospitalization, accompanied by use of a CD drug. The generalized linear model method via negative binomial regression was used to identify predictors. Standardized infection ratios (SIRs) were calculated based on 2 risk-adjusted models: a simple model using descriptive variables and a complex model using descriptive variables and CD testing practices. The performance of each model was compared against cHT-CDI unadjusted rates. Results: The median rate of cHT-CDI events per 100 admissions was 0.134 (interquartile range, 0.023–0.243). Hospital variables associated with cHT-CDI included the following: higher community-onset CDI (CO-CDI) prevalence; highest-quartile length of stay; bed size; percentage of male patients; teaching hospitals; increased CD testing intensity; and CD testing prevalence. The complex model demonstrated better model performance and identified the most influential predictors: hospital-onset testing intensity and prevalence, CO-CDI rate, and community-onset testing intensity (negative correlation). Moreover, 78% of the hospitals ranked in the highest quartile based on raw rate shifted to lower percentiles when we applied the SIR from the complex model. Conclusions: Hospital descriptors, aggregate patient characteristics, CO-CDI burden, and clinical testing practices significantly influence incidence of cHT-CDI. Benchmarking a cHT-CDI metric is feasible and should include facility and clinical variables.

laboratory identification of CD accompanied by administration of an anti-CD antimicrobial, which served as a proxy for clinical interpretation of infection.We then used this candidate definition to evaluate cHT-CDI incidence and to identify hospital-level variables using statistical models to help assess a standardized cHT-CDI risk-adjusted metric.Standardized infection ratios (SIRs), defined as the ratio of observed number of events (cHT-CDI) divided by the predicted (expected) number of events, were calculated based on a simple model regression analysis (using EHR-extractable risk factors as variables) and a complex model (which additionally included CD testing practice variables), an approach that has been used successfully for hospital-onset bacteremia. 13We compared hospital rankings based on the unadjusted rate of cHT-CDI and on the simple and complex model-based SIRs.

Study design and population
5][16] The study used a limited retrospective data set for an epidemiology study and thus was approved and exempted from consent by the New England Institutional Review Board/Human Subjects Research Committee (Wellesley, MA).The study was conducted in compliance with Health Insurance Portability and Accountability Act requirements.

Outcomes and definitions
The date of admission (day the patient's status changed to inpatient) was considered day 1.A cHT-CDI event was defined when the following 2 requirements were met: (1) a final report by the facility laboratory of CD positive (ie, positive result of a single test for facilities relying on single tests or positive confirmation test in facilities with multitest rubrics) within the HO period (ie, day 4 or greater of hospitalization) and ( 2) receipt of at least 1 anti-CD antimicrobial (ie, oral vancomycin, fidaxomicin, or oral metronidazole) with a first qualified antimicrobial day (QAD) in the window period extending 2 calendar days before and 2 calendar days after the positive CD specimen-collection date.This QAD definition is similar to the one used in the CDC Hospital Toolkit for Adult Sepsis Surveillance. 17Community-onset (CO) CDI (CO-CDI) was defined as a first positive CD test within the first 3 days of hospital admission.Patients with a positive CD test in our database within 4 weeks prior to study admission were excluded.Subsequent duplicate tests from a CD-positive patient during the study were also excluded.
Testing intensity was defined as the number of total stool specimens tested for CD in the CO or HO periods divided by the number of total aggregate admissions with any CD test performed.Conceptually, testing intensity reflects the cumulative CD-tested stool samples collected among admissions with any CD test performed.Testing prevalence was defined as the number of admissions with any CD test performed in the period (CO or HO) divided by the total number of aggregate admissions and conceptually reflects the overall proportion of admissions with CD testing.

Statistical analysis
We approached the statistical analysis in 3 steps: (1) identify candidate variables that influence cHT-CDI rates using bivariate analysis; (2) construct simple and complex models to predict cHT-CDI using risk-adjusted SIRs derived from regression models and assess best model fit; and (3) compare hospital rankings using the observed (unadjusted) cHT-CDI rates versus rankings based on SIRs derived from the 2 models.
Step 1.Rates of cHT-CDI were calculated as the number of cHT-CDI events per 100 admissions for quarterly aggregated data.A bivariate analysis using generalized linear models was performed to explore the correlation between cHT-CDI rate and each candidate risk factor (Table 1).The following clinical metrics were measured: CO-CDI prevalence (rate of CO-CDI events per 100 admissions); percentage of intensive care unit (ICU) admissions (per all admissions); average length of hospital stay (LOS) among hospitalized patients (days of hospitalization per admission); CD test prevalence and intensity (as previously defined).We also collected the following patient demographics: number of females per 100 admissions; percentages of patients in each age group (ie, 18-40, 41-64, 65-80, and >80 years).In addition, facility characteristics included bed size, medical school affiliation, urban or rural status.
Step 2. To identify cHT-CDI-associated variables, we used negative binomial regression methods with natural logarithm link function to account for overdispersion and right-skewed data.We conducted 2 modeling analyses: a simple model and a complex model.The simple model used hospital-level variables easily accrued from EHRs and/or already reportable to the NHSN.Candidate variables considered in the simple model included hospital-level characteristics and patient demographics.Our complex model included variables considered in the simple model (but partitioned into quartiles) plus clinical practices of overall CD testing and divided into CO or HO testing intensity and prevalence.
The best models were selected using model fit statistics (ie, Akaike information criteria and Bayesian information criteria) based on the full study cohort data (3,455 quarters of aggregated hospital data associated with 9,134,276 admissions).In addition, we used cross-validation methods in variable selection and confirmed that the full-data model and validation model included the set of variables identical to the best models.Residual (prediction error) diagnosis of the best (complex) model was performed using decile plot, observed versus predicted cHT-CDI events plot, and standardized deviance residuals versus predicted events (Supplementary Figs.S1 and S2 online).
Step 3. We compared hospital rankings based on the observed (unadjusted) cHT-CDI rate and the ranking based on the SIRs from the simple and complex models.The Goodman and Kruskal γ statistic, Spearman correlation, and confidence intervals to measure ordinal association were calculated.We calculated 1-year SIR data (2019) as an example for comparison rankings.Finally, we compared the ranking of hospitals in the top 25th percentile (ie, highest rates) based on unadjusted cHT-CDI rates to their subsequent ranking using SIRs based on the 2 models.
Analyses were conducted using the Statistical Analysis System version 9.4 software (SAS Institute, Cary, NC).

Results
The study included 9,134,276 patient admissions (October 2015-March 2020), which were associated with 17,545 cHT-CDI events

Summary statistics of cHT-CDI prevalence and bivariate analysis results
The cHT-CDI data exhibited a right-skewed distribution.Over the study period, the median rate of cHT-CDI events per 100 admissions was 0.134 (interquartile range, 0.023-0.243)and the mean rate was 0.166 per 100 admissions (SD, 0.18).We observed evidence of decreases in cHT-CDI event rates during 2015-2022 on bivariate analysis (Table 1).All candidate risk factors considered in the analysis were significantly associated with cHT-CDI, except for 2 age groups (65-80 and >80 years) (Table 1).

Variables associated with cHT-CDI: Simple model
Using the simple model, we identified the following as significant variables: higher CO-CDI prevalence; longer average LOS; higher rate of ICU admissions; facilities with larger bed size; urban hospitals; and higher percentage of patients aged 41-64 years.In addition, we observed some changes in the cHT-CDI rate over time (Table 2 and Supplementary Table 1 online).

Variables associated with cHT-CDI: Complex model
Based on the complex model, the following hospital-level variables were significantly associated with higher cHT-CDI event rates: higher CO-CDI prevalence; fourth quartile of average LOS; larger bed size; increased HO testing intensity, increased HO testing prevalence, and teaching hospitals.Increased CO testing intensity and percentage female patients were negatively associated with cHT-CDI event rates (Table 3 and Supplementary Table 2 online).Among identified variables, HO testing intensity, HO testing prevalence, and CO-CDI prevelance yielded higher incidence rate ratios (IRRs).Comparisons of observed versus predicted events showed that the model was well specified (Supplementary Figs.S1 and S2 online).

Comparison of hospital rankings
We demonstrated a potential real-world application of risk adjustment by quantifying changes in rank of the top 25th percentile (ie, worst-performing) hospitals based on observed cHT-CDI rates compared with complex model-based cHT-CDI SIR ranking.Among the 50 hospitals ranked in the fourth quartile based on unadjusted (observed) cHT-CDI rates, 8 (16%) remained at the same percentile level when using complex model cHT-CDI SIR-based ranking, 39 (78%) improved in percentile ranking, and 3 (6%) shifted to lower (worse-performing) percentiles (Fig. 1).Supplementary Figure S3 shows changes in rank by quartile: 19 (38%) stayed in the same ranking category (fourth quartile) with the complex model-adjusted SIR, 18 (42%) improved to the third quartile; 8 (16%) improved to the second quartile, and 2 (4%) improved to the first quartile.Statistical comparison of rankings showed that the unadjusted cHT-CDI rate had a strong ordinal association with the simple model SIR (γ statistic, 0.85; 95% CI, 0.79-0.91).The strength of association between the unadjusted cHT-CDI event rate and complex-model SIR rankings was meaningfully lower (γ statistic, 0.63; 95% CI, 0.52-0.73),thereby demonstrating differential adjustment afforded by the complex model.

Discussion
Our analysis demonstrates a feasible approach that addresses 2 issues of concern with current HO-CDI reporting: (1) using a more specific and clinically oriented CDI definition based on the combination of a positive CD laboratory test after day 3 of admission plus the use of an anti-CD therapeutic agent and (2) incorporating the impact of testing practices in SIR-adjusted rankings.Our complex-model SIR used current facility-and patient-level risk-adjustment variables, such as bed size, age, sex, and LOS, as well as new testing-practice variables: HO testing intensity, HO testing prevalence, and CO testing intensity (negative correlation).These variables of testing practice have a large influence on cHT-CDI rates, demonstrating their importance to risk adjustment.CO-CDI prevalence was also a strong predictor of cHT-CDI, as evidenced by the ordinal increase of higher IRRs.As described previously, 13 the simple model was "nested" within the complex model to demonstrate the additional model fit and risk adjustment afforded by SIRs derived by the latter.0][21] Ultimately, our complex model  SIR incorporated a more robust risk adjustment.In total, 39 of 50 hospitals (78%) in the worst-performing quartile moved to lower (better-performing) percentiles, and 31 (62%) hospitals shifted to lower quartiles after complex-model SIR adjustment, including 2 that shifted from the worst-to the best-performing quartile.
The widespread use of PCR in CDI diagnostic testing has raised concerns, particularly the possibility of false-positives due to the high sensitivity and lower specificity of PCR when used alone for CDI diagnosis. 5The latest CDI testing and treatment guidelines have attempted to clarify best practices; 1 however, the resources needed to implement multiple testing tiers or other "reflex" diagnostic rubrics leave strict laboratory test-based CDI definitions at risk for highly variable sensitivity and specificity.In this study, we attempted to address this clinical concern with 2 novel additions to a cHT-CDI metric: (1) linking a laboratory CD test with use of an anti-CD therapeutic agent within a clinically reasonable time frame (the QAD), which may allow the metric to be more reflective of clinically significant CDI and (2) incorporating measures of CD testing practices directly into the complexmodel SIR.
We also showed that including measures of CD testing intensity and prevalence in the risk adjustment model may help mitigate concerns about variability 22 in testing practices within and among facilities.By including HO testing intensity and prevalence, the complex-model risk adjustment may help improve SIR estimation by allowing the resultant expected rate to flex and adjust to changes in testing practices, whether clinically warranted or not.With that in context, the complex model identified HO testing intensity and prevalence as 2 of the most influential variables in cHT-CDI prediction.This makes clinical sense because repeated testing on the same suspected cHT-CDI case, or an increased percentage of patients who receive at least 1 HO-CDI diagnostic test, will have a higher likelihood of yielding a positive cHT-CDI case.This was not the case during the CO period (first 3 days of admission), when CO testing intensity (more testing on a CDI suspect admission) had a significant negative association with cHT-CDI.CO-CDI was also highly correlated with cHT-CDI, suggesting that although community CDI burden does affect cHT-CDI risk, increased testing of random patients within the CO period (testing prevalence) does not.Thus, increasing mass "surveillance" CD testing to mitigate cHT-CDI designations may be unlikely to have an overall net benefit on patient outcomes.Indeed, the resources needed to sustain such testing, if not clinically warranted, are significant and go against the tenets of antimicrobial and diagnostic stewardship. 2Therefore, including testing practices in the risk-adjusted model should mitigate attempts to influence SIRs by nonclinical CD testing practices and thus promote best practices.
The finding that higher facility CO-CDI prevalence is correlated with cHT-CDI reinforces central infection prevention tenets of contact precautions and indicates that spore contamination is a significant driver of cHT-CDI, 23,24 especially for communities with a high CDI burden.These communities likely include more complex patient case mixes, as suggested by the fact that teaching hospital status, LOS, and a higher number of beds also remained significant in the complex model.A recent CMS ruling now requires both infection prevention and control (IPC) programs and antimicrobial stewardship programs (ASPs) as conditions for CMS participation, 25 and both are important for CDI prevention.Investigators have documented the association between antibiotic use and CDI 26,27 and have estimated that a 30% decrease in the hospital use of antibiotics linked to CDI could reduce 25% of hospital-associated CDI cases. 26Another study reported that an increase of 100 DOT per 1,000 days for high-risk CD agents correlates with a 12% increase in cHT-CDI. 28The NHSN Antimicrobial Use and Resistance module 29 benchmarks antimicrobial use including agents associated with CDI.Furthermore, it is anticipated that a revised cHT-CDI surveillance definition that incorporates anti-CD treatment will also facilitate surveillance for anti-CD treatment (whether CD testing was done or not) using the same data sources.It may become feasible to monitor correlations between the use of antibiotics considered to place patients at high risk for CDI and anti-CD treatment administered without a paired CD test to understand the use of CDI prophylaxis or attempts to avoid registering a cHT-CDI event.Therefore, this analysis not only builds toward a more clinically viable cHT-CDI definition with more robust risk-adjustment but may also provide insights into the relationship between high-risk CDI antimicrobials and therapeutic CDI antimicrobials for both ASP and IPC programs.
This study had several limitations.We did not evaluate individual CDI tests but rather the result of the final CDI test reported by the laboratory because collated EHR data feeds do not always provide visibility as to the sequence of CDI confirmatory tests.Clinical chart review and validation of the cHT-CDI definition were outside the scope of this project.Validation of any new metric against a reference standard of CDI from chart review will need to be evaluated.However, the inclusion of anti-CD agents should improve clinical specificity of a CDI case beyond a mere laboratory designation.Notably, oral metronidazole is used for multiple clinical infections and is not specific to CDI.However, the pairing of an antibiotic requirement with CD testing within 2 days before or after stool collection, and the removal of oral metronidazole as a first line cHT-CDI therapy in more recent CDI guidelines, 1,30 may minimize this limitation over time.Finally, increasing use of oral vancomycin and fidaxomicin as prophylaxis in high-risk CD-colonized patients 31 may obscure specificity of cHT-CDI using our metric.It will be important to monitor the use of CDI prophylaxis, delayed treatment, and testing practices, and to leverage those data to adjust the measure in response to changes in clinical practices so that the cHT-CDI definition continues to reflect true infections and incentivize best-care practices.
The risk adjustment achieved with the complex model is distinct and uniquely distinguishes differential changes in cHT-CDI ranking when compared to unadjusted rates.In addition to incorporating factors included in the simple model, the complex model includes differences in CDI testing practices that, in aggregate, improve model fit, may reduce estimation error, and may more accurately reflect underlying patient risk for cHT-CDI than broad facility-level categories.Facility descriptors, patient characteristics, CO-CDI prevalence, and different aspects of CDI testing intensity and prevalence during the HO and CO periods were significant factors associated with cHT-CDI incidence.A national cHT-CDI metric should endeavor to include these characteristics to empower stewardship and infection prevention programs with the highest quality data and thereby reduce CDI and improve patient care.
Supplementary material.The supplementary material for this article can be found at https://doi.org/10.1017/ice.2023.124

Figure 1 .
Figure 1.Hospital rankings for top-quartile hospitals (designated 1 through 50) based on observed cHT-CDI rates compared with the simple-and complex-model-derived SIR ranking.Gray bars represent rank of the top quartile of hospitals based on observed cHT-CDI rate per 100 admissions, blue diamonds represent the simple-model SIR-based rank, and orange circles represent the complex-model SIR-based rank.Note.cHT-CDI, candidate definition for healthcare facility-onset, treated C. difficile infection; SIR, standardized infection ratio.

Table 1 .
Descriptive Statistics of cHT-CDI Rate and Bivariate Analysis Results

Table 1 .
(Continued ) from 265 acute-care hospitals in the United States.Medical schoolaffiliated hospitals accounted for 38.1% and urban facilities for 60.8%.

Table 2 .
cHT-CDI-Associated Variables in the Simple Model with Estimated Incidence Rate Ratios (IRRs) a Note.CI, confidence interval; CO-CDI, community-onset C. difficile infection; cHT-CDI, candidate definition for healthcare facility-onset, treated C. difficile infection; ICU, intensive care unit; IRR, incidence rate ratio; LOS, length of stay.aFormodel replication purposes, regression coefficients and standard errors are presented in Supplementary TableS1(online).b Estimated increase in cHT-CDI relative to the reference.As an example, for bed size 101-200 IRR, 1.80; holding other variables constant in the model, hospitals with bed sizes of 101-200 are expected to have a cHT-CDI rate 1.80 times greater (80% higher) than the hospitals with bed size 1-50.

Table 3 .
cHT-CDI-Associated Variables Identified in the Complex Model with Estimated Incidence Rate Ratios (IRRs) a a For model replication purposes, regression coefficients and standard errors are presented in Supplementary TableS2(online).b Estimated increase in cHT-CDI relative to the reference.For example, for teaching hospitals IRR, 1.09; holding other variables constant in the model, teaching hospitals are expected to have a cHT-CDI rate 1.09 times greater (9% higher) than nonteaching hospitals.