Introduction
Screening newly admitted patients for multidrug-resistant organism (MDRO) colonization may be an important component of prevention strategies and inform empiric antimicrobial selection, but is resource-intensive. Reference Lin, Tseng and Gatalo1,Reference Robotham, Deeny, Fuller, Hopkins, Cookson and Stone2 Given the low overall prevalence of some MDROs (e.g., carbapenem-resistant Enterobacterales, carbapenem-resistant Pseudomonas aeruginosa) in many healthcare settings, Reference Sansom, Shimasaki and Dangana3 a targeted screening approach that stratifies risk for MDRO colonization at the time of admission may be more practical and cost-effective for screening programs. Reference Shimasaki, Segreti and Tomich4
In the U.S., prior healthcare exposure is a major risk factor for colonization with MDROs and could potentially be leveraged to risk-stratify patients for targeted MDRO screening. Reference Lin, Lyles-Banks and Lolans5 At the institutional level, admission assessment of prior healthcare exposure is difficult due to missing information within the medical record or limited communication between healthcare facilities at the time of patient transfer. Reference Shimasaki, Segreti and Tomich4,Reference Goodman, Taneja and Magder6 In contrast, a centralized public health model has the advantage of being able to efficiently capture patients’ prior exposure to all healthcare facilities in the jurisdiction, with the trade-off that capture of clinical variables may be limited to billing codes (i.e., ICD codes).
We previously developed and validated a Public Health Risk Model to identify carbapenem-resistant Enterobacterales colonization at the time of hospital admission using patient-level healthcare exposure data in the Illinois Hospital Discharge Dataset. Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7 In the original derivation of the model, we identified age, acute care hospitalizations in the prior 365 days (both short-term and long-term hospitals), and ICD-coded infection diagnoses as independent risk factors for carbapenem-resistant Enterobacterales colonization at the time of hospital admission. Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7
In this study, we performed a comprehensive validation of the Public Health Risk Model in a medical intensive care unit cohort. Our study cohort was screened for multiple MDROs at the time of medical intensive care unit admission, resulting in robust characterization of MDRO colonization status. Reference Sansom, Shimasaki and Dangana3 In addition to validating model performance to predict carbapenem-resistant Enterobacterales colonization, we also tested the hypothesis that the model would be generalizable to predict colonization by other MDROs due to shared risk factors. Reference Safdar and Maki8 Additionally, we compared the performance of the model to traditional screening strategies that use locally captured clinical variables to target screening.
Methods
Setting, participants, and study covariates
We conducted a retrospective model validation study at Rush University Medical Center, a tertiary-care medical center in Chicago, Illinois. The study cohort comprised patients ≥18 years old who were admitted to the 25-bed medical intensive care unit from January 11, 2017 through January 10, 2018 and who had been prospectively cultured for MDROs via rectal or fecal swab within 48 hours of admission. Reference Sansom, Shimasaki and Dangana3 Our region is endemic for the MDROs of interest; MDROs of interest included carbapenem-resistant Enterobacterales, carbapenem-resistant Pseudomonas aeruginosa, carbapenem-resistant Acinetobacter baumannii, vancomycin-resistant enterococci, and third-generation cephalosporin-resistant Enterobacterales. Methicillin-resistant Staphylococcus aureus nasal colonization at admission was determined by routine clinical screening, as required per Illinois law under the MRSA Screening and Reporting Act [210 ILCS 83]. Carbapenem-resistant Enterobacterales, carbapenem-resistant P. aeruginosa, and carbapenem-resistant A. baumannii were further classified as a composite group of carbapenem-resistant organisms (CROs) in our analyses. CROs were defined as (1) Enterobacterales isolates exhibiting phenotypic resistance to ertapenem, imipenem or meropenem, or (2) P. aeruginosa or A. baumannii isolates exhibiting phenotypic resistance to imipenem or meropenem. Minimal inhibitory concentration results were interpreted using Clinical Laboratory Standards Institute breakpoints. 9
Data linkage and public health model risk score generation
Each patient’s index medical intensive care unit admission from the validation cohort was linked to the Illinois Hospital Discharge Dataset, 10 with secure data transfer between institutions using the open-source privacy-preserving record linkage system Linkja. Reference Doshi, Trick and Hinami11 The Illinois Hospital Discharge Dataset contains comprehensive encounter-level information for all Illinois hospitalizations in short-term and long-term acute care hospitals. Hospital encounter data in this dataset included patient identifiers, facility identifiers, and encounter characteristics (e.g., dates of hospital admission and discharge, diagnosis codes, and procedure codes). By applying the coefficients from our prior Public Health Risk Model, Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7 the Illinois Department of Public Health retrospectively calculated the probability of MDRO colonization at the time of medical intensive care unit admission for individuals in our cohort. Covariates included age and, during the prior 365 days, number of short-term acute care hospitalizations and mean length of stay, number of long-term acute care hospitalizations and mean length of stay, and prior hospital admission with an ICD-10 diagnosis code indicating probable bacterial infection. Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7 Calculation of the Public Health Risk Score was centralized within state public health bioinformatics infrastructure and was reported as a percent. The linked dataset was de-identified and model inputs were opaque to the research team.
Statistical analyses
Demographic and clinical characteristics were expressed as frequencies with proportions and medians with interquartile ranges, as appropriate. Public Health Risk Model performance to predict MDROs was evaluated by applying the frozen coefficients from the prior model using logistic regression. Model discrimination for MDRO colonization was evaluated as area under the curve (AUC) using receiver operating characteristic (ROC) curves. Model performance was further evaluated by calibration plot and Brier score.
We compared performance of the model for identification of MDRO colonization to traditional screening strategies utilizing patient clinical characteristics typically available to clinicians at time of admission: (1) Hospital transfer (from a short-term or long-term acute care hospital), (2) Presence of either a tracheostomy or a percutaneous gastronomy tube, and (3) Presence of a pressure ulcer. Reference Wilson, Sanderson, Westgate, Winter and Forster12,Reference Park, Kim, Gatchalian and Oh13 The presence of tracheostomy, gastrostomy tube, or pressure ulcer was assessed through daily prospective review of bedside medical records at the time of initial cohort enrollment. Transfer from other hospital was determined retrospectively from the electronic medical record.
Missing values for the model probability (n = 13) after data linkage were assigned a value of 0, as that would be consistent with the value assigned when operationalized. Association of traditional screening strategies with MDRO colonization were evaluated by χ 2 testing. Sensitivity was calculated as the proportion of true positives identified using each approach compared to the reference standard of all patients evaluated by culture detection at the time of medical intensive care unit admission. Specificity was calculated as the proportion of true negatives by each approach compared to the reference standard. The Number Needed to Screen (NNS) to detect one carrier was calculated as the inverse of the positive predictive value. For all calculations, statistical significance was defined as a two-tailed P-value < .05. Statistical analysis was performed using SAS version 9.4 (Cary, NC). Default statistical tests in SAS were used for generation of confidence intervals, including the Hanley and McNeil method for AUC confidence intervals and the asymptotic Wald method for sensitivity.
Results
The public health risk model predicted carbapenem-resistant enterobacterales and healthcare-associated MDRO colonization
The study cohort included 1,250 unique participants, of whom 1,237 (98.9%) were successfully linked to the Illinois Hospital Discharge Dataset. Using the model, most admission-level risk scores were low (median 0.3%, IQR 0.2–0.4%, range 0–74.6%); only 19.2% (n = 238) had a measured risk of ≥ 0.5%, which we determined post hoc as the threshold that optimized the balance between sensitivity and specificity. Among these 1,237 admissions, 27 originally tested positive for carbapenem-resistant Enterobacterales carriage, of which 17 had detection of a carbapenemase gene (16 blaKPC and 1 blaNDM). Regarding other MDROs, 10 individuals tested positive for carbapenem-resistant P. aeruginosa (of which all were negative for presence of carbapenemase genes), 160 positive for vancomycin-resistant enterococci, 217 positive for third-generation cephalosporin-resistant Enterobacterales, 53 positive for methicillin-resistant S. aureus, and 1 positive for carbapenem-resistant A. baumannii. The patient colonized with A. baumannii was co-colonized with carbapenem-resistant Enterobacterales.
The model discriminated well for colonization with carbapenem-resistant Enterobacterales at the time of admission (AUC 0.82, 95% confidence interval [CI] 0.72–0.91). The model also discriminated well for colonization with other healthcare-associated MDROs (i.e., carbapenem-resistant P. aeruginosa, vancomycin-resistant enterococci). The model performed poorly to predict MDROs with known local community reservoirs (i.e., third-generation cephalosporin-resistant Enterobacterales, methicillin-resistant S. aureus) (Table 1, Figure S1).
Table 1. Application of the public health risk model to identify MDROs
The public health risk model performed better than traditional screening strategies for CRO colonization
We next assessed performance of the model compared to traditional screening strategies. We first explored various thresholds to apply the model to identify patients with CRO colonization. Higher risk score cutoff was inversely related to screening sensitivity (Figure 1). A sensitivity table exploring different cutoff values is also shown in Supplemental Table 1. We identified a threshold of ≥ 0.5% as reasonably balanced between NNS and sensitivity for a screening test. Metrics to assess discrimination and calibration were generally favorable (intercept –3.60, slope 5.76, c-statistic/AUC 0.82, Likelihood Ratio P = .001, Wald P < .001, Brier score 0.029; model diagnostics shown in Supplemental Figure 2).
Figure 1. Determination of public health risk model thresholds to predict carbapenem-resistant organism (CRO) colonization. Various thresholds to apply the model to identify patients with CRO colonization were evaluated using number needed to screen (left Y-axis; purple solid line) and sensitivity (right Y-axis; blue dotted line). The absolute risk inferred from the number needed to screen appears higher than the predicted modeled risk because the risk model was originally calibrated to a lower CRO prevalence population (all hospitalized patients in Illinois), whereas the current validation utilized a higher prevalence cohort (intensive care unit patients).
Next, we evaluated the traditional screening strategies that utilize patient risk factors at time of admission, including hospital transfer, presence of a tracheostomy or percutaneous gastrostomy tube, and presence of a pressure ulcer. We found that hospital transfer (P = .02), tracheostomy (P < .001), and presence of pressure ulcer (P < .001) were each associated with CRO colonization (Table 2). We then compared the sensitivity and NNS to detect one CRO carrier using traditional screening strategies to the Public Health Risk Model. The sensitivities of traditional screening strategies were lower than that of the model for identification of CRO colonization (Figure 2). Diagnostic measures for each strategy are shown in Supplemental Table 2.
Figure 2. Sensitivity of targeted carbapenem-resistant organism (CRO) screening strategies. Sensitivity and number needed to screen of different strategies were compared. The sensitivities of traditional screening strategies were lower compared to the public health risk model for identification of CRO colonization. aFor clinical risk factors, the presence of any one risk factor would trigger screening. bNumber needed to screen to detect one CRO carrier was calculated as the inverse of the positive predictive value.
Table 2. Traditional patient risk factors for CRO colonization
Discussion
We found that a risk-based prediction model using patient-level healthcare exposure data from a state public health dataset Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7 could discriminate critically ill patients more likely to harbor healthcare-associated MDROs at the time of hospital admission. We confirmed that the Public Health Risk Model could predict a patient’s carbapenem-resistant Enterobacterales colonization status at time of admission, and for the first time, we demonstrated the model’s generalizability to other healthcare-associated MDROs, including carbapenem-resistant P. aeruginosa and vancomycin-resistant enterococci. The model did not predict MDROs with significant local community reservoirs of transmission, including third-generation cephalosporin-resistant Enterobacterales and methicillin-resistant S. aureus. The sensitivity of the model for discrimination of CRO carriage exceeded that of traditional screening strategies.
The use of a centralized state public health database for MDRO prediction can overcome major limitations in data sharing that occur when patients receive care across multiple healthcare institutions. For example, prior long-term acute care hospital exposure has been identified as one of the strongest risk factors for colonization with MDROs. Reference Prabaker, Lin and McNally14,Reference Goodman, Simner and Klein15 However, for patients transferred directly from a long-term acute care hospital to our hospital, we have not identified a local electronic method to reliably distinguish long-term acute care hospital from other hospitals as the sending facility. Reference Shimasaki, Segreti and Tomich4 Furthermore, patients with long-term acute care hospital exposure in the prior year may have had intervening care at a nursing home or have returned home prior to readmission to our hospital, rendering point-of-origin billing codes or clinical documentation at time of admission less accurate. In a multicenter retrospective study, Goodman et al. found similar limitations in identifying prior long-term care facility exposure using only electronic billing codes or clinical documentation at their institution, highlighting the value of a statewide database to identify prior healthcare exposures. Reference Goodman, Taneja and Magder6 Although there was success with the Illinois Department of Public Health, challenges to use of a centralized state public health database include state-to-state variability in the accessibility of such a dataset and the internal capacity to deploy the prediction model’s parameters by personnel at public health departments.
Targeted screening strategies should be tailored to each institution based on available resources and local MDRO epidemiology. Reference Ambretti, Bassetti and Clerici16 We demonstrated that traditional screening approaches are useful to evaluate risk of CRO colonization and may be used in healthcare settings where state public health databases are not readily available. Some clinical variables, such as presence of a tracheostomy or pressure ulcer, are easily obtained through physical examination. Our findings align with prior studies demonstrating variable success in identifying patients with CROs using clinical risk factors. Frequently identified risk factors include prior healthcare exposure (e.g., hospitalization, long-term care residency), Reference Shimasaki, Segreti and Tomich4,Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7,Reference Prabaker, Lin and McNally14,Reference Goodman, Simner and Klein15,Reference Mathers, Vegesana and German-Mesner17–Reference Salomao, Guimaraes and Duailibi19 exposure to antimicrobial agents,Reference Lin, Ray, Rezny, Runningdeer, Weinstein and Trick7,Reference Goodman, Simner and Klein15,Reference Dai, Zhang and Pan18–Reference Sullivan, Ichikawa, Dudley, Li and Aberg20 invasive devices or procedures (e.g., mechanical ventilation, tracheostomy, central line),Reference Goodman, Simner and Klein15,Reference Dai, Zhang and Pan18,Reference Giannella, Freire and Rinaldi21 presence of pressure ulcers,Reference Braga, Pirett, Ribas, Gontijo Filho and Diogo Filho22 and history of MDROs .Reference Goodman, Simner and Klein15,Reference Sullivan, Ichikawa, Dudley, Li and Aberg20,Reference Giannella, Freire and Rinaldi21 The variation in identified CRO risk factors across models may be attributable to host and organism heterogeneity, differences in regional CRO epidemiology, CRO misclassification, and limitations of data capture by the electronic health record.Reference Goodman, Simner and Klein15
Our study demonstrates the potential utility of a public health database to improve infection prevention. Public health departments may provide actionable information to local healthcare institutions and providers that extends beyond traditional sharing of reportable diseases. In Illinois, we are testing near-real time automated notification of high-risk patients identified through the current Public Health Risk Model to admitting healthcare facilities using an existing alerting infrastructure (the Illinois XDRO registry 23 ). The statewide discharge database used to generate these risk scores is updated quarterly, which may introduce a lag of up to 3 months in available data for risk score generation. Although several of the strategies evaluated in our study had limited sensitivity to identify CRO carriage, this does not negate their usefulness in selecting populations for targeted screening. For example, a facility with very limited resources could prioritize a higher specificity approach by performing CRO screening for patients admitted with tracheostomies, which required screening of only 6 individuals to identify one CRO carrier in our cohort.
There are several limitations that should be considered when interpreting the results of our study. First, we considered culture detection of MDRO colonization to be the reference standard for this study, though culture detection may be less sensitive than molecular detection methods when organism burden is low. Reference Sansom, Shimasaki and Dangana3,Reference DAgata, Gautam, Green and Tang24 Second, our study cohort comprised medical intensive care unit patients in a region where CROs are endemic. Reference Shimasaki, Segreti and Tomich4,Reference Lapp, Crawford and Miles-Jay25,Reference Ray, Lin, Weinstein and Trick26 Thus, our findings may not be generalizable to non-critically ill populations or to other geographic regions. Third, the Illinois Hospital Discharge Dataset has current limitations that influence its ability to accurately inform risk prediction, namely that data updates occur quarterly with lags up to 6 months, and lack of encounter data from skilled nursing facilities.
In conclusion, we found that a risk model based on a state-level public health hospital discharge database could assess risk in critically ill patients for colonization with healthcare-associated MDROs, demonstrating improved sensitivity over traditional screening strategies.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2026.10397
Acknowledgements
We extend our deepest gratitude to the patients and staff in our medical intensive care unit for making this study possible. We also thank Kausthub Jayar for his assistance with data extraction.
Financial support
The project described was supported by cooperative agreement U54CK000607 from Centers of Disease Control and Prevention (CDC). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CDC.
Competing interests
All authors report no conflicts of interest relevant to this article.
Open access
