A “resistance calculator”: Simple stewardship intervention for refining empiric practices of antimicrobials in acute-care hospitals

Objective: In the era of widespread resistance, there are 2 time points at which most empiric prescription errors occur among hospitalized adults: (1) upon admission (UA) when treating patients at risk of multidrug-resistant organisms (MDROs) and (2) during hospitalization, when treating patients at risk of extensively drug-resistant organisms (XDROs). These errors adversely influence patient outcomes and the hospital’s ecology. Design and setting: Retrospective cohort study, Shamir Medical Center, Israel, 2016. Patients: Adult patients (aged >18 years) hospitalized with sepsis. Methods: Logistic regressions were used to develop predictive models for (1) MDRO UA and (2) nosocomial XDRO. Their performances on the derivation data sets, and on 7 other validation data sets, were assessed using the area under the receiver operating characteristic curve (ROC AUC). Results: In total, 4,114 patients were included: 2,472 patients with sepsis UA and 1,642 with nosocomial sepsis. The MDRO UA score included 10 parameters, and with a cutoff of ≥22 points, it had an ROC AUC of 0.85. The nosocomial XDRO score included 7 parameters, and with a cutoff of ≥36 points, it had an ROC AUC of 0.87. The range of ROC AUCs for the validation data sets was 0.7–0.88 for the MDRO UA score and was 0.66–0.75 for nosocomial XDRO score. We created a free web calculator (https://assafharofe.azurewebsites.net). Conclusions: A simple electronic calculator could aid with empiric prescription during an encounter with a septic patient. Future implementation studies are needed to evaluate its utility in improving patient outcomes and in reducing overall resistances.

2002 criteria, to guide the empiric initiation of broad-spectrum agents. 10 Further work has shown that these definitions perform poorly in predicting resistance. 11 Some prescribers use scores composed of acute illness indices, such as the APACHE-II for septic shock 12 or the CURB-65 for community-onset pneumonia, 13 to guide empiric coverage for MDROs, even though those scores were developed and validated for other purposes. Therefore, prescribers need directive and reliable tools to improve appropriate targeting of antimicrobials to populations at high risk of MDROs or XDROs and to reduce the misuse of broad-spectrum therapeutics. 7 The management at the initial acute phase of sepsis is crucial to patient outcomes. 14 There are 2 common time points at which decisions are made pertaining to initial management of acute sepsis, and errors frequently occur. 7 The first is upon admission (UA), with regard to MDROs such as extended-spectrum β-lactamase-producing Enterobacterales (ESBL), 15 methicillinresistant Staphylococcus aureus (MRSA), 16 and Pseudomonas aeruginosa. The second is in cases of nosocomial infection, with regard to XDROs, such as vancomycin-resistant Enterococcus (VRE), 17 and carbapenem-nonsusceptible gram-negative bacteria, such as Enterobacterales (CRE), 18 Acinetobacter baumannii (CRAB), 19 and P. aeruginosa (CRPA). 20 Prior attempts to develop tools to predict resistance have been limited by several factors. Some scores lacked comprehensiveness and were either focused on a specific pathogen (eg, differentiating ESBL from CRE), 21 on a specific drug (eg, predicting resistances to piperacillin-tazobactam, cefepime, or meropenem in the ICU), 22 or were limited to patients with a single infectious syndrome (eg, pneumonia or UTI). 23,24 These studies addressed certain aspects of empiric prescription, but they did not address the whole process of decision making that prescribers face, particularly among cognitively impaired patients, in which the infectious syndrome (and naturally the offending pathogen) at the initial acute phase are difficult to determine. 7 Moreover, prior scores were limited by low performance, 22 small sample size, 25 the absence of validation data sets, 22 requiring parameters that were unpractical to extract bedside (eg, Charlson scores), 25,26 or reliance on additional non-standard-of-care testing (eg, arterial blood gases to calculate APACHE for patients with mild sepsis). 22 In addition, prescribers in general are reluctant to memorize scores. 7 We aimed to generate a simple electronic calculator to predict both MDRO UA and nosocomial XDRO infections, and only require parameters to be entered which are readily available at the bedside.

Methods
We conducted a retrospective cohort study at Shamir Medical Center (SMC), Israel. The derivation data set for the MDRO UA score consisted of patients with sepsis admitted to the emergency room from August 28, 2016, to December 12, 2016 (overall, 106 consecutive days, a convenience sample based on a-priori power calculations). The derivation data set for the nosocomial XDRO score consisted of hospitalized septic patients from the entire calendar year of 2016. The institution's ethics committee approved the study prior to initiation.
We applied the following inclusion criteria: patients aged >18 years, with presence of sepsis, 27 from whom blood cultures were obtained. According to hospital regulations, blood cultures are ordered from every hospitalized patient with sepsis, disregarding its severity or its origin. Data used for the derivation of the MDRO UA score data set were restricted to patients with sepsis from whom blood cultures were obtained during the first 2 calendar days of hospitalization (ie, the day of ER admission was considered day 1). The nosocomial XDRO score was based on a data set of septic patients from whom blood cultures were obtained on day 4 of hospitalization and thereafter. Patients from whom blood cultures were drawn on day 3 of hospitalization were not included in either data set. Patients with monomicrobial, microbiologically confirmed, anaerobic or fungal infection were excluded because these infections are subjected to a different set of prediction rules. 28 Multidrug-resistant organisms (MDROs) and extensively drug-resistant organisms (XDROs) We utilized broad definitions of MDROs and XDROs as determined by an international consortium in 2012, 2 and we modified them to include pathogens resistant to the initial first line of therapy recommended by official guidelines. 29 MDROs were defined as follows: (1) MRSA, (2) ampicillin-nonsusceptible enterococci, (3) penicillin-or ceftriaxone-nonsusceptible Streptococcus pneumoniae, (4) P. aeruginosa, (5) A. baumannii, or (6) Enterobacterales nonsusceptible to 1 or more third-generation cephalosporin (eg, ceftriaxone, ceftazidime).

Data collection derivation data set
Clinical data were retrieved from all records. Mortality data were retrieved from a national registry governed by the Israeli Ministry of the Interior. More than 200 potential score components were captured (Supplementary Table 1 online), including demographics, healthcare exposures, comorbidities, acute illness indices, microbiological data, antimicrobial treatments, and outcomes. The infectious clinical syndrome was determined based on established criteria. 32 The microbiological diagnosis was determined based on blood culture results or culture results from nonsterile sites that were coupled to the appropriate infectious syndrome (eg, respiratory culture for patients with pneumonia, urine culture for patients with urinary-tract infection and so on). Only parameters that were readily available at the bedside to a clinician equipped with an electronic chart were incorporated into the models.

Validation data sets
The score performances were tested on different historical validation data sets. Among them, 6 were from SMC and 1 was from Detroit Medical Center. The same inclusion, exclusion, and stratification criteria were applied to all data sets. In a few of the validation data sets, only patients with positive blood culture results were included. The MDRO UA score was validated for 3 data sets, and the nosocomial XDRO score was validated for 4 data sets (Supplementary Table 2 online).

Statistical analyses
All analyses were performed using Stata version 13.1 software (StataCorp, College Station, TX). Regarding model validation, all variables of interest were tested using a forward-stepwise logistic regression model requiring a P value <.15 by the likelihood ratio test for inclusion. After the addition of significant variables, variables with a P value ≥.20 were excluded. Variables with a negative coefficient in the model were then excluded for the utility of the prediction score. Each remaining parameter was assigned a score equal to 10 times its coefficient, and scores for individual parameters were summed to determine a total score. A receiver operating characteristic (ROC) curve was generated and the area under the curve (AUC) was calculated to assess model performance. Sensitivity, specificity, positive predictive values (PPV) and negative predictive value (NPV) were calculated following the assignment of a cutoff score. A free web-based calculator was created that responds to all platforms of stationary and mobile devices.

Results
Overall, 4,114 patients were included: 2,472 patients in the derivation (n = 1,536) and validation (n = 936) data sets of the MDRO UA score and 1,642 patients in the derivation (n = 802) and validation (n = 840) data sets of the nosocomial XDRO score.

Descriptive characteristics of individuals in the derivation data sets
We included 1,536 consecutive adults with sepsis UA and 802 with nosocomial sepsis. Most patients were elderly (53.8% of the UA data set and 77.1% of the nosocomial data set), with a mean and median age of 62 ± 22 years for the UA data set and 77 years (range, 18-103) for the nosocomial data set. Individuals in both data sets had frequent interaction with healthcare settings and invasive procedures prior to the index hospitalization, including permanent residency at long-term care facilities (11% of the UA data set and 19% of the nosocomial data set), recent (<6 months) invasive procedure (15% of the UA data set and 47% of the nosocomial data set), and previous recent (<3 month) hospitalizations (25% of the UA data set and 39% of the nosocomial data set). The mean Charlson combined condition scores were 4.2 ± 3.5 for the UA data set and 8 ± 3.8 for the nosocomial data set. 26 Past MDRO carriage was evident among 12% of the UA data set and 33% of the nosocomial data set, and previous recent (<3 months) antibiotic therapy was documented among 41% of the UA data set and 76% of the nosocomial data set. Respiratory infections were the most common infectious syndrome among patients with sepsis UA (36%) and among patients with nosocomial sepsis (43%). The complete descriptive analysis of both populations is provided in Supplementary Table 1 (online).

The MDRO upon admission score
Patients with MDRO infection were significantly older and had higher rates of recent exposure to healthcare and to antibiotics (Table 1). Multiple background conditions were associated with MDROs, including impaired functional status and cognition at baseline as well as enhanced acute illness indices. As expected, DAAT was more common in patients with MDROs, and clinical outcomes were worse (Table 1).

The nosocomial XDRO score
Among patients with nosocomial sepsis, prior exposures to healthcare settings were much more substantial among patients with XDROs including recent long-term care facility (LTCF) stays, exposure to antibiotics, or past carriage of MDRO/XDRO. Indexes of comorbidities 26 and acute illness indices (ie, severity of sepsis, 27 Pitt bacteremia score, 33 and McCabe score 34 ) were also elevated among XDRO patients. Patients with nosocomial XDRO infections had frequent DAAT, and in nearly half of these patients, appropriate initiation of therapy was delayed for >2 days. This delay was reflected in significantly worse outcomes ( Table 1). The final nosocomial XDRO score included 7 parameters as depicted in Table 3. Using 36 points as the cutoff yielded 90% sensitivity, 72% specificity, 46% PPV, 97% NPV, and an ROC AUC of 0.87 (95% CI, 0.84-0.91) (Fig. 1). Table 4 reflects the performances of the 7 different validation historical data sets. The ROC AUC range was 0.7-0.88 among the validation data sets of the MDRO UA score, and the ROC AUC range was 0.66-0.75 among the validation data sets of the nosocomial XDRO score.

Web calculator
We created a free web calculator (https://assafharofe.azure websites.net).

Discussion
In this study, we present a simple, bedside tool to assist in empiric prescription of antibiotics during the first encounter with a septic patient. This tool could be implemented as a stewardship intervention in hospitals. For the past 80 years, antibiotics have been an efficacious and safe therapeutics, but there are concerns of it becoming a 'wasted resource,' 35 due to frequent errors in empiric prescription of broad-spectrum agents. 36 Practitioners need reliable tools to guide empiric decisions and to avoid errors, which can be harmful to patients and to facilities. To our knowledge, this tool is among the largest score creation efforts for MDROs and XDROs conducted to date. In comparison with previous studies, 21,22,24 the performance of our calculator was better. Moreover, this calculator addresses a wider range of situations, reflecting the broader perspective anticipated from a clinician, while managing a patient with acute sepsis. The high NPVs in both scores, of 98% and 96%, could assist prescribers in safely avoiding overuse of broad-spectrum agents. This tool could be a simple and effective stewardship intervention; broad-spectrum agents, on top of their detrimental ecological impact and the fiscal burden they impose, are also frequently less efficacious and more toxic to the patient. 6 Prescribers are already using "prediction tools" that are less established were not originally developed to predict resistances and displayed much lower performances (both in the original studies and herein): the Duke-2002, 8 modified Duke-2002, 10 APACHE-II, 12 and CURB-65 (for patients with community-onset pneumonia). 13 Mobile devices are used nowadays by nearly every prescriber worldwide, including in developing countries. The calculator is responsive with all devices and is compatible with network security settings. A shortcut to this calculator could be pasted to home screens of private devices, and to organizational computers located at ERs, regular floors, and in ICUs. The calculator is anonymous; it is designed to assist hospitalists, internists, surgeons, intensivists, clinical pharmacists, and every adult prescriber. However, the calculator should not be used in lieu of a comprehensive medical evaluation, and it should not replace infectious diseases (ID) consultation when appropriate. ID experts will always be needed to tailor management, and in many facilities, they are responsible for prescription of broad-spectrum antibiotics. Therefore, the calculator may be able also to assist prescribers in determining when a formal ID consultation is needed. Sometimes a unique clinical or epidemiological finding is evident: a pathognomonic rash, travel to a certain area, acknowledgement of current epidemics. These unique scenarios necessitate special management considerations and usually the direct involvement of an ID specialist. When no unique exposure or finding is evident, prescribers frequently think broadly of 4 categories of pathogens among septic adults: gram-positive organisms (sometimes with special focus on Enterococcus), gram-negative organisms, anaerobes, and fungi. Anaerobes and fungi (mainly Candida spp) are relevant in certain syndromes (eg, intra-abdominal infections, catheter-related bloodstream infections) or other scenarios (eg, following a gastrointestinal endoscopic procedure); therefore, they should be managed based on a different set of prediction rules. 28 The purpose of this calculator was  to assist clinicians, at the bedside, with prescribing anti-gram-positive agents and anti-gram-negative empiric regimens.
Our study has several limitations. The tool is based on broadly applicable determinations, both in terms of the definitions we used for stratifying the level of resistance (ie, MDRO and XDRO), and our extrapolations to the spectrum of antibacterial coverage by agents. The calculator should also be validated in other centers, to determine its generalizability. On the validation data sets that it was tested, with all historical cohorts with data limited to specific subgroups, we observed a wide range of performances, some unfavorable. The performances with the validation data sets derive the overall performance of the tool. However, in 6 of the validation data sets, only patients with positive blood cultures were included, as opposed to the derivation data set, which included the entire background population from which the cases (ie, patients with MDRO or XDRO) arose. If MDROs or XDROs are not prevalent in a facility, the tool is irrelevant, though this situation is uncommon nowadays. In some hospitals, XDRO infections may be prevalent upon admission, but this situation is rare. 37 The performance of the calculator might differ according to the percentages of MDROs and XDROs in different populations and might change over time, in accordance to temporal changes in MDROs and XDROs epidemiology. An additional limitation of our study is that the calculator does not address certain possible additional empiric prescribing dilemmas (eg, enterococcal, anaerobe, and fungal infections). 28 However, it addresses the 'backbone' of empiric regimens for the majority of situations.
To conclude, we developed a tool to assist prescribers confronted with septic patients, which is free, anonymous, and accessible at the bedside, though it lacks additional validated data. Our future goal is to test the tool in a prospective interventional study on several outcomes, considering DAAT, mortality rates, and outcomes that measure the ecological burden of antimicrobials.   (1) Methicillin-resistant staphylococcus aureus (MRSA), (2) ampicillin non-susceptible enterococci, (3) penicillin or ceftriaxone nonsusceptible Streptococcus pneumoniae, (4) Pseudomonas aeruginosa, (5) Acinetobacter baumannii, and (6) Enterobacterales nonsusceptible to 1 or more third-generation cephalosporin (eg, ceftriaxone, ceftazidime). b This refers to the status of the patient prior to infection onset (include bed-ridden). c This refers to any type of invasive procedure, including any type of surgery (from minor to major, ie, the whole spectrum), endoscopies, permanent central line insertions, any percutaneous procedure (eg, coronary angiography), ascites paracentesis, and more.