Surgical site infections (SSIs) cause substantial morbidity, often incurring reoperation and unanticipated hospitalization, with estimated attributable costs of >$3 billion annually. Reference Zimlichman, Henderson and Tamir1,Reference Kirkland, Briggs, Trivette, Wilkinson and Sexton2 Preventing infection after colon surgery is a national priority for the >300,000 procedures performed annually in the United States, 3 and their associated SSI rates are as high as 14%–25%. Reference Howard, Datta, Cunnick, Gatzen and Huang4,Reference Kiran, El-Gazzaz, Vogel and Remzi5 The high-volume, high-risk impact of these procedures has led to mandatory reporting of colon SSIs by the Centers for Medicare & Medicaid Services (CMS) to facilitate improvements in surgical processes and outcomes. 6,7 Hospital performance is tied to financial penalties through both the Hospital Value-Based Purchasing (VBP) Program and the Hospital-Acquired Condition (HAC) Reduction Program. Reference Blumenthal and Jena8–10
Diagnostic claims codes have proven to be an accurate and efficient proxy for SSIs at the hospital level. Prior studies have shown that these codes can be used to accurately rank hospitals by SSI rates based upon national validation efforts involving review of full-text medical records using Centers for Disease Control and Prevention (CDC) SSI criteria. Reference Huang, Placzek and Livingston11–Reference Platt, Kleinman and Thompson15 In fact, these codes identify a large proportion of SSI cases that would be missed by hospital surveillance. Reference Letourneau, Calderwood, Huang, Bratzler, Ma and Yokoe16 As a result, these codes have been adopted by the CMS Hospital Inpatient Quality Reporting Program to validate hospital SSI reporting. 17
The CMS adjusts hospital-specific rates of colon SSIs for several risk factors for SSI in an effort to create a fair playing field for hospitals serving more complex surgical populations. However, this adjustment lacks several relevant variables. Neither the CMS nor the CDC account for concomitant intra-abdominal procedures as a potential risk factor for SSI. These procedures increase the procedure time and complexity and have been associated with increased colon SSI risk. Reference Caroff, Wang and Zhang18,Reference Calderwood MS, Murphy, Platt and SS19 Similarly, other comorbidities, such as tobacco use, Reference Augenstein, Colavita and Wormer20–Reference Moreno Elola-Olaso, Davenport, Hundley, Daily and Gedaly24 renal disease, Reference Daley, Khuri and Henderson25 vascular disease, Reference Karamanos, Kandagatla, Watson, Schmoekel and Siddiqui26 cirrhosis, Reference Darouiche, Wall and Itani27 and malignancy, Reference Darouiche, Wall and Itani27 have been shown to affect SSI as well, but these are not included in adjustment models. Electronically available comorbidities derived from International Classification of Disease (ICD) codes also have been shown to improve risk adjustment and significantly change hospital SSI rankings. Reference Caroff, Wang and Zhang18,Reference Jackson, Leekha and Magder28
Additionally, surgical approach (laparoscopic or open) is not included in the CMS model despite clear evidence of its benefit to equitable ranking of hospitals by SSI risk. Data from the National Surgical Quality Improvement Program (NSQIP) show that laparoscopic colon surgery carries only half the SSI risk of open colon surgery and that laparoscopic patients tend to be younger and fitter and have significantly fewer comorbidities. Reference Kiran, El-Gazzaz, Vogel and Remzi5 This pattern has been shown repeatedly in other studies. Reference Howard, Datta, Cunnick, Gatzen and Huang4,Reference Onder, Benlice, Church, Kessler and Gorgun29–Reference Gandaglia, Ghani and Sood31 Nevertheless, laparoscopic and open colon procedures are grouped together by CMS to generate hospital-specific SSI rates and rankings. This CMS method combines a number of different colon procedures with widely varying patient populations, SSI rates, and uptake among hospitals, which raises questions about its ability to properly adjust for these variations and mitigate the downstream impact on hospital rankings. In this study, we evaluated the impact of separately ranking hospitals using open versus laparoscopic approach, and we explored more informative and equitable ways of comparing colon SSI rates.
Methods
We evaluated hospital SSI rates and generated rankings following colon surgery in fee-for-service Medicare beneficiaries between January 1, 2009, and November 30, 2013, based on Medicare Part A inpatient claims data. Reference Letourneau, Calderwood, Huang, Bratzler, Ma and Yokoe16,Reference Calderwood, Huang, Keller, Bruce, Kazerouni and Janssen32 We identified colon surgeries using the 37 ICD-9 procedure codes used by the CDC and CMS at that time for national SSI surveillance and reporting, including 8 laparoscopic and 29 open procedures. 33 When a laparoscopic and an open procedure were coded on the same date, the procedure was considered to be an open approach.
For each colon procedure, we identified patient descriptors, including age, sex, race, and comorbidity score (based on Elixhauser score Reference Elixhauser, Steiner, Harris and Coffey34 ) at the time of the procedure. The number of procedures and associated patient characteristics were aggregated at the hospital level. The frequency at which each procedure was performed with other concomitant colon and noncolon intra-abdominal procedures was assessed overall and by hospital. We used χ2 tests to assess crude associations, unadjusted for risk factors or for correlation within hospital between patient characteritics and surgical approach.
SSI determinations were made using previously validated administrative claims codes indicating postsurgical infection. Reference Letourneau, Calderwood, Huang, Bratzler, Ma and Yokoe16 This method has been shown to accurately rank hospitals by their colon SSI rates and is used by the CMS Hospital Inpatient Quality Reporting Program to validate hospital SSI reporting. Reference Huang, Placzek and Livingston11–Reference Letourneau, Calderwood, Huang, Bratzler, Ma and Yokoe16 We assessed claims made within 30 days of the surgical procedure using admininistrative claims codes suggestive of a deep or organ-space SSI: ICD-9 procedure codes 54.0, 54.11, 54.19, 86.04, 86.22, and 86.28, and ICD-9 diagnostic codes 567.21, 567.22, 567.29, 567.38, 569.5, 569.61, 569.81, 682.2, 879.9, 998.31, 998.59, and 998.6. SSI rates were calculated within hospital for each colon procedure type, among all colon procedures, and according to laparoscopic versus open status.
To exclude pre-existing infections, we excluded procedures for which an SSI claims code was designated as present on admission during the index surgical hospitalization. For patients who underwent another major surgery in the 30-day postoperative surveillance window, we censored our surveillance at the time of the subsequent surgery.
Our main analyses used generalized linear mixed model logistic regression with age, sex, race, Elixhauser comorbidity score, concomitant colon and noncolon intra-abdominal procedures, and laparoscopic status to predict SSI risk for each procedure type, accounting for clustering at the hospital level. We calculated an adjusted SSI rate for each hospital based on the population SSI rate times the hospital-specific odds ratio from the model. We ranked hospitals by their adjusted SSI rate.
Our analyses were replicated for 3 categories of procedures: (1) all colon procedures (ie, the 37 ICD-9 procedure codes listed above); (2) open procedures only (29 procedure codes); and (3) laparoscopic procedures only (8 procedure codes). A plot of adjusted hospital rankings comparing SSI rates for open versus laparoscopic procedures was generated and compared using a test of correlation. We also created 2 subgroups for sensitivity analyses by (1) excluding rare procedures that were performed by <25% of hospitals and by (2) limiting the analysis to the 5 colon procedures for which both analogous open and laparoscopic versions exist. Reference Caroff, Chan and Kleinman36 (Supplementary Table 1 online) Models were rebuilt and resulting hospital rankings compared within each of these subgroups.
Results
In total, 694,813 colon procedures were performed at 4,093 hospitals involving 508,135 Medicare patients during the study period. These Medicare patients had a mean age of 74.3 years (standard deviation [SD], 9.7); 42.5% were male; 32.3% had other concomitant colon procedures, and their mean comorbidity score was 4.7 (SD, 7.8).
We detected large differences between those undergoing open versus laparoscopic procedures. Overall, the unadjusted hospital mean SSI rate was 9.8% (SD, 11.2) for all procedures, but it was 11.4% (SD, 11.7; range, 0–38.5) for open procedures and 5.8% (SD, 12.4; range, 3.9–10.5) for laparoscopic procedures. The proportion with a concomitant colon procedure was 39.5% versus 23.1% among open versus laparoscopic procedures (P < .001), and the proportion with a concomitant noncolon intra-abdominal procedure was 37.9% versus 20.6%, respectively (P < .001). Among those receiving open procedures, the mean hospital Elixhauser score was 5.1 (SD, 3.1) compared to 2.7 (SD, 2.8) among those receiving laparoscopic procedures.
The proportion of hospitals performing specific procedures varied widely, with a range of 0.1% to 92.8% across procedure codes (Table 1). The mean numbers of colon procedures per hospital were 119.5 (SD, 139.9) for open procedures and 65.2 (SD, 90.8) for laparoscopic procedures. Moreover, 811 hospitals (19.8%) used only an open approach, and among the 3,282 hospitals (80.2%) that used any laparoscopy, proportions of laparoscopic procedures varied widely (range, 0.6%–100%). Few hospitals (n = 70, 1.7%) used only laparoscopy. Mean hospital-specific Elixhauser scores varied by procedure from 2.1 to 8.3, and the mean Elixhauser score across all hospitals was 4.3 (SD, 2.8). Multivariable models showed that increasing Elixhauser score was associated with increased SSI risk: OR, 1.06 per unit Elixhauser score (95% CI, 1.04–1.07) for all procedures; OR, 1.04 (95% CI, 1.02–1.05) for open procedures only; and OR, 1.18 (95% CI, 1.14–1.22) for laparoscopic procedures only. Concomitant intra-abdominal noncolon procedures also showed increased SSI risk (Table 2). Laparoscopic surgery was strongly associated with reduced SSI risk (OR, 0.5; 95% CI, 0.4–0.5) relative to open surgery.
Baseline Characteristics By Procedure Code
Characteristics Associated with Surgical Site Infection (SSI) for All Colon Procedures, Open Procedures, and Laparoscopic Procedures
Note. OR, odds ratio; CI, confidence interval.
Hispanic and North American Native race were associated with increased SSI risk, although in the laparoscopic-only group, the confidence limits included the possibility of no effect. For all procedures, the OR for Hispanic race was 1.2 (95% CI, 1.1–1.3) and for Native Americans, the OR was 1.4 (95% CI, 1.2–1.6). For open procedures only, the OR for Hispanic race was 1.3 (95% CI, 1.2–1.4) and for Native Americans the OR was 1.4 (95% CI, 1.2–1.6). For laparoscopic procedures only, the OR for Hispanic race was 1.1 (95% CI, 1.0–1.4) and the OR for Native Americans was 1.3 (95% CI, 0.9–1.8) (Table 2). Similar results were obtained when we excluded procedures performed by <25% of hospitals and when we included only procedures with analogous open and laparoscopic versions.
Hospital rankings based on adjusted SSI rates differed significantly when comparing overall procedures, open procedures only, and laparoscopic procedures only (Table 3). When hospital ranks based only on open procedures were compared to ranks based only on laparoscopic procedures, 2,125 hospitals (66.2%) changed rank by 1 or more quartiles and 334 hospitals (10.4%) moved 3 quartiles, from best- to worst-performing quartiles or vice versa. Hospital rank based on overall versus open SSI rates differed but to a lesser degree: 800 hospitals (19.9%) changed rank by 1 or more quartiles, and 3 (0.1%) hospitals moved 3 quartiles. Similar results were obtained when excluding procedures performed by <25% of hospitals and when including only procedures with analogous open and laparoscopic versions (Supplementary Tables 2 and 3 online).
Changes in Hospital Surgical Site Infection Quartile Ranks By All Colon Procedures, Laparoscopic Procedures, and Open Procedures
Note: SSI, surgical site infection; N/A, values in these columns or rows represent hospitals that did not perform the designated open or laparoscopic procedures. 1, best-performing quartile; 4, worst-performing quartile. Values in bold represent agreement between each ranking approach.
Scatterplots of hospital rank by open-only versus laparoscopic-only SSI rates revealed a strikingly poor visual correspondence. The Spearman rank correlation coefficient was 0.2 for individual rank (Fig. 1). Similar results were obtained when excluding procedures performed by <25% of hospitals and when including only procedures with analogous open and laparoscopic versions (Supplementary Figs. 1 and 2 online).
Scatterplot of hospital rankings by laparoscopic and open colon surgical-site infection rates.
Discussion
Although colon surgeries are grouped by the CDC and CMS to determine hospital ranking and, in turn, reimbursement, these procedures are diverse and differ significantly in how often they are performed, how many hospitals perform them, and how often they result in SSI. In particular, open and laparoscopic procedures confer a meaningfully different risk of SSI, and lead to markedly different hospital rankings when considered separately. Properly adjusting for these differences in a single measure will be technically challenging and may not be possible or desirable.
Currently, the CMS adjusts colon SSI rates for age, sex, diabetes, American Society of Anesthesiologists (ASA) score, body mass index, closure status, and whether the procedure occurred in an oncology hospital, but the CMS fails to account for additional key differences listed below that are critical to making fair comparisons. Accounting for additional patient comorbidities, concomitant procedures, and laparoscopic versus open surgical approach affected expected SSI rates. Insufficient adjustment for complexity of illness places hospitals that care for a more complex patient population at a disadvantage because the patients they treat more often require an open surgical approach, a higher-risk procedure, or multiple simultaneous procedures. In fact, some evidence shows that the CMS HAC Reduction Program disproportionately penalizes major teaching hospitals, safety net hospitals, and hospitals with a more complex case mix. Reference Rajaram, Chung and Kinnier35
Even with optimal risk-adjustment, it is doubtful that inherent differences between open and laparoscopic procedures can be reconciled when these procedures are grouped together. In this study, hospital rankings based upon open procedures were very poorly correlated with rankings based upon laparoscopic procedures. For many hospitals, SSI rankings differed by 3 quartiles when their performance involving open procedures was compared to laparoscopic procedures. This finding suggests that these 2 fundamentally different types of procedures should not be combined into a single ranking schema on which hospital quality and performance penalties are based. Subgroup analysis of only commonly performed procedures or only procedures that can be performed by either laparoscopic or open approach reinforced these results. Ranking open and laparoscopic procedures separately would allow more appropriate adjustment for patient-level variables. A stratified ranking format would lessen the disadvantage faced by hospitals whose patients more often require an open approach out of clinical necessity by no longer forcing a comparison with lower-risk laparoscopic procedures at other centers.
Separately, because laparoscopic surgery ought to be encouraged and more widely used for its benefits in reducing SSI risk, even in certain medically complex patients, Reference Caroff, Chan and Kleinman36 it may be important to address the adoption of laparoscopic surgery in quality measures because separate rankings for open and laparoscopic procedures could reduce a hospital’s or surgeon’s incentive to optimize the use of laparoscopy. More likely, the overt differences in SSI risk by approach would increase laparoscopic use over time and ultimately enable separate studies for SSI prevention for those patients who have no alternative to an open approach.
Notably, Hispanic and North American Native race were associated with increased SSI risk. This effect was statistically significant among open procedures but not laparoscopic procedures. Although unmeasured confounders may explain this association, a direct effect would further support use of laparoscopy.
Altogether, these findings suggest an unequal playing field on which US hospitals must compete for CMS reimbursement, with a large minority of hospitals at great disadvantage because they disproportionately serve the most ill population and perform high-risk colon procedures, including those that require an open approach, those that are not offered by most hospitals, and those that involve multiple concomitant procedures or patients with higher comorbidity scores.
This study had several limitations. We did not have access to all variables used by the CMS for risk adjustment, specifically body mass index and dedicated oncology hospital status. We also did not have information on the skill or experience of the surgeon. However, these additional variables are unlikely to sufficiently explain differences found in SSI from an open versus laparoscopic surgical approach. Although the data were derived from colon procedures performed from 2009 to 2013, the fundamental differences between laparoscopic and open procedures and the respective patient populations are unlikely to have changed significantly. In addition, the sensitivity and predictive value of claims codes for identification of SSI surveillance have been nationally validated for use as an effective ranking approach which allows for large populations to be evaluated and compared.
In conclusion, current CMS SSI hospital rankings fail to adequately account for key patient and surgical differences that would greatly enhance fair interhospital comparisons. These rankings currently impact financial disincentives applied to hospitals. CMS SSI adjustment models and hospital rankings should be generated separately for open and laparoscopic procedures and should additionally account for comorbidity scores and concomitant intraabdominal procedures that are readily available from routinely collected administrative data.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2022.153
Acknowledgments
We thank Mr. Abinav Gowda his help with project management and coordination.
Financial support
This study was supported by the Agency for Healthcare Research and Quality (grant no. 5R18HS021424-05).
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
Open access
