Skip to main content Accessibility help

Prediction of Recurrent Clostridium Difficile Infection Using Comprehensive Electronic Medical Records in an Integrated Healthcare Delivery System

  • Gabriel J. Escobar (a1), Jennifer M. Baker (a2), Patricia Kipnis (a1) (a3), John D. Greene (a1), T. Christopher Mast (a4), Swati B. Gupta (a5), Nicole Cossrow (a4), Vinay Mehta (a4), Vincent Liu (a1) (a6) and Erik R. Dubberke (a7)...



Predicting recurrent Clostridium difficile infection (rCDI) remains difficult. METHODS. We employed a retrospective cohort design. Granular electronic medical record (EMR) data had been collected from patients hospitalized at 21 Kaiser Permanente Northern California hospitals. The derivation dataset (2007–2013) included data from 9,386 patients who experienced incident CDI (iCDI) and 1,311 who experienced their first CDI recurrences (rCDI). The validation dataset (2014) included data from 1,865 patients who experienced incident CDI and 144 who experienced rCDI. Using multiple techniques, including machine learning, we evaluated more than 150 potential predictors. Our final analyses evaluated 3 models with varying degrees of complexity and 1 previously published model.


Despite having a large multicenter cohort and access to granular EMR data (eg, vital signs, and laboratory test results), none of the models discriminated well (c statistics, 0.591–0.605), had good calibration, or had good explanatory power.


Our ability to predict rCDI remains limited. Given currently available EMR technology, improvements in prediction will require incorporating new variables because currently available data elements lack adequate explanatory power.

Infect Control Hosp Epidemiol 2017;38:1196–1203


Corresponding author

Address correspondence to Gabriel J. Escobar, MD, Systems Research Initiative, Kaiser Permanente Division of Research, 2000 Broadway Ave (032 R01), Oakland, CA 94612-2304 ( or John Greene, MA, Systems Research Initiative, Kaiser Permanente Northern California Division of Research, 2000 Broadway Ave, Oakland, CA 94612 (


Hide All
1. Lessa, FC, Winston, LG, McDonald, LC. Burden of Clostridium difficile infection in the United States. N Engl J Med 2015;372:23692370.
2. Kwon, JH, Olsen, MA, Dubberke, ER. The morbidity, mortality, and costs associated with Clostridium difficile infection. Infect Dis Clin North Am 2015;29:123134.
3. Olsen, MA, Young-Xu, Y, Stwalley, D, et al. The burden of Clostridium difficile infection: estimates of the incidence of CDI from US administrative databases. BMC Infect Dis 2016;16:177.
4. Freedberg, DE, Salmasian, H, Cohen, B, Abrams, JA, Larson, EL. Receipt of antibiotics in hospitalized patients and risk for Clostridium difficile infection in subsequent patients who occupy the same bed. JAMA Intern Med 2016;176:18011808.
5. McFarland, LV. Renewed interest in a difficult disease: Clostridium difficile infections—epidemiology and current treatment strategies. Curr Opin Gastroenterol 2009;25:2435.
6. Bouza, E. Consequences of Clostridium difficile infection: understanding the healthcare burden. Clin Microbiol Infect 2012;18(Suppl 6):512.
7. McFarland, LV, Elmer, GW, Surawicz, CM. Breaking the cycle: treatment strategies for 163 cases of recurrent Clostridium difficile disease. Am J Gastroenterol 2002;97:17691775.
8. Zilberberg, MD, Reske, K, Olsen, M, Yan, Y, Dubberke, ER. Risk factors for recurrent Clostridium difficile infection (CDI) hospitalization among hospitalized patients with an initial CDI episode: a retrospective cohort study. BMC Infect Dis 2014;14:306.
9. Sheitoyan-Pesant, C, Abou Chakra, CN, Pepin, J, Marcil-Heguy, A, Nault, V, Valiquette, L. Clinical and healthcare burden of multiple recurrences of Clostridium difficile infection. Clin Infect Dis 2016;62:574580.
10. Hu, MY, Katchar, K, Kyne, L, et al. Prospective derivation and validation of a clinical prediction rule for recurrent Clostridium difficile infection. Gastroenterology 2009;136:12061214.
11. Eyre, DW, Walker, AS, Wyllie, D, et al. Predictors of first recurrence of Clostridium difficile infection: implications for initial management. Clin Infect Dis 2012;55:S77S87.
12. Hebert, C, Du, H, Peterson, LR, Robicsek, A. Electronic health record–based detection of risk factors for Clostridium difficile infection relapse. Infect Control Hosp Epidemiol 2013;34:407414.
13. D’Agostino, RB Sr., Collins, SH, Pencina, KM, Kean, Y, Gorbach, S. Risk estimation for recurrent Clostridium difficile infection based on clinical factors. Clin Infect Dis 2014;58:13861393.
14. Kollef, MH, Chen, Y, Heard, K, et al. A randomized trial of real-time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med 2014;9:424429.
15. Escobar, GJ, Dellinger, RP. Early detection, prevention, and mitigation of critical illness outside intensive care settings. J Hosp Med 2016;11:S5S10.
16. Escobar, GJ, Turk, BJ, Ragins, A, et al. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med 2016;11:S18S24.
17. Watt, M, Dinh, A, Le Monnier, A, Tilleul, P. Cost-effectiveness analysis on the use of fidaxomicin and vancomycin to treat Clostridium difficile infection in France. J Med Econ 2017;20:678686.
18. Nelson, RL, Suda, KJ, Evans, CT. Antibiotic treatment for Clostridium difficile–associated diarrhoea in adults. Cochrane Database Syst Rev 2017;3:CD004610.
19. Wilcox, MH, Gerding, DN, Poxton, IR, et al. Bezlotoxumab for prevention of recurrent Clostridium difficile infection. N Eng. J Med 2017;376:305317.
20. Escobar, GJ, Greene, JD, Gardner, MN, Marelich, GP, Quick, B, Kipnis, P. Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med 2011;6:7480.
21. Liu, V, Kipnis, P, Rizk, NW, Escobar, GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med 2012;7:224230.
22. Escobar, GJ, Gardner, MN, Greene, JD, Draper, D, Kipnis, P. Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care 2013;51:446453.
23. Selby, JV. Linking automated databases for research in managed care settings. Ann Intern Med 1997;127:719724.
24. Deyo, RA, Cherkin, DC, Ciol, MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45:613619.
25. Zilberberg, MD, Reske, K, Olsen, M, Yan, Y, Dubberke, ER. Development and validation of a recurrent Clostridium difficile risk-prediction model. J Hosp Med 2014;9:418423.
26. Dubberke, ER, Reske, KA, Yan, Y, Olsen, MA, McDonald, LC, Fraser, VJ. Clostridium difficile–associated disease in a setting of endemicity: identification of novel risk factors. Clin Infect Dis 2007;45:15431549.
27. Dubberke, ER, Yan, Y, Reske, KA, et al. Development and validation of a Clostridium difficile infection risk prediction model. Infect Control Hosp Epidemiol 2011;32:360366.
28. Hastie, T, Tibshirani, R, Friedman, JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York: Springer Verlag; 2009.
29. Allison, PD. Logistic Regression Using SAS: Theory and Application. 2nd ed. Cary, NC: SAS Institute; 2012.
30. Hastie, T, Tibshirani, R, Friedman, J, Franklin, J. The elements of statistical learning: data mining, inference and prediction. Mathemat Intelligenc 2005;27:8385.
31. Hosmer, DW, Lemeshow, S. Applied Survival Analysis: Regression Modelling of Time to Event Data. Hoboken, NJ: Wiley; 2008.
32. Cook, DA, Duke, G, Hart, GK, Pilcher, D, Mullany, D. Review of the application of risk-adjusted charts to analyse mortality outcomes in critical care. Crit Care Resusc 2008;10:239251.
33. Crowson, CS, Atkinson, EJ, Therneau, TM. Assessing calibration of prognostic risk scores. Stat Method Med Res 2014;25:16921706.
34. Cook, NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007;115:928935.
35. Pencina, MJ, D’Agostino, RB Sr, D’Agostino, RB Jr, Vasan, RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157172; discussion 207–212.
36. Estrella, A, Mishkin, FS. Predicting US recessions: financial variables as leading indicators. Rev Econ Statist 1998;80:4561.
37. McDonald, EG, Milligan, J, Frenette, C, Lee, TC. Continuous proton pump inhibitor therapy and the associated risk of recurrent Clostridium difficile Infection. JAMA Intern Med 2015;175:784791.
38. Deshpande, A, Pasupuleti, V, Thota, P, et al. Risk factors for recurrent Clostridium difficile infection: a systematic review and meta-analysis. Infect Control Hosp Epidemiol 2015;36:452460.
39. Kuntz, JL, Johnson, ES, Raebel, MA, et al. Predicting the risk of Clostridium difficile infection following an outpatient visit: development and external validation of a pragmatic, prognostic risk score. Clin Microbiol Infect 2015;21:256262.
40. Kuntz, JL, Smith, DH, Petrik, AF, et al. Predicting the risk of Clostridium difficile infection upon admission: a score to identify patients for antimicrobial stewardship efforts. Perm J 2016;20:2025.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Infection Control & Hospital Epidemiology
  • ISSN: 0899-823X
  • EISSN: 1559-6834
  • URL: /core/journals/infection-control-and-hospital-epidemiology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
Type Description Title
Supplementary materials

Escobar et al supplementary material

 Word (251 KB)
251 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed