Introduction
Nosocomial transmission of tuberculosis (TB) gained national attention in the United States during the 1980s in the context of rising incidence, fueled by the surging HIV/AIDS epidemic and weakened public health infrastructure. Incidence of TB declined from 10.4 per 100,000 in 1992 to a nadir of 2.2 per 100,000 in 2020 and nosocomial transmission has been rare in the US in recent years. Reference Collins and Blumberg1,2 However, incidence has increased to 2.9 cases per 100,000 in the US and 3.2 per 100,000 in Massachusetts in 2023. Reference Williams, Pratt, Walker, Price, Stewart and Feng3,4 At the same time, many hospitals face staffing and capacity challenges, making resource-efficient decision-making for patients with suspected TB more urgent.
Centers for Disease Control and Prevention (CDC) guidelines for the evaluation of persons with suspected pulmonary TB, recommend collecting three sputum samples for acid-fast bacilli (AFB) smear and mycobacterial culture 8–24 hours apart, while implementing airborne infection isolation (AII). Reference Jensen, Lambert, Lademarco and Ridzon5 Approximately half of persons with pulmonary TB have negative sputum smears, and cultures usually take at least two weeks to turn positive. Reference Dugdale, Rubins and Lee6,Reference Lewinsohn, Leonard and LoBue7 Nucleic acid amplification tests (NAAT) for TB have higher sensitivity than AFB smears and have been incorporated into national guidelines for the diagnosis of TB. Reference Lewinsohn, Leonard and LoBue7 However, the sensitivity of NAAT remains significantly lower than that of culture, particularly when the specimen is AFB smear-negative. Reference Lombardi, Di Gregori, Girometti, Tadolini, Bisognin and Dal Monte8,Reference Ross, Anwar and Wickremasinghe9 Therefore, reflexive discontinuation of AII, based on AFB smear and NAAT results alone, may be premature and risk transmission to other patients and to healthcare personnel.
Given the risk of premature discontinuation of AII, healthcare facilities often require additional review of patient records by infection prevention and control (IPC) staff, including infection preventionists (IPs) and physicians, before discontinuation of AII. In addition to reviewing the medical record, IPC staff may discuss the patient’s case with the primary team and/or relevant consultants (eg, infectious diseases, pulmonary medicine) to determine whether TB is sufficiently unlikely to warrant discontinuation of AII. These case-by-case determinations are labor- and time intensive.
Clinical decision support systems (CDSS) have been used to improve patient care through supporting clinicians in the implementation of best practices and clinical guidelines. The objective of this study was to develop and validate a CDSS that could be embedded in the electronic health record (EHR) to support discontinuation of AII when appropriate and, for patients determined to be at higher risk of TB, collect and summarize relevant data to support individual evaluation by IPC staff, thereby enabling efficient case review.
Methods
Developing the model framework
At Massachusetts General Hospital (MGH), a large academic medical center in the Mass General Brigham (MGB) healthcare system, individuals being evaluated for TB who have negative sputum AFB smears and PCRs are evaluated by IPC staff on a case-by-case basis to determine whether TB is sufficiently improbable to favor AII discontinuation. TB evaluation approaches at MGB have been previously described. Reference Dugdale, Zachary and McEvoy10 Individuals undergoing evaluation for TB are given a “TB-Risk” infection status in the EHR and AII is maintained. We constructed a CDSS to expedite AII discontinuation while preserving robust sensitivity to maintain AII in the setting of possible AFB- and NAAT-negative TB. We assembled a team of content matter experts, including IPs and infectious disease physicians with extensive experience in IPC and in TB management. Informed by published epidemiologic data and clinical guidelines, questions were drafted, with an associated scoring system initially proposed by the project lead (KCZ). These questions and the associated scoring system were then iteratively revised by Delphi method consensus and by live application of the draft tool toward test cases of selected hospitalized patients evaluated for TB at MGH from January 1, 2023, to June 30, 2024.
Case–control CDSS validation substudy
To validate the CDSS, we first replicated the CDSS questions and associated scoring system in a REDCap form. Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde11,Reference Harris, Taylor and Minor12 We queried the MGB electronic data warehouse, spanning eight MGB acute care facilities in multiple cities, Reference Dugdale, Zachary and McEvoy10 to identify 104 individuals hospitalized with TB and 104 hospitalized age- and date of test-matched controls with at least three negative mycobacterial cultures from sputum specimens obtained within a 7-day period between July 2016 and December 2022. We then applied the CDSS framework retrospectively to these cases and controls through chart abstraction into REDCap. Clinical characteristics of the 104 hospitalized individuals with TB have been published previously. Reference Dugdale, Zachary and McEvoy10 We generated summary statistics for all components of the CDSS. We identified potential predictors of TB by calculating unadjusted odds ratios with complete case analysis and using a chi-square test for comparison with p-values <0.05 considered statistically significant. Statistical analyses were conducted in Stata v17 (College Station, TX).
Refining the model
Variables from the case–control study found to be negatively associated with TB case prediction, or that did not contribute to the predictive ability of the tool, were eliminated from the CDSS or were preserved with downweighted scores. We iteratively assessed the impact of upweighting the scores of predictors of TB case status that were identified in the case–control analysis. With each alteration in the model, we assessed sensitivity, specificity, and area under the curve (AUC) of the CDSS scoring system. We sought to develop a model that demonstrated 100% sensitivity to detect culture-positive TB while also maximizing specificity, maintaining consistency with clinical judgment, and facilitating collection of details relevant to individualized IPC review. We also conducted a subgroup analysis looking at model performance pre-COVID (2016–2019) and post-COVID (2020–2022).
Estimating the impact on IPC workload
To estimate the potential impact of the tool on IPC person-time at MGH, we multiplied the specificity of the tool (ie, the proportion of individuals with suspected TB whose isolation precautions would be correctly, automatically removed by the tool without IPC review) by the number of individuals with suspected TB hospitalized at MGH annually (ie, >300). We then applied a conservative estimate of 30 minutes of IPC person-time required for manual review per patient evaluated for TB to estimate overall IPC person-time saved with the tool annually.
Human subjects
The study was approved by the MGB Institutional Review Board (2012P002359).
Results
Achieving Delphi consensus for the initial model
We drafted a clinical scoring system incorporating epidemiological characteristics, medical conditions known to increase the risk of TB, symptoms, and the results of diagnostic testing, including tuberculin skin tests (TSTs), interferon-gamma release assays (IGRAs), lower respiratory tract specimens (AFB smear and NAAT), and chest imaging (Figure 1, Table 1). Reference Jensen, Lambert, Lademarco and Ridzon5,Reference Lewinsohn, Leonard and LoBue7 Each component of the scoring system (ie, chest imaging, epidemiology, history of active or latent TB, past medical history, symptoms, and sputum/bronchoscopy results) was given a minimum and maximum score; scores were summed across components to produce a total score.
Figure 1. Diagram of the TBorNotTB CDSS development process. Steps in the design, development, and validation of a complex Smartform-based clinical decision support system (CDSS) to guide clinicians through the diagnostic evaluation of hospitalized individuals with suspected pulmonary TB in low prevalence settings.
Table 1. Components of the final (Revised) TBorNotTB CDSS scoring system
TB: TB; IGRA: interferon gamma release assay; AFB: acid fast bacilli; PCR: polymerase chain reaction; CT: computed tomography; CXR: chest radiograph.
1 The score range reflects the absolute minimum and absolute maximum points allowed for the section; scores outside of this range were brought to their nearest bound for the calculation of total score.
2 The CDSS requires some form of chest imaging in order to complete the form. Results of chest computed tomography (CT) are prioritized over chest radiography.
3 There is a maximum cumulative score of 3 points for these particular questions regarding epidemiology.
4 Conditions that meet the MGB definition of immunocompromising state have been published previously. Reference Dugdale, Zachary and McEvoy10
5 The patient is only given this negative score if they are not immunocompromised.
6 Medications active against TB included: Rifampin, isoniazid, pyrazinamide, ethambutol, linezolid, tedizolid, amikacin, streptomycin, bedaquiline, clofazimine, pretomanid, rifabutin, rifapentine, levofloxacin, ciprofloxacin, and moxifloxacin.
During the Delphi process, infectious disease physicians provided insights on the consistency of the CDSS with clinical judgment. IPC staff provided guidance regarding elements of the patient’s history often lacking in the medical record at the time of individualized assessment to ensure collection of these details by the tool. MGB Digital Health information technology specialists contributed expertise around structured data collection and technical constraints.
Improving usability of the initial model
Usability of the tool, focused on minimizing redundancy and maximizing clarity, was prioritized. Questions in the tool were reordered such that if the user indicated that there was no available chest radiology result at the time of CDSS activation, it would advise obtaining such imaging and returning to the tool without requiring the user to respond to the other questions. Branching logic was employed to display only questions that followed logically from previous responses; help text was added for clarity. For example, when asking about whether an individual’s symptoms have resolved or markedly improved with treatment for an alternative diagnosis and without any medications active against TB, a list of specific medications active against TB was added. Finally, the group reviewed the tool to ensure person-first language, formatting of epidemiologic questions to minimize stigma, and consideration of equity in the application of the tool to individuals from diverse backgrounds.
Predictors of TB disease
Several predictors of TB infection were identified in the case–control analysis (Table 2). First, presence of a cavitary lesion or other findings specifically described as suspicious for TB on the radiology report was strongly associated with active TB as compared with the absence of these findings (chest radiograph [CXR]: odds ratio [OR]: 4.0; 95% confidence interval [CI]: 1.5, 11.5; chest computed tomography [CT]: OR: 3.3; 95% CI: 1.7, 6.2; P < 0.01). Prior residence in a highly TB endemic country (defined as anywhere outside of the United States, Canada, Western Europe, Australia, or New Zealand) was the strongest epidemiologic risk factor for TB identified (OR: 5.7; 95% CI: 2.6, 12.8; P < 0.01). Other traditional epidemiologic risk factors for TB were not significantly predictive, although our study was underpowered for such analysis (Table 2).
Table 2. Predictors of pulmonary TB infection in CDSS validation study
OR: odds ratio; TB: TB, IGRA: interferon gamma release assay; TST: tuberculin skin test; CXR: Chest X-ray; CT: computed tomography.
Having a history of a prior diagnosis of active TB was uncommon among individuals in our study (controls: 1/104 [1%], cases: 5/104 [5%]), and we lacked power to detect a statistically significant difference in prior TB diagnosis (P = 0.10). History of latent TB was much more common (controls: 18/104 [17%], cases: 24/104 [23%]), but there was no statistically significant difference between groups (P = 0.30). Having a history of a positive IGRA result was a strong predictor of having active TB (OR: 6.7; 95% CI: 3.4, 13.7; P <0.01). Although IGRA and TST are not required for active TB evaluation, having a recent negative IGRA or TST within the past 30-days was negatively associated with TB (OR: 0.3; 95% CI: 0.1, 0.6; P <0.01). However, 14 of 104 individuals (14%) with culture-confirmed TB had a negative IGRA or TST within the 30 days prior to their TB diagnosis.
When assessed both individually and collectively, having a past medical history of conditions considered to increase the risk for TB were not associated with an increased odds of TB in our study. Reference Lewinsohn, Leonard and LoBue7 Immunocompromised status was more common among controls than among cases (OR: 0.5; 95% CI: 0.3, 1.0; P = 0.04), as was active head and neck cancer (OR: 0.0; 0.0, 0.74; P = 0.02). Similarly, except for a history of weight loss (OR: 2.3; 95% CI: 1.3, 4.2; P <0.01), no traditional TB symptoms were found to be predictors of active TB. Resolution or marked improvement of symptoms with treatment for an alternative diagnosis was strongly negatively associated with TB (OR: 0.0; 95% CI: 0.0, 0.1; P < 0.01).
Model refinement and performance
The initial model as designed through Delphi consensus demonstrated 100% sensitivity and 16% specificity with an AUC of 0.83. We made iterative revisions to the scoring system based on the results of the case–control study. First, given the lack of association between past medical history and active TB in our study, points for past medical history were removed; these questions were still preserved to aid in individualized review by IPC staff. Given the strong associations with the presence of active TB, scores for prior residence in a TB endemic country, weight loss, and substantial improvement in TB symptoms with treatment for an alternative diagnosis were upweighted. Questions regarding non-residence epidemiologic risk factors were downweighted. The overall subsection maximum score for epidemiology risk factors was also increased. Scores were higher for cases (median: 16, interquartile range [IQR]: 10, 20) than for controls (median: 4, IQR: 0, 9) in the final model (P < 0.01; Figure 2). The final model demonstrated 100% sensitivity and 27% specificity with an AUC of 0.87. Model performance was similar pre-COVID (ie, 100% sensitivity, 31% specificity, and AUC 0.93) and post-COVID (ie, 100% sensitivity, 23% specificity, and AUC 0.82).
Figure 2. Distribution of scores between cases and controls in the final model. A histogram demonstrating the counts of total scores for controls (in red) and cases (in blue) is shown. Scores for controls ranged from −6 to 20. Scores for cases ranged from 2 to 24. The median values for total scores for controls (in red dashed lines) and cases (in blue dashed lines) are shown. Areas shaded in purple reflect overlap between the distributions of total score for cases and controls.
Coding and implementation
A SmartForm-based tool (ie, advanced note template) was developed in the electronic health record Epic™ (Epic Systems Corporation, Verona, WI; Figure 3). During the build phase, the tool was accessed in a support environment (ie, an environment that contains patient data but does not write to the medical record) by the team of infectious disease and IPC experts to evaluate the CDSS usability with a variety of test cases. Based on these tests, further refinements were made to optimize frontline clinician understanding and user experience. In the live version of the tool, when the clinician completes the SmartForm, a note containing the information entered is automatically populated into the patient’s chart.
Figure 3. Screenshot of TBorNotTB CDSS in the Epic electronic medical record. From the top left, TBorNotTB guides clinicians to enter relevant recent radiology results. If no chest imaging is available (computed tomography or radiography), the clinician cannot proceed with using the tool and is advised of the requirement for radiological assessment. If chest imaging is available, the clinician can advance to document relevant epidemiology, followed by history of active or latent TB, then past medical history, symptoms, and finally sputum/bronchoscopy results. Using branching logic, questions are either hidden or appear based on the answers provided (eg if the answer is “No” to “Has the patient ever had an IGRA or TST,” the subsequent questions about results will not be revealed). Though not visible to the user of the tool, a weighted risk score is calculated in the background. If a final weighted score is below the established threshold, the TBorNotTB CDSS notifies the clinician using the tool that they can remove airborne infection isolation (AII). The tool also automatically removes the “TB-Risk” identifier on the patient’s chart upon signing the note. If, however, the risk score is above the threshold, TBorNotTB provides local site contact information for infection control to review the case further. If there is missing testing, TBorNotTB may also suggest further testing. Example screenshot from Epic™ (© 2024 Epic Systems Corporation). Video demonstration (mp4) of two patient scenarios using TBorNotTB CDSS is available in supplemental material, accessible at journals.cambridge.org.
Projected impact on IPC person-time
We estimated that use of the CDSS would replace >80 manual reviews conducted by IPC personnel each year, saving >40 IPC person-hours of time annually at MGH.
Discussion
We developed and validated a novel real-time CDSS (TBorNotTB) to guide clinicians through the diagnostic evaluation of hospitalized individuals with suspected pulmonary TB in low-prevalence settings. We found that prior residence in a TB endemic country, positive IGRA, weight loss, continuation of symptoms despite treatment for alternative diagnoses, and findings concerning for TB on chest imaging were significant predictors of TB. CDSS contents and scoring were iteratively refined based on Delphi consensus and the case–control analysis. The revised CDSS demonstrated 100% sensitivity and 27% specificity to detect TB among hospitalized patients in a low-prevalence setting.
TB incidence in the US is on the rise. Concurrently, many hospitals across the US are experiencing capacity crises, making the expeditious evaluation of persons with possible infectious pulmonary TB even more urgent to preserve limited AII rooms. To provide guideline-recommended evaluation, optimize facility resources, and mitigate risk of nosocomial transmission, it is essential to determine when TB has been excluded with a high degree of confidence. AFB smears of respiratory specimens have limited sensitivity, even when multiple samples are submitted. NAATs improve sensitivity, but some individuals with culture-positive TB still have negative NAATs. Discontinuation of AII for TB often requires review by IPC personnel, but many facilities do not have adequate IPC personnel. Reference Clifford, Newhart, Laguio-Vila, Gutowski, Bronstein and Lesho13,Reference Reese, Gilmartin and Smathers14 Individualized review of patients with suspected TB, as well as contact tracing in the setting of nosocomial exposures, are very time- and resource-intensive endeavors for an already overstretched IP workforce. Reference Clifford, Newhart, Laguio-Vila, Gutowski, Bronstein and Lesho13–Reference Kelly, D’Agostino, Andrada, Liu and Larson15
CDSS like TBorNotTB can support infection control activities by serving a range of functions, including packaging data for public health reporting, swiftly isolating and efficiently deisolating patients, triaging patients for testing and/or prophylactic treatment, and reducing cognitive and documentation burden for frontline clinicians. CDSS for infection control can also optimize hospital bed use and other resources. CDSS tools have been developed and deployed to support timely and appropriate deisolation for COVID-19 and suspected COVID-19, Reference Dugdale, Rubins and Lee6,Reference Lupei, Li and Ingraham16 to support test-based clearance for MRSA and VRE, Reference Paras, Craig, Germaine, Green, Vernick, Ha and Shenoy17 to screen patients for Ebola virus disease based on travel history and public health advisory, Reference Landman, Goralnick and Teich18 and to evaluate and recommend isolation, testing, and prophylactic treatment for patients with suspected mpox. Reference Lazarus, Green and Jerry19,Reference Albin, Lazarus and Hysell20 Unlike paper-based tools, digital EHR-based tools to facilitate guideline-based infection control practice can be updated in real time to align with evolving science and policy, including in the context of emerging infectious diseases.
Several clinical prediction rules and scores are already in use to improve the detection of TB in a range of contexts and populations. Risk scores have been developed, for example, to stratify risk for adult contacts of index TB cases in Peru and for TB in smokers in Malaysia. Reference Saunders, Wingfield and Tovar21,Reference Tengku Khalid, Wan Mohammad, Nik Husain and Ab Samat22 Clinical risk prediction models have also been developed to predict pulmonary TB among individuals with suspected TB in Ethiopia, to predict bacteriologically confirmed TB in adults receiving antiretroviral therapy for HIV in Ethiopia, and to improve specificity of TB diagnosis among children living with HIV in Vietnam, Cameroon, and South Africa. Reference Wolde, Clements and Alene23–Reference Malik, Gandhi and Marcy25 While many other clinical decision support tools have been developed to direct TB treatment and testing, the TBorNotTB tool is the first to be leveraged to support IPC within healthcare settings.
The tools developed in our study and others highlight that CDSS is a powerful and flexible approach to address changing circumstances and novel science and technology. CDSS can be adapted to address different aspects of care in a standardized way, incorporating new epidemiological or other data, as well as advancements in laboratory methodologies over time. Healthcare facilities can adapt such tools to their local procedures or laboratory capacity, such as incorporating different testing protocols for evaluating suspected TB; some facilities require three negative AFB smears prior to removal of isolation precautions whereas others rely more heavily on NAAT-based testing. Reference Lewinsohn, Leonard and LoBue7,Reference Dugdale, Zachary and McEvoy10,26 Another strength of CDSS is that the tool itself can collect and organize data that can be analyzed for future updates to structure or scoring to facilitate quality improvement.
Our analysis has several limitations. First, the CDSS test characteristics were assessed based on data from a single healthcare system in a high-income, low-prevalence setting. Epidemiologic risk factors and clinical characteristics among individuals with TB may differ in other low-prevalence cities or in high-prevalence settings, which could alter the performance of the tool. For example, in our setting some chronic medical conditions previously seen to increase the risk for TB disease such as diabetes and cirrhosis were not associated with TB in our population. Reference Lin, Wu and Lin27,Reference Yoo, Kim, Han, Rhee, Shin and Lee28 These findings likely reflect enrichment for the presence of medical conditions in the control group that would lead clinicians to pursue three diagnostic AFB smears despite low to moderate pretest probability for TB. In these situations, evaluation for TB may have been pursued given the diagnostic uncertainties related to and clinical consequences of respiratory infection in the setting of immunocompromise or underlying lung disease, for example. Second, the CDSS validation dataset was derived from retrospective chart review, so there were some instances in which the EHR lacked sufficient detail to answer all questions. To address this, prospective assessment of the post-implementation performance of the CDSS, including its influence on duration of patient isolation, is planned. Third, the CDSS was only validated for use among hospitalized individuals undergoing evaluation for TB and not for ambulatory persons, although about half of all patients diagnosed with TB in the US have an initial hospitalization. Reference Taylor, Marks, Ríos Burrows, Weis, Stricof and Miller29 Future analyses assessing application in the ambulatory population are needed.
In conclusion, we developed a clinical decision support system for the evaluation of suspected TB in a low-prevalence setting that demonstrated excellent sensitivity and modest specificity. This TBorNotTB CDSS is unique in its focus and application. Embedded into the electronic medical record system, this CDSS could help reduce both the risks of nosocomial TB transmission and the IPC person-time spent reviewing individuals with suspected TB.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2024.214
Acknowledgments
The authors wish to thank Jasmine B Ha, MS, MGB Digital Health, for project management support.
Financial Support
This work was supported by U.S. Centers for Disease Control and Prevention (NU50CK000622 [ESS]), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K08 HD101342 [CMD]), and the Massachusetts General Hospital Fund for Medical Discovery (CMD). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the funders.
Competing interests
The authors have no conflicts of interest to declare.
Open access
