CLINICIAN’S CAPSULE
What is known about the topic?
Sex-specific, high-sensitivity cardiac troponin T (hs-cTnT) cut-offs increase the specificity of a myocardial infarction (MI) diagnosis.
What did this study ask?
Do sex-specific, hs-cTnT rule-out cut-offs enable ruling out MI in more patients while maintaining sensitivity?
What did this study find?
Sex-specific, hs-cTnT cut-offs ruled out MI in more patients than universal cut-offs; however, differences between sex-specific and universal cut-offs are small.
Why does this study matter to clinicians?
Sex-specific, rule-out, hs-cTnT cut-offs may enable more patients to be ruled out after a single hs-cTnT measurement.
INTRODUCTION
Patients with potential acute coronary syndrome (ACS) account for up to 6% of emergency department (ED) presentationsReference Goodacre, Cross and Arnold1, Reference Buhuiya, Pitts and McCaig2 and 25% of admissions.Reference Goodacre, Cross and Arnold1 In ED patients with chest pain, a high-sensitivity cardiac troponin T (hs-cTnT) result below the assay’s limit of detection (LoD,<5 ng/L) at the time of ED arrival can rule out acute myocardial infarction (MI) with high sensitivity and NPV.Reference Zhelev, Hyde and Youngman3, Reference Pickering, Than and Cullen4 The 2015 European Society for Cardiology guidelines for non-ST-elevation ACS state that MI can be ruled out of patients with an initial hs-cTnT concentration below 5 ng/L taken, provided that the hs-cTnT measurement is performed more than 3 hours after symptom onset.Reference Roffi, Patrono and Collet5 The limit of quantitation approved by the U.S. Food and Drug Administration (FDA), 6 ng/L, can achieve similar high diagnostic sensitivity for MI.Reference McRae, Innes and Graham6
Sex-specific diagnostic cut-offs for high-sensitivity troponin assays have been proposed for ruling-in MI.Reference Rubini Giménez, Twerenbold and Boeddinghaus7–Reference Mueller and Kavsak10 In the rule-in scenario, sex-specific cut-offs improve diagnostic specificity and classification performance.Reference Rubini Giménez, Twerenbold and Boeddinghaus7–Reference Mueller and Kavsak10 We hypothesized that sex-specific rule-out cut-offs would improve the classification performance of a single hs-cTnT measurement performed at ED arrival for rapidly ruling out MI with negligible loss of sensitivity. This would enable more patients to have MI safely ruled out after a single hs-cTnT measurement.
The objective of this study was to quantify the diagnostic performance of very low concentrations of hs-cTnT drawn at the time of ED arrival in male and female chest pain cohorts. The performance of several candidate rule-out cut-offs was quantified in males and females, and a net reclassification index calculated relative to both the manufacturer’s stated LoD (<5 ng/L) and the FDA-approved limit of quantitation (<6 ng/L).
METHODS
This is a pre-planned secondary analysis of an observational cohort study designed to quantify test characteristics of various rapid diagnostic pathways using an hs-cTnT assay. Previously published studies using this cohort have validated the test characteristics of an undetectable hs-cTnT concentration at ED arrival as a universal (i.e., not sex-specific) cut-off to rule out acute MI.Reference McRae, Innes and Graham6 A secondary analysis was performed on a subsample of 722 patients with hs-cTnT concentrations measured at 2-hour intervals to validate published 2-hour rapid diagnostic algorithms.Reference McRae, Innes and Graham11
Setting
This observational study analysed 1 year of administrative and registry data from four adult EDs in Calgary, Alberta (population 1.2 million). These four sites have a combined annual ED census of approximately 305,000 visits, including approximately 12,000 visits with a Canadian Emergency Department Information Systems presenting complaint of “Chest Pain – Cardiac Features.” One hospital is the regional percutaneous coronary intervention site, whereas the other three have coronary care units. These hospitals share a common, linked ED information system and administrative database. Patients with suspected cardiac chest pain identified at triage have blood work, including hs-cTnT, drawn at the time of ED arrival according to a nurse-initiated diagnostic protocol.
All four sites use a Roche Elecsys® high-sensitivity, fifth generation, cardiac troponin T assay performed on the cobas e601 instrument. This assay has a limit of blank of 3 ng/L, an LoD of 5 ng/L, and a 99th percentile of 14 ng/L in a healthy population. The assay is run on eight separate instruments across the four hospitals. Patients with hemolyzed initial hs-cTnT samples were excluded, because standard practice is to redraw hemolyzed hs-cTnT samples. Local practice recommendations considered MI ruled out if a patient’s hs-cTnT concentration was less than 14 ng/L when measured more than 6 hours after onset of the patient’s most significant symptoms.Reference Crowder, Jones and Lang12
Patients
The study included patients age 18 years or older who presented to the participating EDs between January 1 and December 31, 2013, with a standardized triage code of “Chest Pain – Cardiac Features” or “Cardiac Type Pain” (epigastric, neck, jaw, or arm pain concerning for angina) assigned by ED triage nursing staff, and who underwent hs-cTnT testing as part of the nurse-initiated protocol within 60 minutes of ED arrival. The 60-minute window was chosen to capture patients who would have had the hs-cTnT assay ordered at the time of ED arrival. Patients with ST-elevation MI or cardiac arrest in the ED (identified using freetext ED diagnosis or on further case adjudication), and those with abnormal kidney function (estimated glomerular filtration rate [eGFR]<60 ml/min/1.73 m2 using the CKD-EPI equation) were excluded. The cohort of patients was stratified into separate male and female cohorts.
Data sources
Patient characteristics were extracted from the ED administrative database. Outcome data were obtained by linking the ED and hospital discharge abstract databases, Alberta provincial vital statistics, and the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) registry. The ED administrative databases include electronic time stamps for all clinical encounters, including time of arrival, physician assessment, disposition decisions, and diagnostic and therapeutic interventions.13 APPROACH collects prospective data on all patients admitted with a cardiac diagnosis or who have a cardiac catheterization or revascularization procedure in Alberta.Reference Ghali and Knudtson14 Data sources were linked using a deterministic linkage based on a provincial personal health number, date of birth and date of service, with a linkage success rate greater than 99%.
Outcomes
The primary outcome was the incidence of MI on the index visit or within 7 days of ED arrival. MI diagnosis was made by treating clinicians based on clinical and electrocardiogram features, hs-cTnT results, and results of noninvasive or invasive cardiac investigations.
The MI outcome was ascertained using the International Classification of Diseases, 10th Revision (ICD-10) codes for the primary diagnosis of MI (I21.0-I21.9) from hospital discharge abstract databases or as recorded in multiple fields for diagnosis in the APPROACH registry. Patients whose initial hs-cTnT concentration was less than 15 ng/L, and who had an outcome flagged in either the APPROACH or administrative data, had their outcomes adjudicated using an electronic medical record review by an emergency physician, certified as a Fellow of the Royal College of Physicians of Canada (FRCPC), using the Third Universal Definition of MI criteria.Reference Thygesen, Alpert and Jaffe15
Analysis
Descriptive statistics for the study cohort were generated. Test statistics, including sensitivity, specificity, predictive values, likelihood ratios, and classification performance (percentage of patients ruled out) for hs-cTnT concentrations ranging from 3-10 ng/L, were generated. The net reclassification index was calculated for various combinations of sex-specific hs-cTnT concentrations.Reference Pencina, D’Agostino, D’Agostino and Vasan16, Reference Leening, Vedder and Witteman17 For binary tests, the net reclassification index (ranging in values from −2 to +2) is the sum of the proportional changes in sensitivity and specificity for different test cut-offs. The net reclassification index (event) quantifies the proportional change in sensitivity, whereas the net reclassification index (non-event) quantifies the proportional change in specificity.Reference Leening, Vedder and Witteman17 The reference universal hs-cTnT concentrations for net reclassification index calculation included both the manufacturer’s stated LoD (<5 ng/L) or the FDA-approved limit of quantitation (<6 ng/L). Differences in the proportion of patients ruled out using different cut-offs were compared using Pearson’s chi-square test. We sought to identify sex-specific hs-cTnT cut-off concentrations that permitted the highest proportion of patients to be ruled out while maintaining 98.5% sensitivity, which is a minimally acceptable level previously used and described in the literature.Reference Than, Herbert and Flaws18–Reference Pickering, Flaws and Smith20 Statistical analyses were performed using SAS version 9.2 (SAS Institute, Inc., Cary, NC) and R version 3.0.3 (www.r-project.org).
The study was approved by the University of Calgary Conjoint Health Research Ethics Board without the need for informed consent.
RESULTS
This study included 7,130 patients: 3,931 (55.1%) men and 3,199 (44.9%) women (Figure 1). The incidence of the primary outcome, 7-day diagnosis of MI, was 7.4% among men and 3.4% among women (Table 1). Among all patients, hs-cTnT concentrations lower than the manufacturer’s LoD (5 ng/L) ruled out 7-day MI in 2,399 patients (33.6%) with 99.8% sensitivity (95% CI 98.44-100) (Table 2); hs-cTnT concentrations lower than the FDA-approved limit of quantitation (6 ng/l) ruled out 7-day MI in 3,038 patients (42.6%) with 99.8% sensitivity (95% CI 98.44-99.98) (Table 3). Sex-specific classification performance, sensitivity, and negative likelihood ratios for hs-cTnT concentrations between<3 ng/L and<10 ng/L are shown in Table 4.
Standards for Reporting of Diagnostic Accuracy Studies (STARD) patient flow diagram.
Patient demographics and outcomes
EMS=emergency medical services; IQR=interquartile range; MI=myocardial infarction.
Performance of sex-specific hs-cTnT cut-off combinations for ruling out 7-day MI outcome compared with hs-cTnT concentration<5 ng/L
-LR=Negative Likelihood Ratio; CI=confidence interval; NRI=Net Reclassification Index.
Performance of sex-specific hs-cTnT cut-off combinations for ruling out 7-day MI outcome compared with hs-cTnT concentration<6 ng/L
-LR=Negative Likelihood Ratio; CI=confidence interval; NRI=Net Reclassification Index.
Test characteristics of hs-cTnT concentrations<3 ng/L to<10 ng/L within 60 minutes of ED arrival for ruling out 7-day MI in separate male and female cohorts
CI=confidence interval.
The combination of sex-specific rule-out cut-offs achieving a sensitivity greater than 98.5% for 7-day MI with the greatest proportion of patients ruled out was a cut-off of<8 ng/L for men and<7 ng/L for women. This combination ruled out 7-day MI in 3,979 (55.8%) patients with a sensitivity of 98.5% (95% CI 96.7-99.4), NPV of 99.9% (95% CI 99.7-99.9), and a negative likelihood ratio of 0.025 (95% CI 0.01-0.055). Compared to a universal rule-out cut-off of<5 ng/L, this sex-specific approach had a net reclassification index of 0.222, based on a net reclassification index (event) of −0.012 and a net reclassification index (non-event) of 0.234. This combination of sex-specific, rule-out cut-offs would enable a 22.2% absolute increase in the proportion of patients who are able to be ruled out with a single hs-cTnT measurement compared to a universal rule-out concentration of<5 ng/L (p<0.0001, Table 2). Compared to a universal rule-out cut-off of<6 ng/L, sex-specific, rule-out concentrations of<8 ng/L for men and<7 ng/L for women had a net reclassification index of 0.127 based on a net reclassification index (event) of −0.012 and a net reclassification index (non-event) of 0.139. This combination of sex-specific, rule-out cut-offs would enable a 13.2% absolute increase in the proportion of patients who are able to be ruled out with a single hs-cTnT assay, compared with a universal rule-out concentration of<6 ng/L (p<−0.0001, Table 3).
The combination of sex-specific, rule-out cut-offs achieving a sensitivity greater than 99% for 7-day MI with the greatest proportion of patients ruled out was a cut-off of<6 ng/L for men and<7 ng/L for women. This combination ruled out 7-day MI in 3,317 (46.5%) patients with a sensitivity of 99.5% (95% CI 38.1-99.9), NPV of 99.9% (95% CI 99.8-99.99), and negative likelihood ratio of 0.009 (95% CI 0.003-0.039). Compared with a universal rule-out cut-off of<5 ng/L, this sex-specific approach had a net reclassification index of 0.134, based on a net reclassification index (event) of −0.003 and a net reclassification index (non-event) of 0.137. This combination of sex-specific, rule-out cut-offs would enable a 12.9% absolute increase in the proportion of patients who are able to be ruled out with a single hs-cTnT measurement compared with a universal rule-out concentration of<5 ng/L (p<0.0001, Table 2). Compared to a universal rule-out cut-off of<6 ng/L, sex-specific rule-out concentrations of<6 ng/L for men and<7 ng/L for women had a net reclassification index of 0.039, based on a net reclassification index (event) of −0.003 and a net reclassification index (non-event) of 0.042. This combination of sex-specific, rule-out cut-offs would enable a 3.9% absolute increase in the proportion of patients who are able to be ruled out with a single hs-cTnT assay compared with a universal rule-out concentration of<6 ng/L (p<−0.0001, Table 3).
DISCUSSION
We used observational and registry data to quantify test characteristics and classification performance of sex-specific cut-off concentrations for a single hs-cTnT measurement performed within 60 minutes of ED arrival. Previous work has shown that sex-specific cut-offs for ruling in MI increase classification performance by increasing the specificity of the assay.Reference Rubini Giménez, Twerenbold and Boeddinghaus7–Reference Mueller and Kavsak10 We hypothesized, based on these findings, that similar improvements in classification performance would be observed for sex-specific, rule-out cut-off concentrations, with minimal loss of sensitivity. This would potentially enable more patients to have MI ruled out with a single hs-cTnT measurement compared to currently recommended universal rule-out cut-off concentrations.Reference Zhelev, Hyde and Youngman3–Reference Roffi, Patrono and Collet5
The best-performing combination of sex-specific hs-cTnT rule-out concentrations in this cohort was<8 ng/L for men and<7 ng/L for women. This combination of sex-specific, rule-out cut-offs had a 98.5% sensitivity and ruled out MI in 55.8% of patients, compared with 33.6% for the LoD of the assay (<5 ng/L) and 42.6% for the FDA-mandated limit of quantitation (<6 ng/L). These improvements in classification performance stem from improvements in specificity using sex-specific cut-offs, as indicated by the net reclassification index (non-event) with very little loss in sensitivity, as indicated by the very small negative net reclassification (event). Thus, using sex-specific cut-offs should lead to a 13% to 22% absolute increase in the number of patients who are able to have MI safely ruled out with a single hs-cTnT measurement.
Our data suggest that the specificity gained using sex-specific cut-offs of<8 ng/L for men and<7 ng/L for women comes with a small loss of sensitivity, approximately 1.3% lower than the sensitivity of a universal rule-out cut-off of<5 ng/l. End-users will need to make the judgment whether the value of achieving these gains in specificity offsets the small loss of sensitivity.
This study and its findings have three specific limitations. Firstly, these data are from an observational study examining diagnostic performance of the hs-cTnT assay as used in clinical practice in multiple hospitals. Patients were identified based on the standardized triage complaint, as assigned by a triage nurse. Because patients had an initial troponin drawn as part of a nurse-initiated protocol, it is possible that this cohort had a slightly lower risk profile compared with patients for whom an ED physician is evaluating for a potential ACS. However, the standardized triage complaints used to identify patients correspond to the American Heart Association research definition of potential ACS symptoms,Reference Luepker, Apple and Christenson21 and have been shown to have both construct and outcome validity.Reference Bullard, Thomas and Villa Roel22 Moreover, the patient demographics and 7-day MI incidence are similar to other North American cohorts.Reference Mahler, Riley and Hiestand23, Reference Stopyra, Miller and Hiestand24 Because the inclusion criteria focused on chest pain, these findings may not be generalizable to patients with other symptoms of ACS, such as dyspnea or nausea.
Secondly, outcomes were ascertained using administrative and registry data, based on the diagnosis of MI made clinically by attending physicians. Outcomes were only adjudicated for patients with flagged outcomes and an hs-cTnT<15 ng/L. However, the administrative and registry data used for outcome ascertainment have been shown to be highly reliable for the diagnosis of recent MI when compared with adjudicated data from medical records.Reference Quan, Parsons and Ghali25 Alberta Vital Statistics records all deaths and the APPROACH registry captures all cardiac admissions, cardiac catheterizations, and revascularization procedures in the province of Alberta,Reference Ghali and Knudtson14 minimizing the risk of missed outcomes. Given prior validation of MI diagnosis in these data sources, it is unlikely that these data overestimate the sensitivity and NPV because of false-negative misclassification.Reference Ghali and Knudtson14, Reference Quan, Parsons and Ghali25
Finally, these data suggest classification improvement for ruling out MI by using 1 ng/L to 3 ng/L deviations from the LoD of the hs-cTnT assay. These small differences are within the expected analytic variability of the hs-cTnT assay at low troponin concentrations.Reference Kavsak26–Reference Wu, Christenson and Greene31 The sex-specific, rule-out cut-offs achieving acceptable diagnostic sensitivity (8 ng/L for men, 7 ng/L for women) are sufficiently similar to a universal cut-off of<5 ng/L or<6 ng/L that gains in classification performance and operational efficiency may only be marginal in real-world clinical settings, and indeed misclassification is possible. Thus, we view these findings as hypothesis-generating and encourage other research groups who have examined the single-troponin rule-out strategyReference Zhelev, Hyde and Youngman3, Reference Pickering, Than and Cullen4, Reference MacGougan, Christenson, Innes and Raboud19, Reference Carlton, Khattab and Greaves32 to attempt to reproduce this work in identifying sex-specific, rule-out concentrations that can improve classification efficiency.
CONCLUSIONS
Sex-specific hs-cTnT cut-offs for ruling out MI at the time of ED arrival may offer improved classification performance compared to universal rule-out cut-offs. This improvement is based on a gain in specificity with preserved high diagnostic sensitivity, meaning that the adoption of sex-specific, rule-out cut-offs could rule out MI in a larger proportion of patients than universal cut-offs while preserving acceptable sensitivity for MI. However, the difference in hs-cTnT concentration between universal and sex-specific, rule out cut-offs is small, possibly within the variation range of the assay, potentially limiting real-world clinical impact. We encourage sex-specific analysis of other chest pain cohorts to confirm these findings.
Acknowledgements
The authors gratefully acknowledge the assistance of Katrina Koger and Shabnam Vatanpour in the preparation of this manuscript.
Funding
This study was funded by an operating grant from the Canadian Institutes of Health Research (MOP-130316).
Competing interests
PK reports grants/honorariums/consultant/advisor fees from Abbott Laboratories, Abbott Point of Care, Beckman Coulter, Ortho Clinical Diagnostics, Roche Diagnostics, and Siemens Healthcare Diagnostics with respect to cardiac troponin testing. McMaster University has filed patents with PK as an inventor in the acute cardiac biomarkers area. JA, AM, GI, and EL have received research grants from Roche Diagnostics for an unrelated cardiac troponin study.
