Over the last 20 years, the method of management of coarctation of the aorta has been the subject of debate. This debate continues, as illustrated by the commentary provided by Ebels and his colleaguesReference Ebels, Maruszewski and Blackstone1 in response to our studyReference Wong, Benson, Van Arsdell, Karamlou and McCrindle2 employing decision analysis to evaluate systematically the value-weighted outcomes for a therapeutic strategy based either on surgery or interventional catheterisation. The process of peer review of our submitted manuscript generated a protracted debate regarding the validity of its methodology. The editorial commentary provided by Ebels and his colleagues,Reference Ebels, Maruszewski and Blackstone1 however, continues to demonstrate that there are ongoing misperceptions which require further specific clarification. We submit that the commentators have misunderstood the methodology, and misinterpreted the results of the analysis. We also question their criticisms of the literature reviewed.
In the commentary, there appears to be a misunderstanding as to what a decision analysis is, and what it is not. A decision analysis is not an assessment of real-life decisions, but does provide a framework for modelling and analyzing how such decisions might be made. It also allows us to determine which factors might influence decision making, and to what degree they will alter the final decision. These factors include the spectrum of plausible outcomes, their probability of occurrence, the relative values or weights, called utilities, of those outcomes, together with a relative value of the treatment modality itself. Determination and assignment of utilities by standard gamble interview is a method of estimating a quantitative value for a given outcome or combination of outcomes. In analysis of cost-effectiveness, that value would be the costs associated with that particular treatment and its outcomes. In decision analysis, the utility value represents a trade-off in terms of added mortality. Additionally, a model using decision analysis permits the probability and utility of outcomes to be varied to determine threshold values at which the final weighted utility adjusted score for one treatment might exceed the alternative, a feature known as sensitivity analysis. We infer that the commentators consider that we have been examining real-life decisions, while instead we have performed a systematic modeling of factors that might simulate an informed decision-making process. In real life, decision makers are simultaneously taking into account the relative merits of treatments, the probabilities of possible outcomes, and the equivalency or weight of those outcomes, as well as the preferences of themselves and their patients, but they are doing this largely based on intuition and recall, which is fraught with bias. Decision analysis systematically models the process in a manner that is driven by the data.
“Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research… Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients’ predicaments, rights, and preferences in making clinical decisions about their care”.Reference Sackett, Rosenberg, Gray, Haynes and Richardson3 The strict use of clinical expertise without consideration of available evidence, or reliance solely on the most recent clinical or research evidence without consideration of other mitigating factors, deviates from this definition, and consideration of both is required when making a clinical decision.
Decision analysis provides a methodology for a systematic examination of a clinical problem, and permits incorporation of the best evidence available from research with clinical expertise and patient preferences for alternatives and outcomes (utilities). When applying decision analysis to a specific clinical problem, two criterions are required. First, there should be an uncertainty about a particular clinical strategy for a patient with a given illness and, second, the strategies should contain advantages and countervailing disadvantages, which may differ both quantitatively and qualitatively.Reference Detsky, Naglie, Krahn, Naimark and Redelmeier4 If one strategy clearly dominates due to better outcomes, less risk of adverse effects of treatment, and better utilities, then a decision analysis is not necessary. In this regard, for coarctation of the aorta, it is the differences in the relative advantages and disadvantages of either surgical repair or balloon dilation that makes direct comparison between the two modalities difficult. The risks, complications, and benefits of surgery as opposed to those of catheter-based interventional procedure are qualitatively and quantitatively different. Is a chylothorax as a result of a surgery comparable to an aneurysm of a femoral artery from catheterization? Is the avoidance of a thoracotomy scar worth a potential increased risk of an aneurysm at the site of balloon dilation? It is these types of questions which cannot be addressed by a randomized clinical trial, the analysis of which would be dominated by a single outcome, but which are effectively handled in a decision analysis.
In the current era, technology and technique, be they surgical or interventional, has made the mortality low for most congenital cardiac lesions. Thus, to rely solely on rates of mortality, which would have wide and overlapping confidence limits, and which Ebels and colleaguesReference Ebels, Maruszewski and Blackstone1 suggest should be the sole indicator for a successful therapy, is no longer appropriate. Instead, we must direct our attention towards decreasing morbidity associated with recommended therapies. In this regard, decision analysis provides a means of comparing two modalities, while retaining the ability to account for different types and rates of morbidity and mortality and their relative values, utilities, or weights. The decisions we as physicians make incorporate an inherent bias based on anecdote, experience, and recall. The techniques of decision analysis allow us to examine a clinical dilemma systematically while minimizing such bias, and adjusting for the perceived disutility of a given procedure and its outcomes.
Ebels and colleaguesReference Ebels, Maruszewski and Blackstone1 suggest that the literature from which the decision tree and probabilities of outcomes was compiled was inaccurate and dated, citing that they no longer reflect the current standard of practice. The existing literature, nonetheless, is the data at hand, and the data that would be accessed by decision makers, regardless of how incomplete or dated it may seem. Yet collection of data is only the starting point for a decision analysis. Surveying the literature permits generation of an approximation of rates of mortality and morbidity, while recognizing they may not be contemporary. The power of decision analysis lies in its sensitivity analysis, a fact seemingly misunderstood by Ebels and his colleagues.Reference Ebels, Maruszewski and Blackstone1 A sensitivity analysis permits variation of rates of mortality and morbidity within the range of reported estimates or the limits of statistical confidence or clinically plausible limits. Our methodology incorporated this as a primary focus of the analysis, and included values that would fall into the perceived spectrum of contemporary results. Additionally, our data reflects current perceptions. For example, when the perioperative mortality in the neonatal group was reduced to below 4%, surgery then became the preferred option. The European Association for Cardiothoracic Surgery database, as cited by Ebels et al,Reference Ebels, Maruszewski and Blackstone1 notes that current neonatal mortality rates for surgical repair are between 0.5–0.7 per cent. As applied in our analysis, this would also predict that surgery would be the preferred option for neonates in the contemporary era if these estimates are valid. Indeed, contemporary practice in most institutions would be to offer a surgical option for neonatal coarctation. As we did not have access to the unpublished data cited by Ebels et al,Reference Ebels, Maruszewski and Blackstone1 and most decision makers would not as well, it further supports the robustness of the sensitivity analysis and the advantage of a decision analysis.
In addition, Ebels et al appear to focus only on mortality, and have neglected to include the effect of morbidity on a given decision with no information on the severity or rates of morbidity. With no comparative data from interventional databases, the data provided by Ebels et alReference Ebels, Maruszewski and Blackstone1 is one-sided. Given that our analysis provides a systematic quantitative method for incorporating morbidity, disutility and inherent bias for both surgical and catheter-based interventions, as well as values or weights for differing outcomes, this approach is decidedly more applicable to the current era, and more closely achieves the aims of evidence-based clinical decision making.
A concern raised during the review of our manuscript was that surgeons and lay persons were not included among those interviewed to determine utilities. While surgeons were not deliberately excluded nor did they explicitly decline to participate, we were unable to schedule sufficient dedicated time with them to complete the standard gamble interview. However, we do not believe that this alters the validity of our methodology or findings. In a previous study from our institution using identical decision analysis methodology to evaluate surgical versus transcatheter device closure of suitable ventricular septal defects, surgeons were included and compared to other groups of respondents, including lay persons.Reference Aleem, Karamlou, Benson and McCrindle5 Final adjusted expected values for lay persons were similar to staff cardiologists and cardiology trainees, favouring transcatheter device closure, while surgeons and surgical trainees indicated a draw. This difference was largely explained by the fact that the surgeons and surgical trainees tended to estimate a higher mortality associated with transcatheter device closure and, importantly, did not perceive any disutility to surgery. This means that given a hypothetical choice between a perfect procedure with no mortality and no complications, the surgeons and surgical trainees would either prefer surgery or have no preference. This is in contrast to lay persons who are more adversative to invasive procedures and similar to cardiologists and other non-surgical respondents in their preferences. While we did not include lay persons in the groups of persons interviewed for the current study, based on our previous study we do not feel that this would alter our results.
As providers of care for those with congenital cardiac disease, we are charged with providing expert, and hopefully unbiased, recommendations regarding such care. Patients rely on us to inform them of various options for treatment, and many times turn to us as to provide opinion as to which option may be best. When the literature is scarce, or there are no clear recommendations regarding a given course of action, we are forced to rely on our recall of our own past experiences, or those in our Institution, which usually have not been systematically reviewed. This, as argued both by ourselves and Ebels and colleagues,Reference Ebels, Maruszewski and Blackstone1 is inherently associated with bias. Decision analysis provides a means systematically to model the data-derived factors that would influence a decision or recommendation. Our study should be viewed as another piece of evidence that clinicians might use, in addition to published case series and small clinical trials, to inform their own practice of evidence-based clinical decision-making. It provides another means of systematically evaluating a complex problem, to which there is currently no definitive solution. To imply that our decision analysis would lead to the belief that those working in our institution provide biased counselling and dictate to patients regarding decision-making and recommendations is completely inappropriate, incorrect, and detracts from the debate. We are attempting to provide an additional perspective as to how better make a recommendation, and provide optimal care of patients within the framework of evidence based medicine.
