2 Application to patient decision aids

Hammond’s meta-theories have been applied to social policy in many domains including medicine. There are many applications within the medical field; however, this article pertains specifically to treatment choices made by patients. For many treatment decisions, there is a dominant treatment choice, or a single “best” therapeutic action. This choice is easy because the treatment has a clear advantage — effectiveness is high and/or side effects are low. These treatment choices do not necessitate much assistance. However, there exist a number of medical conditions for which no single treatment choice dominates. These are the cases in which the patient must make tradeoffs between different dimensions or features of treatment options. For example, risk and potential benefit are often positively correlated, and a choice must depend upon how an individual values the potential benefits and harms. Currently there is no best therapeutic course of action for menopause, and women are commonly asked to choose whether or not to begin hormone replacement therapy (HRT). In addition to alleviating some of the symptoms of menopause (weight gain, hot flashes, poor memory, etc.), HRT also helps to protect the body from osteoporosis. However, HRT is also associated with an increased risk of breast cancer, heart disease and stroke. Therefore, each individual patient must weigh the increased health risks against the benefits, protection from osteoporosis and reduction in symptoms. Other similar examples include treatment for menorrhagia, ischaemic heart disease, atrial fibrillation, abnormal uterine bleeding, prostate cancer, and breast cancer.

These types of treatment choices are very difficult for patients and their physicians cannot guide them to the “right” answer. Therefore, a class of decision aids has been developed to help patients make difficult treatment choices between two or more options. Patient decision aids (PtDAs) are designed to educate patients on the various treatment choices and their associated risks and benefits, help patients understand the values they place on those risks and benefits, and improve communication between patients and doctors. The emphasis on improving patient decision making evolved during the shift in patient care philosophy from informed consent to informed choice or shared decision making. PtDAs were heralded as a way to improve informed choice, reduce geographic differences in treatment choices, combat the rise of consumerism in medicine, and increase patient satisfaction with the treatment process (O’Conner et al., Reference O’Connor, Stacey, Entwistle, Llewellyn-Thomas, Rovner, Holmes-Rovner, Tait, Tetroe, Fiset, Barry and Jones2003).

The number of PtDAs steadily increases each year, and the medical community has recently begun to evaluate the effectiveness of these programs. In 2003, the Cochrane Collaboration Patient Decision Aid Review Group published an extensive evaluation of the effectiveness of PtDAs. In 2006, the Society for Medical Decision Making held the International Patient Decision Aids Standards (IPDAS) symposium, which was designed to debate whether PtDAs were the best way to improve patient decision making. Subsequently, the journal Medical Decision Making dedicated an entire issue to the future of shared decision making in 2007, which contained a focus on evaluating the effectiveness of PtDAs (Reference Helfand, Barnato and Holmes-RovnerHelfand, Barnato, & Holmes-Rovner, 2007). What has emerged is a set of conflicting opinions about the value of PtDAs and, more basically, about the criteria upon which PtDAs should be evaluated. Reviewing this debate shows that its arguments parallel those between the correspondence and coherence theorists of decision making. Hence, the goal of this paper is to re-cast this discussion in light of these two meta-theories in the hope of learning lessons from a previous incarnation of this debate.

3 The debate about the effectiveness of patient decision aids

The Cochrane Collaboration Patient Decision Review Group (2003) summarized the results of 35 randomized controlled trials that evaluated the effectiveness of PtDAs in comparison with control groups receiving the usual care or simpler decision aids. Following the Cochrane Review, the IPDAS symposium was created to debate whether or not patient decision aids are the best way to improve clinical decision making. A vote after the symposium revealed that medical professionals are divided in their opinion of the effectiveness of PtDAs with about half believing PtDAs to be effective in improving clinical decision making thus favoring their widespread implementation and half believing PtDAs do not improve clinical decision making thus opposing their implementation (Reference Holmes-Rovner, Nelson, Pignone, Elwyn, Rovner, O’Connor, Coulter and Correa-de-AraujoHolmes-Rovner et al., 2007). Although a number of other issues were addressed, a major portion of the debate focused on the criteria for evaluating the success of PtDAs. Wendy Nelson, of the HHS National Cancer Institute, described the current mixed bag approach to evaluating PtDAs: “What is a quality decision, and what constitutes quality decision making? Because we lack an agreed-upon definition of quality, proponents of decision aids have operationalized quality in a wide variety of ways” (Reference Holmes-Rovner, Nelson, Pignone, Elwyn, Rovner, O’Connor, Coulter and Correa-de-AraujoHolmes-Rovner et al., 2007, p. 602).

Researchers evaluating PtDAs determine the effectiveness of a given decision aid by assessing whether or not it improves patient decision making. There have been two distinct approaches to operationalizing improved decision making, one that roughly captures the coherence approach and one that more closely approximates the correspondence approach. Researchers adopting the coherence approach often ask whether the use of PtDAs improves the congruence between patients’ stated values and their treatment choices (Reference O’Connor, Bennett, Stacey, Barry, Col, Entwistle, Fiset, Holmes-Rovner, Khangura, Llewellyn-Thomas and RovnerO’Connor et al., 2007; Reference O’Connor, Stacey, Entwistle, Llewellyn-Thomas, Rovner, Holmes-Rovner, Tait, Tetroe, Fiset, Barry and JonesO’Connor et al., 2003). In contrast, researchers adopting the correspondence approach typically ask whether the use of PtDAs results in increased quality of life (Reference McCaffery, Irwig and BossuytMcCaffery, Irwig, & Bossuyt, 2007). These two methods frequently lead to different conclusions about the effectiveness of PtDAs. We argue that the two sides in the PtDA controversy represent a renewed debate about correspondence versus coherence criteria. The following sections will discuss how the correspondence and coherence theories have been applied to the area of PtDAs and the benefits and difficulties associated with adopting each approach.

4 The correspondence approach to evaluating patient decision aids

Hammond’s 1955 article, “Probabilistic functioning and the clinical method,” marked the beginning of the correspondence approach to decision making. Although formally articulated by Hammond in the 1950s, the correspondence metatheory was inspired by Egon Brunswik’s general theory of cognition under uncertainty. The correspondence approach focuses on aligning human judgment with empirical accuracy. Typically, correspondence theorists have evaluated decision making by comparing human judgment to an external gold standard — often objective facts (Reference HammondHammond, 1996).

In the context of patient decision making, the logical application of the correspondence approach would be to examine whether patient treatment choices corresponded to the best treatment choices. However, PtDAs have been developed for treatment choices for which there exists no single best therapeutic action. Therefore, this obvious correspondence criterion is eliminated. Instead, “correspondence” theorists evaluating PtDAs argue for the implementation of other gold standards such as survival, function and well-being. More “accurate” choices lead to a greater chance of survival, greater post-treatment function and improved quality of life. Health outcomes have been evaluated using general health measures such as the 12-item Short-form Health Survey or measures that assess condition-specific health outcomes (Reference O’Connor, Stacey, Entwistle, Llewellyn-Thomas, Rovner, Holmes-Rovner, Tait, Tetroe, Fiset, Barry and JonesO’Connor et al., 2003). However, very few studies have used these measures. Out of the 35 randomized controlled trials of PtDAs reported in the Cochrane Review, only seven studies measured general or condition specific health outcomes. Five of these seven studies found no significant differences in health outcomes for patients who used decision aids. Furthermore, these metrics appear to have been subsequently abandoned. The IPDAS symposium, designed to evaluate the effectiveness of PtDAs, defined a good patient decision as “one that is informed and consistent with the patient’s values” (Reference Holmes-Rovner, Nelson, Pignone, Elwyn, Rovner, O’Connor, Coulter and Correa-de-AraujoHolmes-Rovner et al., 2007, p. 603). This definition is generally inconsistent with the correspondence approach, and, therefore, the majority of recent research on PtDAs has not included correspondence criteria. In fact, a recent meta-analysis of the effectiveness of PtDAs did not include any these outcome measures, abandoning the correspondence approach in favor of coherence (Reference O’Connor, Bennett, Stacey, Barry, Col, Entwistle, Fiset, Holmes-Rovner, Khangura, Llewellyn-Thomas and RovnerO’Connor et al., 2007).

There are difficulties associated with the correspondence approach that may have led some researchers to favor a coherence approach to evaluating PtDAs. First, the correspondence measures are difficult to evaluate in the short term. Metrics such as survival and long-term health outcomes require many years of follow-up data to be collected after the initial randomized controlled trial. Therefore, if these were the only measures adopted, it may be difficult to conduct timely research. In addition, some researchers have adopted the position that decision-making should be evaluated before the outcome of the decision is known. In support of this position, Reference Baron and HersheyBaron and Hershey (1988) documented the outcome bias in evaluating decisions; participants rated the quality of the decision and the competence of the decision maker more favorably after a positive outcome than a negative outcome. Furthermore, Caplan and colleagues (1991) found a negative relationship between outcome severity and judgments of the appropriateness of care by anesthesiologists.

5 The coherence approach to evaluating patient decision aids

Due to the lack of a clear gold standard and the difficulties associated with implementing the correspondence approach, a number of researchers have opted to evaluate the efficacy of treatment decisions by using coherence criteria in lieu of correspondence criteria. This approach to the study of decision making developed in parallel to Hammond’s correspondence approach. Ward Edwards is largely credited with beginning this movement in 1954 with his article on “The theory of decision making.” Edwards’ research examined the rationality of human judgment under uncertainty using probability theory as the cornerstone of rationality. Edwards evaluated judgmental ability by comparing human judgment to mathematical models — as opposed to empirical accuracy. Edwards focused on a specific type of probability calculus — Bayes’ Theorem. Later coherence theorists also adopted Expected Utility Theory (EUT) and its variants and their axioms as additional yardsticks for rationality.

Most research on PtDAs comes from the coherence approach. Because there is no objective gold standard or single best treatment, most research examining the effectiveness of PtDA asks whether the aid helps the patient to make a coherent choice (Reference O’Connor, Bennett, Stacey, Barry, Col, Entwistle, Fiset, Holmes-Rovner, Khangura, Llewellyn-Thomas and RovnerO’Connor et al., 2007; Reference O’Connor, Stacey, Entwistle, Llewellyn-Thomas, Rovner, Holmes-Rovner, Tait, Tetroe, Fiset, Barry and JonesO’Connor et al., 2003). In fact, two explicit goals of many PtDAs are to help patients clarify their values for various health states and make treatment choices that are congruent with those values. For example, a patient who values treatment effectiveness should be more likely to choose a treatment with a greater success rate. However, if she values quick recovery times or pain minimization, then she should be more likely to choose treatments that match this. Many PtDAs include values clarification exercises to help patients articulate their preferences; 30 out of 51 PtDAs included in a recent meta-analysis incorporated some explicit values clarification exercise (Reference O’Connor, Bennett, Stacey, Barry, Col, Entwistle, Fiset, Holmes-Rovner, Khangura, Llewellyn-Thomas and RovnerO’Connor et al., 2007). These exercises can include rating different possible health outcomes, forced choice tasks where patients choose between options with a single difference, and time tradeoff tasks where the patient is asked to decide how many years of perfect health they would trade in exchange for a certain number of years in some less desirable health state. The goal of these exercises is to determine how an individual patient values the relative risks and benefits associated with the different potential outcomes. Therefore, the coherence approach to determining the efficacy of a PtDA is to examine whether treatment choices made by patients using PtDAs are congruent with their stated values (Reference Holmes-Rovner, Kroll, Rovner, Schmitt, Rothert, Padonu and TalarczykHolmes-Rovner et al., 1999; Reference Lerman, Biesecker, Benkendorf, Kerner, Gomez-Caminero, Hughes and ReedLerman et al., 1997; Reference O’Connor, Wells, Tugwell, Laupacis, Elmslie and DrakeO’Connor et al., 1999)

Like the correspondence framework, researchers have argued that the coherence approach to evaluating patient judgments has some weaknesses. Specifically, evaluating the rationality of a treatment decision by its congruence to elicited preferences for health states relies on the validity of the system used to elicit those preferences. There are some reasons to question the validity of values clarification exercises commonly built into PtDAs (Reference Nelson, Han, Fagerlin, Stefanek and UbelNelson, Han, Fagerlin, Stefanek, & Ubel, 2007). These exercises assume that preferences for health states are stable and that patients are accurately able to predict preferences for future health states. However, research suggests that these assumptions could be questioned.

The coherence criterion of value congruence relies on the assumption that patients have measurable and stable preferences for health states and treatment choices. This assumption has been hotly contested by decision making researchers since the 1970s. A large body of research has demonstrated that preferences appear to be constructed at the time of judgment and are sensitive to the mode of elicitation and to framing effects (Reference SlovicSlovic, 1995; Reference Slovic and LichtensteinSlovic & Lichtenstein, 1983; Reference Tversky and KahnemanTversky & Kahneman, 1981). Therefore, in the context of a PtDAs, preferences for health states may be subject to framing effects, and values clarification exercises may help to determine preferences — not discover them. This again calls into question the use of value congruence as a yardstick for rationality.

Additionally, the validity of the values that patients assign to health states is also questionable. Even if we assume that patient preferences for health states are measurable and stable, values clarification exercises rely on the assumption that patients are accurately able to predict their utility for future health states. For example, imagine a man diagnosed with localized prostate cancer needs to choose between two approaches to dealing with the disease: watchful waiting or prostatectomy. In a very small proportion of men, a prostatectomy may be effective in preventing a metastasis of the cancer. However, the side effects of prostatectomy, impotence and incontinence, occur in a large proportion of men. In contrast, there are no side effects, other than perhaps increased stress, to watchful waiting, and the cancer will not metastasize in the majority of cases — although this number is smaller than with a prostatectomy. Therefore, the patient’s choice would be based on his utilities for the potential health states. Which would be more difficult to live with: a small increase in the likelihood that the cancer would metastasize or the risk of impotence/incontinence? In order to make the decision, the patient has to be able to successfully predict how he would feel in these previously unexperienced health states. Much research has shown that people are unable to predict their future feelings (Reference Damschroder, Zikmund-Fisher and UbelDamschroder, Zikmund-Fisher, & Ubel, 2005; Reference Gilbert, Pinel, Wilson, Blumberg and WheatleyGilbert, Pinel, Wilson, Blumberg, & Wheatley, 1998; Reference Riis, Loewenstein, Baron, Jepson, Fagerlin and UbelRiis et al., 2005; Reference Ubel, Loewenstein and JepsonUbel, Loewenstein, & Jepson, 2005a; Reference Ubel, Loewenstein, Schwarz and SmithUbel, Loewenstein, Schwarz, & Smith, 2005b; Reference Wilson and GilbertWilson & Gilbert, 2005). Reference Wilson and GilbertWilson and Gilbert (2005) argued that people are unsuccessful at “affective forecasting” because they mispredict their emotional reactions to future events. In particular, people fail to anticipate how readily they will adapt to emotionally difficult situations. This phenomenon has been demonstrated with affective predictions for various health states as well (e.g., Brickman, Coates, & Janoff-Bulman, Reference Brickman, Coates and Janoff-Bulman1978; Reference Ubel, Loewenstein, Schwarz and SmithUbel et al., 2005b). For example, Reference Sackett and TorranceSackett and Torrance (1978) found that patients with chronic health conditions, such as dialysis, have a greater quality of life than is predicted by people without a chronic illness. Therefore, the basic premise upon which values clarification exercises are designed — namely that patients can predict their utilities for future health states — is called into question.

The variables we have summarized in the preceding sections do not represent an exhaustive list of the measures used to evaluate PtDAs. There are additional variables that cannot be easily classified as fitting into either approach or that represent a hybrid of the two approaches. For example, researchers have examined a number of psychological measures such as anxiety, depression, regret, and decisional conflict. Some take a reduction in these psychological variables as evidence for the effectiveness of PtDAs (Reference O’Connor, Stacey, Entwistle, Llewellyn-Thomas, Rovner, Holmes-Rovner, Tait, Tetroe, Fiset, Barry and JonesO’Connor et al., 2003; Reference O’Connor, Bennett, Stacey, Barry, Col, Entwistle, Fiset, Holmes-Rovner, Khangura, Llewellyn-Thomas and RovnerO’Connor et al., 2007). Similarly, some researchers argue that other non-psychological measures such as persistence with a chosen treatment choice, increased participation in decision making, and reduced indecision after PtDA provide evidence that PtDAs are effective tools. Finally, some researchers examine whether the use of PtDAs improves the accuracy of risk perception, which is often determined by normative solutions such as Bayes’ Theorem.

Although some measures do not fit neatly into the correspondence/coherence distinction, we find it a useful classification tool for PtDA evaluation metrics. Correspondence measures speak to the empirical accuracy of judgments, while coherence measures speak to the logical consistency of judgments. Measures from these two categories directly evaluate the quality of patient judgments. Variables outside this classification system would at best be considered indicators of good judgment.