Introduction, Mandeep K. Dhami, Anne Schlottmann, and Michael R. Waldmann

In conclusion, rather than present a summary of the preceding chapters, we invited nine eminent past presidents of the Society for Judgment and Decision Making (SJDM) to provide personal perspectives on the concept of JDM as a dynamic skill. These scholars were not asked to comment on the chapters in this book, but rather to highlight their personal points of contact with the notion of JDM as a dynamic skill. The following perspectives offer historical accounts, and also point to future lines of research.

Shanteau describes how over the years he has highlighted the importance of training and skill acquisition in JDM, but feels “blue” that this view has not been more popular. Wallsten remembers the benefits of learning for JDM performance found in a study that he conducted 30 years ago, and confesses that he has only recently begun to revisit this important finding. Fischhoff points out that a sound understanding of the normative implications of tasks has laid a better foundation for the study of dynamically changing skills, especially in development. Levin and colleagues provide useful examples of their research on the developmental and neurological bases of JDM skills. Reyna highlights how her fuzzy trace theory taps into JDM processes that develop over time and experience, has neurological correlates, and may be evolutionarily adaptive. Baron reveals how he now finds himself in search of the developmental origins of the types of moral heuristics and biases that he has studied during his career. Hogarth shares three steps he has developed during decades of teaching decision making that can help people make better decisions. Klayman reveals that despite decades of studying learning and development of JDM, he still seeks a greater understanding of how decision makers “get that way.” Finally, Birnbaum points to the methodological factors that have limited our understanding of JDM as a skill, and presents a challenge for future researchers: to explain how and why JDM skills change. Overall, the following perspectives provide a rare glimpse of the personalized views of those who have made significant contributions to the field of human JDM.

Introduction

Health depends, in part, on deliberate decisions. Some are private, such as deciding whether to wear bicycle helmets and seat belts, follow safety warnings, use condoms and fry (or broil) food. Other decisions involve societal issues, such as whether to protest about the siting of an incinerator or halfway house, vote for fluoridation and ‘green’ candidates or support sex education.

Sometimes, single choices have large effects on health risks (e.g. buying a car with airbags, taking a dangerous job, getting pregnant). At other times, the effects of individual choices are small, but accumulate over multiple decisions (e.g. repeatedly ordering broccoli, wearing a seat belt or using the escort service in parking garages). Yet other times, choices intended to reduce health risks achieve nothing or the opposite (e.g. responding to baseless cancer scares, subscribing to quack treatments).

In order to make health decisions wisely, individuals must understand the risks and benefits associated with alternative courses of action. They also need to understand the limits to their own knowledge and to the advice proffered by various experts. This chapter considers how to describe people's beliefs about health risk issues, as a step toward designing (and evaluating) interventions designed to improve their choice. A fuller account would also consider the roles of emotion, personality, culture and social processes (Lerner & Keltner, 2000).

People cannot know what they want in all possible situations. Rather, they must construct specific preferences from more basic values. In making these inferences, people must seek cues in a world that might be indifferent, helpful, or manipulative. The better they understand the factors shaping their inferences, the better their chance of learning what they want. An analogous challenge faces researchers, decision analysts, counselors, and others seeking to elicit people's values. It also faces those (advertisers, politicians, pundits, etc.) hoping to manipulate those factors and get people to convince themselves that they want a particular product, candidate, ideology, or lifestyle.

Identifying the factors shaping behavior is psychology's central challenge. To study theoretically relevant factors, researchers must control irrelevant ones. Over time, those nuisance factors often prove important in their own right. McGuire (1969) has depicted the history of psychology in terms of turning “artifacts into main effects.” Table 35.1 assembles parts of this history in terms of the four essential elements of any behavior: the organism whose preferences matter, the stimulus intended for evaluation, the response mode for expressing preferences, and potentially distracting contextual elements.

In the 25 years since Table 35.1 was created, the attention paid to its various topics has varied, reflecting researchers' sensitivity to them as artifacts and interest in them as main effects. For example, affective influences on preferences are better understood now, partly due to advances in psychophysiology (Slovic, Finucane, Peters, & MacGregor, 2002).

BEHAVIORAL RESEARCH SHOWS NEED FOR POLICY RESEARCH AND ITS LIMITS

MacCoun (2004) insightfully uses a Bayesian framework to characterize debates regarding the relative validity of competing policies and the theories underlying them. That framework allows a thoughtful sorting and integration of empirical, analytical, and philosophical evidence. It reveals cognitive and motivational barriers to fulfilling the Bayesian vision of explicit, coherent hypothesis evaluation. It provides a forensic guide to more and less deliberately malevolent attempts to distort debates in non-Bayesian ways. In a sense, MacCoun does for contentious situations what Ruth Beyth-Marom and I tried to do for individual deliberations (Fischhoff & Beyth-Marom, 1983).

MacCoun raises one potential limit to this framework, posed by the observation of persistently non-Bayesian behavior. These biases have long been used to demonstrate the frailties of human judgment and, with them, the need for compensatory policies. For example, if people cannot understand risks, then, arguably, they need strong regulatory protections, in order to keep dangerous products off the market. Arguably, they also need manipulative public health measures in order to protect them from themselves (e.g., high cigarette taxes, social marketing of abstinence or condom use).

However, MacCoun notes that persistent violations also might suggest a fundamental flaw in the normative model. An extreme position is that people cannot think in the prescribed way, making it an inappropriate aspiration. As a result, it is investigators' responsibility to discern the method in any persistent apparent madness.

Accurate estimates of the fundamental constants of physics, such as the velocity of light or the rest mass of the electron, are central to the enterprise of science (Pipkin & Ritter, 1983). Like any measurements, they are subject to uncertainties from a variety of sources. Reliable assessments of this uncertainty are needed (1) to compare the precision of different measurements of the same quantity; (2) to assess the accuracy of other quantities derived from them; and, most crucially, (3) to evaluate the consistency of physical theory with the current best measurements. Thus, as Eisenhart pointed out, “A reported value whose accuracy is entirely unknown is worthless” (1968, p. 1201).

It is not unusual to encounter individual examples of errors in measurements of physical quantities that turn out to be disconcertingly large relative to the estimated uncertainty. One well-known case was in Millikan's oil-drop experiment in 1912 to determine e, whose result turned out 15 years later to be off by 0.6%, or three standard deviations, due to reliance on a faulty value for the viscosity of air (Cohen, Crowe, & Dumond, 1957). A more recent example concerns measurements of |v+-|, the parameter that measures the degree of violation of CP (charge–conjugation–parity) invariance. The six measurements prior to 1973 agreed reasonably, but more accurate measurements since then differ consistently by about seven standard deviations from the pre-1973 mean, a discrepancy that remains unexplained in terms of experimental procedure (Franklin, 1984).

Heuristics and biases are in the public domain. They appear in texts, courses, books, and applications of many kinds. These endorsements speak to the persuasiveness of the basic research, the severity of the applied problems, and the face validity of debiasing as a possible solution. However, as the research itself shows, popularity alone proves little. Erroneous beliefs can be firmly and widely held even in applied fields with high stakes (Bunker, Barnes, & Mosteller, 1977; Byrne, 1986, Gilovich, 1991). One cannot expect practitioners to read the research literature. Moreover even peer-reviewed journals can be penetrated by vigorous advocates of flawed work. Nor can one expect them to extract simple lessons from individual applications. Coincidental events are always at play, in explanation, prediction, or prescription. Both radical skeptics and rabid enthusiasts can find ad hoc reasons for creating the picture that they seek regarding the applicability of research.

Achieving cumulative wisdom regarding the generalizability of heuristicsand-biases research requires interpreting the available evidence within a stable, comprehensive framework – one that identifies a set of potentially relevant contextual factors and provides bridging theories for characterizing actual situations in its terms. When those situations are in the lab, such a framework might reveal otherwise neglected complexity (when investigators manipulate one feature, without considering concurrent changes in others or the restricted circumstances in which their focal change is possible). With real-world situations, an interpretative framework might reduce otherwise bewildering complexity, by focusing attention on features whose influence (if any) will emerge over time.

This appendix reproduces examples of transcripts of mental model interviews from our first mental model study, which was of radon. The guidelines offered in Chapter 4 reflect our experiences in this and later studies and were not all followed completely in this first study. In these transcripts, you can see where the interviewer has completed the first-phase follow-ups and gone on to more specific questions. You may wish to copy the interviewer worksheet reproduced in Figure 4.1 and try using it to keep track of the topics covered in these examples. Note that one of the interviewees asks the interviewer a question designed to obtain information, but the interviewer defers the answer.

Since early in the HIV/AIDS epidemic, it has been apparent that people's understanding of its risks are a primary public health concern. People must learn to avoid some behaviors (e.g., unsafe sex, sharing IV drug needles) while not avoiding others (e.g., giving blood, living and learning with infected individuals). As with many other risks, the information that people need is both quantitative and qualitative. For example, they require both a credible estimate of the prevalence of contaminated blood and a feeling for signs that their blood bank or drug partner is above (or below) that average. Moreover, an understanding of how the risks are created and controlled may be essential for making quantitative estimates intuitively plausible (e.g., “how could it be that condoms only reduce risks by a factor of 10?”). That understanding may also provide the feelings of self-efficacy needed to deal with an issue, and to lead others toward appropriate behavior.

Recognizing these factors, public health officials have invested extensive resources into getting the message out about HIV/AIDS (CDC, 1994; Institute of Medicine, 1996). These efforts have included TV spots and specials, school posters and pamphlets, as well as courses and curricula. Considering the magnitude of this effort and the stakes riding on its success, it is essential that the content of the messages be appropriate. This chapter reports an application of the mental models approach to this task.

The Need for a Systematic Approach

An effective communication must focus on the things that people need to know but do not already. This seemingly simple norm is violated remarkably often in risk communications. Rather than conduct a systematic analysis of what the public believes and what information they need to make the decisions they face, communicators typically ask technical experts what they think people should be told. Rather than subject draft communications to rigorous empirical evaluation by individuals like those who will use them, communicators pass them around to staff or expert committees for approval. Those passing judgment may know very little about either the knowledge or the needs of the intended audience.

Under such conditions, it is not surprising that audiences often miss the point and become confused, annoyed, or disinterested. If the communicators feel that they have done everything that is expected of them, they may conclude that their audience was responsible for the communications failure. If the public is blamed for being stupid, irrational, or hysterical, it is deprived of both information and respect in the future.

In some cases, the failure to inform reflects willful neglect or disinterest. Communicators would just as soon not have the public know the facts, or they may lack the resources needed to take this part of their job seriously. More often, though, we suspect that the failure reflects the lack of systematic procedures for finding out what people know and need to know, and for confirming empirically that a communication has been effective.

Reproduction of the risk communication brochure Global Warming and Climate Change. In its original form, the three details booklets reproduced at the back of this Appendix are housed in pull-out pouches at the back of Parts 1, 2, and 3 of the main brochure. The general structure of the brochure is shown in Figure A.1. Copies of the original version can be purchased at cost through the Department of Engineering and Public Policy, Carnegie Mellon University.

Mental models interviews usually produce a rich array of lay beliefs about a risk. If one could afford to do a hundred or more of these interviews, the results should also provide relatively accurate estimates of how common various beliefs are in the population being studied. However, the expense of these interviews typically precludes doing more than a modest number. In order to estimate prevalence, we rely on closed-form questions, which can be much more efficiently administered to large numbers of respondents.

Objectives of Questionnaire Studies

Knowing the frequency of concepts in the target population allows communicators to identify both widely shared correct concepts upon which a message can be built and widespread misconceptions that need to be addressed. Questionnaires can also be used to explore specific issues suggested in the mental models interviews but not adequately resolved. For example, our mental models interviews about climate change suggested that many people confused the terms “climate” and “weather.” Both refer to the condition of the atmosphere, measured in terms of wind, temperature, humidity, atmospheric pressure, precipitation, and so on. In most places, weather can change from hour-to-hour, day-to-day, and season-to-season. In contrast, climate is the average or overall pattern of weather in a place. For example, San Diego has a “Mediterranean climate,” meaning that temperatures are generally moderate year-round, there is limited rainfall, and humidity is typically low.