In the fall of 2021, many hospitals in the United States were overwhelmed with COVID-19 patients, most of whom had refused to be vaccinated against the disease. Many of these nonvaccinators appealed to moral principles concerning freedom and rights, which they took to outweigh the consequences of their decision. They claimed the right to make decisions about their own bodies and the right to freedom from government control over personal behavior. Some politicians supported these views even to the point of trying to prohibit schools and private businesses from imposing mandates for mask wearing or vaccination. Note that the expected consequences of nonvaccination are bad for everyone. Vaccination reduces the probability of serious illness for the individual, and it reduces the probability of an infected person, even one without symptoms, transmitting the disease to others. If effects on others are morally relevant, then nonvaccination is not only individually irrational but also immoral, unless some other moral principle outweighs these effects.
This case is an example of a frequent conflict between moral principles that people advocate and try to follow, on the one hand, and the expected consequences of following those principles, on the other. The moral principles at issue are inconsistent with moral principles based on utilitarianism, which holds that choice options should be evaluated in terms of their expected consequences for all those affected, but this is not all that makes these principles irrational. The choice of nonvaccination for oneself conflicts with expected utility theory (discussed later in the chapter) as applied to individual choices; it is a losing gamble. And opposing vaccinations for others is simply harmful to them, which by itself is inconsistent with any concept of morality.
Apparent examples of this sort of inconsistency in the real world have been extensively documented. In many cases, the analysis of expected consequences is based on economics rather than utilitarian analysis, but the conclusions of economic analysis are generally consistent with those that utilitarian analysis would imply.Footnote 1 Apparent inconsistencies have been found in allocation of resources to large humanitarian tragedies (Bhatia et al., Reference Bhatia, Walasek, Slovic and Kunreuther2021); in insurance decisions by firms and individuals (Johnson et al., Reference Johnson, Hershey, Meszaros and Kunreuther1993); in excessive attention to some risks coupled with neglect of others (Breyer, Reference Breyer1993; Kunreuther & Slovic, Reference Kunreuther and Slovic1978; Sunstein, Reference Sunstein2002); in tax policy (McCaffery, Reference McCaffery1997); in economic policies concerning trade, price controls, and wages (Caplan, Reference Caplan2007); and elsewhere.
All these realistic cases (and many more) support the argument that people’s moral judgments, when put into practice, can lead to consequences that people themselves would consider worse on the whole than what might have been achieved. But the real world is complicated. It is possible that the principles can have a utilitarian defense after all. For example, many of these apparently self-defeating policies arise through the functioning of institutions, such as legislatures and courts, that are imperfect yet better than any feasible alternatives, so that any attempt to overturn their results would, in the long run, make matters worse as a result of weakening these institutions. It thus becomes reasonable to ask whether people really apply nonutilitarian principles when they make moral judgments. One way to answer this question is to do psychology experiments, and those are the main topic of this chapter. At issue is the question of whether we can demonstrate truly irrational and nonutilitarian reasoning in hypothetical or real judgments under controlled conditions and, if so, whether we can learn something about the determinants of these judgments.
8.1 Normative, Descriptive, and Prescriptive “Models” in Experimental Psychology
Since the nineteenth century, psychologists have studied reasoning in contexts in which right answers are defined by some formal theory such as the logic of syllogisms. A common finding is that reasoning did not conform well to the model, thus, Henle (Reference Henle1962) begins by pointing out that “[t]he question of whether logic is descriptive of the thinking process, or whether its relation to the thinking process is normative only, seems to be easily answered. Our reasoning does not, for example, ordinarily follow the syllogistic form, and we do fall into contradictions” (p. 366). Around the same time (the 1950s and 1960s), others were comparing human judgments to other normative models, including probability and statistics (Bruner et al., Reference Bruner, Goodnow and Austin1956; Chapman & Chapman, Reference Chapman and Chapman1969; Meehl, Reference Meehl1954). In retrospect, we can think of such research as comparing “descriptive models” – psychological accounts of what people are doing – to normative models. The term “model” is inappropriate because it implies some sort of formal system. A few such systems exist, but the term is used even when they do not.
Kahneman and Tversky (Reference Kahneman and Tversky1979; Tversky, Reference Tversky1967; Tversky & Kahneman, Reference Tversky and Kahneman1981) began to apply this approach to decisions as well as judgments (and their 1979 paper proposed a true descriptive model that accounted fairly well for choices among simple gambles). Their normative model was expected utility theory in the form proposed by Savage (Reference Savage1954), in which both probability and utility were subjective (even if numerical probabilities were included in problem statements). (See also Chapter 7 in this volume.) Given this normative model, researchers could not always determine whether a given decision conformed to the model or not. For example, one person might prefer $10 for sure over a gamble with a 0.6 probability of $25 and a 0.4 probability of $0. Another person might prefer the gamble. The former person’s utility for $25 might be less than twice as high as her utility for $10, and she might think of 0.6 as “essentially an even chance,” so that her subjective probability of winning would be closer to 0.5. Thus, for her the expected utility (subjective probability times subjective utility) of the gamble would be less than that of $10 for sure.
To overcome this problem and show that choices were inconsistent with the normative model, Tversky and Kahneman (Reference Tversky and Kahneman1981) emphasized the use of framing effects, in which the same choice, in terms of consequences and their probabilities, was offered in different words. If subjects made different choices in the two versions, then they could not be following the normative model, which concerns consequences and probabilities. A classic example was the Asian disease problem, in which some subjects were told that an Asian disease was approaching and 600 deaths would be expected if nothing was done. In one version, the subjects chose between “200 saved” and a 0.33 chance to save 600. In another version the choice was between “400 die” and a 0.67 probability that 600 would die. Most subjects in the “saved” condition chose “200 saved,” and most subjects in the “die” condition chose the gamble.
This experiment had two properties that have received little attention in the extensive literature about it. One is that it is essentially a moral problem, not an individual choice like the money gambles used in other studies. It is moral because it is a decision about the well-being of other people. Research on decisions had slipped from a focus on expected utility to a focus on utilitarianism. Utilitarianism is the natural extension of expected utility to decisions for many people.Footnote 2 The utilitarian normative model here is to base the decision on the expected number of deaths (usually assuming that the subjective probabilities match the given probabilities).
The second property of the Asian disease problem concerns strong preferences for the two options. The expected utilities of the two options are close. Thus, strong preferences for different options violate a feature of utilitarianism (and other moral theories), which is to treat all lives equally. In gains, for example, a strong preference for “save 200” implies that the extra 400, beyond the 200 saved, are given less weight than twice that of the first 200 lives. Slovic (e.g., Reference Slovic2007) has explored this finding of unequal treatment extensively. One way to think of this phenomenon is in terms of the curve relating total disutility to number of deaths. People tend to make decisions as if the slope of this curve decreases: the millionth death matters less than the tenth, or the first.
Here is another example of the move from individual to moral decisions. The pertussis vaccine used to prevent whooping cough in the 1980s would often cause a disease very much like the one it prevented but at a much lower probability. Despite the clear benefits, many people resisted (and still resist) vaccination (Asch et al., Reference Asch, Baron, Hershey, Kunreuther, Meszaros, Ritov and Spranca1994; Sherman et al., Reference Sherman, Vallen, Finkelstein, Connell, Boland and Feemster2021). Ritov and Baron (Reference Ritov and Baron1990) found, in a laboratory study, that many people would not want such a vaccine, because (presumably) they would not want to cause the disease through their action. Ritov and Baron also found that people would also oppose requiring the vaccine as a public health measure. The individual decision was purely a matter of self-interest, but the public health decision was moral, because it concerned the well-being of other people.
Note that, in this case, the self-interested decision is irrational (from the perspective of expected utility) because omission of the vaccine increases personal risk. Could we say that the moral omission is also irrational because it means that more people will be sick? Some moral systems have a rule against using people as means to help others, and it could be argued that those who suffer from the side effects will serve as means to prevent disease in a greater number. Yet it seems inconsistent to say that the decision that is rational for each individual is immoral when applied to the population.
In these examples, the general approach of comparing laboratory decisions to normative models can be, and has been, extended from individual decisions to moral decisions, often with the implicit use of utilitarianism as a normative model. Further research, some of which I review here, finds that the departures from utilitarianism are systematic. As noted, some of these departures result from distortions in the way people think about quantities. Many others result from the application of nonutilitarian principles to the problems of interest.
These principles may be absolute or “prima facie,” that is, considerations that can be overridden by other considerations (Ross, Reference Ross1930/2002). Examples are: “We have a right to control our bodies,” “Do no harm” (meaning do no harm through action, as opposed to omission), “Do not use people as means to achieve better outcomes for others,” or “Do not kill innocent people.” They are often called “moral intuitions” (Hare, Reference Hare1981) or “moral heuristics” (Sunstein, Reference Sunstein2005). “Heuristics” originally referred to weak methods that might be helpful in solving problems, such as: “Do you know a related problem?” (Polya, Reference Polya1945), but the term was used by Tversky and Kahneman to refer to judgment tasks. An example is judging the probability that someone is a member of a group by the similarity of that person to prototypical members of the group, thus ignoring other relevant attributes such as the size of the group (Tversky & Kahneman, Reference Tversky and Kahneman1974).
When this view is extended to moral judgments, other problems arise. In principle, a heuristic is a “fast and frugal” method that often works but sometimes does not.Footnote 3 In morality, though, some of these heuristics seem to become hardened into rules that people knowingly apply, believing that they constitute the best possible moral judgments. Theologians and philosophers defend these rules as normative in this way (e.g., the rule about not using people as means). Such rules are often part of deontology, a class of moral systems based on rules, rights, and duties, which go beyond simply bringing about the best expected consequences for all.Footnote 4 Thus, much of the research on moral judgment focuses not so much on heuristics but on the contrast between deontological rules and utilitarianism. In this research literature, the terms “deontological” and “utilitarian” are not meant to imply that choices are based on representations of either system, just that they are consistent with what those choices would be.Footnote 5
Here I use the term “(moral) intuition,” for what others call moral heuristics. I think it captures the idea that the relevant moral principles tend to be evoked immediately upon presentation of a moral problem, without any extra effort to look for other relevant considerations.
All normative models are controversial to some degree, including Bayesian probability theory and expected utility theory (e.g., Ellsberg, Reference Ellsberg1961), but utilitarianism seems more controversial than most of the others that are studied psychologically, in part because it yields conclusions that seem to conflict with strong moral intuitions held by philosophers and psychology researchers as well as by experimental subjects. Hare (Reference Hare1981) has dealt with this conflict explicitly and in depth. His approach turns out to be surprisingly relevant to experimental psychology (as I discuss later).
But there are other reasons for looking for biases relative to a utilitarian normative model, even for those who do not accept utilitarianism as truly normative. Specifically, if people consistently violate the utilitarian standard in the same biased way (as in favoring harmful omissions over less harmful acts), we should not be surprised if the real consequences turn out to be clearly worse than if the utilitarian standard were followed.Footnote 6 As I suggested, many examples in the real world can be explained in terms of such biases. Thus, the study of violations of utilitarianism can at least help us understand why things in the real world are not as good as they could be. If the violation of utilitarian standards is the result of following nonutilitarian moral rules, then we at least learn the potential cost of following those rules.
Of course, much more could be said in defense of utilitarianism (e.g., Hare, Reference Hare1981, whose other work is nicely summarized by Singer, Reference Singer2002), but this is not the place for it.Footnote 7
In the 1980s, a third type of “model” became apparent, which was called “prescriptive” (Bell et al., Reference Bell, Raiffa and Tversky1988).Footnote 8 The idea is that normative models specify a standard and prescriptive models tell us what to do in order to do better by that standard. The distinction arose because the idea of “decision analysis” was exactly to bring decisions into conformity with various forms of expected utility theory, but decision analysis had many techniques that were not part of that theory, and most of them lead to approximations at best, for example, ways of estimating probability and utility of outcomes. Expected utility theory tells us about the mathematical relations among inputs and outputs. Decision analysis is not the only prescription for good decision making (where good is defined in terms of expected utility). Others are “decision architecture” (Thaler & Sunstein, Reference Thaler and Sunstein2008), and various educational programs.
As an example of the distinction, consider the concept of division in arithmetic. The formal definition (normative model) is (roughly): if A/B=C, then A=BC; we define division (A/B) in terms of multiplication. But this definition does not tell us how to do it. Many people now learn “long division” as a prescriptive procedure to divide large decimal numbers using successive approximations, starting with the leftmost digits in the dividend. To “understand long division” is to see why this procedure leads to a normatively good conclusion (exact for integers but sometimes just a good approximation).
For utilitarianism as a normative model, various prescriptive models have been proposed. One is simply to start with knowledge of the normative idea and then ask if it is obvious which option is better, for example, when the matter involves substantial benefits to some people at the cost of small inconvenience to others (e.g., vision tests for driver’s licenses). Another is decision analysis itself when it includes estimates of utilities for different groups of people who are affected. (This is close to “welfare economics.”) Another is to follow a rule, such as: “Do not commit adultery.” Hare (Reference Hare1981) suggests that, in real cases where adultery is an issue, any attempt to evaluate probabilities and utilities will be so biased by emotional factors that it is likely to lead to erroneous and very harmful consequences. (J. S. Mill made similar arguments.) Thus, the best way to conform to the utilitarian standard is sometimes to try to do something else. Moral intuitions may also be prescriptive in this sense. Hare (following Sidgwick) thinks they usually are, but the research described here suggests that some are more harmful than helpful. We might say that a major prescriptive question for utilitarians is how to bring up (and educate) children so that they do not fall prey to the harmful intuitions.
The approach to the study of moral judgment discussed here is an extension of the general approach to the study of judgments and decisions just summarized. We look for biases, that is, systematic departures from normative models. We try to explain these in terms of descriptive models. Based on what we have found, we then (ideally) try to design prescriptions for fixing these biases. This approach makes the study of moral judgment part of applied psychology, part of an attempt to make things better, like clinical psychology or educational psychology, except that we try to make the normative model, the standard for success, explicit. Alternative approaches, not discussed here, involve description “for its own sake,” without any attempt at evaluation and improvement.
8.2 Methods and Biases
In this section I will discuss several experimental methods and possible biases, organized by method rather than substantive topic, although I comment on the normative approach to some of the topics. All of these methods are potentially capable of showing that judgments or hypothetical decisions are nonutilitarian.
It is worth noting that essentially all of the nonutilitarian biases I describe here are the result of processes also found in nonmoral situations. Cushman and Young (Reference Cushman and Young2011) and Greene (Reference Greene and Sinnott-Armstrong2008) have argued explicitly for the parallelism between “cognitive biases” and patterns found in moral judgment.
8.2.1 Framing Effects
A framing effect, as noted already, is found when two equivalent cases yield different responses. An example from moral psychology is the effect on fairness judgments of describing tax rate differences as surcharges or bonuses, holding constant the same pre-tax and post-tax income levels. McCaffery and Baron (Reference McCaffery and Baron2004), inspired by classroom demonstrations reported by Thomas Schelling, presented subjects with examples like the following (edited for simplicity):
Childless Surcharge
Low income: A married couple with $25,000 total income and two children pays $3,000 in taxes, as a couple.
The same couple, if it had no children, would pay a surcharge of $1,000.
High income: A married couple with $100,000 total income and two children pays $30,000 in taxes, as a couple.
The same couple, if it had no children, would pay a surcharge of $3,000.
Child Bonus
Low income: A married couple with $25,000 total income and no children pays $4,000 in taxes, as a couple.
The same couple, if it had two children, would get a bonus of $1,000.
High income: A married couple with $100,000 total income and no children pays $33,000 in taxes, as a couple.
The same couple, if it had two children, would get a bonus of $3,000.
For the Child Surcharge, most subjects judged the surcharge as too high for the low-income family and too low for the high-income family. For the Child Bonus, they judged the bonus as too low for the low-income family and too high for the high-income family. Yet the high-income bonus and surcharge are the same, as are the low-income bonus and surcharge. Although the question is about fairness, a moral property, nothing here depends on utilitarianism as such. The intuitions about fairness that drive this result cannot be consistent with utilitarianism because they lead to different consequences depending on the description.
In another example of a framing effect, Harris and Joyce (Reference Harris and Joyce1980) told subjects that a group of partners had opened a business (e.g., selling plants at a flea market). The partners took turns operating the business, so different amounts of income were generated while each partner was in control, and different costs were incurred. Subjects favored equal division of profits when they were asked about division of profits, and they favored equal division of expenses when asked about expenses. Because expenses and profits were unequal in different ways, their two judgments conflicted. This result depends on an intuitive principle of equality, and, again, the inconsistency does not depend on utilitarianism.
A more complex framing effect concerns the effect of marriage (McCaffery & Baron, Reference McCaffery and Baron2004). When asked directly, many subjects favor “marriage neutrality,” which means that marriage does not affect the total taxes paid. People also favor progressive taxation, which means that those with higher incomes pay a higher percentage in taxes. Finally, people tend to favor “couples neutrality,” which means that couples with the same income pay the same tax, regardless of which earner makes more. Careful reflection (left as an exercise for the reader) implies that these three principles are incompatible. One of them must give.Footnote 9 This is, like the child bonus/surcharge, a logical inconsistency, hence a form of framing effect, which involves focusing on the question that is asked, an “isolation effect” (discussed later).
8.2.2 Contrast of Utilitarian and Nonutilitarian Options
Other methods involve asking subjects to decide between two options, one of which is consistent with utilitarianism and the other of which is not. The nonutilitarian option deviates by exemplifying a particular bias.
8.2.2.1 Omissions
A great deal of research has concerned action/omission dilemmas such as the vaccine case already described, in which people are more willing to accept the harms caused by omission than the harms caused by action. Although Ritov and Baron (Reference Ritov and Baron1990) coined the term “omission bias” as a name for this bias, that term was misleading. A simple bias toward omission would be a bias toward the default, whatever it is. Although a default bias does exist, it plays a minor role in the bias at issue (Baron & Ritov, Reference Baron and Ritov1994). Another determinant is the amplification effect, in which the consequences of action are simply given more weight than those of omission. If both options involve gains rather than losses, the amplification effect induces a bias toward action, which can be large enough to overcome the default bias.
Recent studies have tended to concern a set of dilemmas originally designed by philosophers as extreme cases on which to test, and try to explain, their moral intuitions (e.g., Foot, 1967/1978). In the simple trolley case, a runaway trolley is headed toward five people and will kill them if nothing is done. You can divert the trolley onto another track where it will kill only one person. Most people think diversion is the best response. In the “footbridge” version, the only way to stop the trolley is to push a large man off a footbridge, so that he falls on the track and blocks the trolley, being killed in the process. Most people resist this solution, and many experiments have tried to examine and explain this sort of difference.
A potential issue for experiments like these is what question to ask. In many experiments, the researcher asks: “Is it acceptable to push the man …?” The problem with this is that “acceptable” applies only to a single option, and utilitarians (and others concerned with decisions) must ask the question “compared to what?” The relevant question for us is which option is better, morally. Deontology often makes distinctions between what is permitted, forbidden, or morally required. Because these categories apply to options, not choices, it is possible for both options in a choice to be acceptable, or both forbidden. Other alternatives that work for everyone are: “Which option should [the agent] choose?” and “Which option should you choose [if you were the agent]?” Some studies have asked replaced “should” with “would.” This may be interesting, but some people say, explaining themselves, “I know that I should do it, but I could not bring myself to actually do it” (Baron, Reference Baron1992).
8.2.2.2 The Nature of “Omission Bias”
The literature has identified two major determinants of “omission bias”: deontological rules and the use of a limited concept of causality.Footnote 10
Rules favoring omission are more common than those favoring action (Baron & Ritov, Reference Baron, Ritov, Bartels, Bauman, Skitka and Medin2009). Rules that prescribe acts are usually conditional on some role. A physician, once accepting a patient, is morally and legally obliged to try to save the patient’s life (unless instructed otherwise) but a rule requiring anyone to try to save every life at risk is impossible to take seriously. Likewise, a rule against performing abortions is easier to follow than a rule requiring prevention of miscarriages in similar situations.
“Utilitarian moral dilemmas” often involve rules of this sort, such as prohibitions against killing, or tampering with human genes that affect future generations. When these rules are understood to be absolute (as discussed in Section 8.2.2.3), we would expect that subjects would object to action regardless of how beneficial its consequences are. These results are found (Baron & Ritov, Reference Baron, Ritov, Bartels, Bauman, Skitka and Medin2009). Thus, one determinant of the usual bias favoring omissions over less harmful acts, is the result of specific rules that are understood to be absolute (or nearly so).
Another determinant concerns causality (Cushman, Reference Cushman2008). We can (loosely) classify judgments of causality into two categories. One category, which includes “but for” causality, may be called “make a difference.” You are (perhaps partially) causally responsible for some outcome if something under your control could have made a difference in whether the outcome occurred or not. This view does not distinguish acts and omissions as such. It is often applied to tort law, especially lawsuits against someone who is supposed to take care to avoid harming others. Utilitarianism implies make-a-difference causality, at least when the options are clearly laid out and both possible.Footnote 11
The other category might be called direct causality. By this view, you are causally responsible for some outcome if there is a chain of events between your action and the outcome, with each link in the chain following some known principle of causality, such as the laws of physics (but any science will do). By this view, people may sometimes be held morally responsible for outcomes that they could not have avoided. (Spranca et al., Reference Spranca, Minsk and Baron1991, report a few instances of this.) Young children tend to consider outcomes only, thus judging that an act is wrong if it causes harm by accident (Piaget, Reference Piaget1932). The apparent bias toward harmful omissions over less harmful acts seems to be closely related to direct causality.
Supporting a role for perceived direct causality, Baron and Ritov (Reference Baron and Ritov1994, Experiment 4) compared the original vaccination case (in which vaccination deaths were side effects) with a “vaccine failure” case, in which the deaths that result if the vaccination is chosen are caused by its failure to prevent the natural disease. The bias against vaccination (action) was much stronger in the original condition than in the vaccine failure condition.
Royzman and Baron (Reference Royzman and Baron2002) compared cases in which an action caused direct harm with those in which an action caused harm as a side effect (i.e., “caused” only in the make-a-difference sense). For example, in one case, a runaway missile is heading for a large commercial airliner. A military commander can prevent collision with the airliner either by interposing a small plane between the missile and the large plane or by asking the large plane to turn, in which case the missile would hit a small plane now behind the large one. The indirect case (the latter) was preferred. In Study 3, subjects compared indirect action, direct action, and omission (i.e., doing nothing to prevent the missile from striking the airliner). Subjects strongly preferred omission to direct action but only weakly preferred omission to indirect action. Baron and Ritov (Reference Baron, Ritov, Bartels, Bauman, Skitka and Medin2009, Study 3) found similar results; they also found that perceived action was the main determinant of bias against action.
Greene et al. (Reference Greene, Cushman, Stewart, Lowenberg, Nystrom and Cohen2009) found that direct causality is a matter of degree. The most resistance to action occurred when a physical effect of action (hands-on pushing a man) caused a death, compared to cases in which the causal link between action and outcome involved more steps.
In sum, it seems that the bias against beneficial action is the result of at least two factors other than default bias: the perception of direct causality, as opposed to make-a-difference causality; and the commitment to particular rules that prohibit certain actions.
All of these studies, it should be noted, are consistent with sometimes extreme individual differences, with some subjects making the utilitarian response almost all the time. These subjects apparently do follow make-a-difference causality. In some experiments, we have found subjects who equate inaction with standing by in the face of evil, as with those German citizens who tolerated Hitler (e.g., Spranca et al., Reference Spranca, Minsk and Baron1991).
Note that some of these studies also ask about “blame” or “responsibility.” The latter term is ambiguous between causal, moral, and legal meanings (Malle, Reference Malle2021). The former is sometimes subsumed under the term “punishment,” which is examined more directly (and less ambiguously) in other experiments (later in this chapter).
8.2.2.3 Protected Values
Some deontological rules are taken to be absolute (Baron & Ritov, Reference Baron, Ritov, Bartels, Bauman, Skitka and Medin2009). Tetlock (e.g., Reference Tetlock2003), has used the term “sacred values” for essentially the same phenomenon, and Roth (Reference Roth2007) has used the term “repugnant transactions” for moral prohibitions on transactions such as a live donor selling a kidney. These protected values (PVs) are thus “protected” from trade-offs with other considerations. Some PVs are based on religion, but many are held by atheists, such as rules against cloning or genetic engineering of humans. In such cases, people say they should not violate the rule (usually a prohibition) no matter how great the benefits are. However, when asked to try hard to think of cases in which the benefits would be great enough, or when given some possible counterexamples, most people admit that the rules are not in fact absolute, so they seem to be absolute only as a result of insufficient reflection (Baron & Leshner, Reference Baron and Leshner2000; Tetlock et al., Reference Tetlock, Mellers and Scoblic2017).
Protected values may function as heuristics that serve the purpose of avoiding further thought about whether some trade-off is warranted (Hanselmann & Tanner, Reference Hanselmann and Tanner2008). Thus, they appear to be nonutilitarian. However, J. S. Mill (Reference Mill1859) argued that we should follow certain moral rules even if it seems clear that the consequences of breaking them in some situation would be better than those of following the rule. Suppression of free speech was an example. The idea here is that our own judgments about expected consequences in such cases are not trustworthy; we are subject to self-serving biases and ordinary error. We do not need to deceive ourselves in order to follow such rules. When asked to join a terrorist cell, a person today might think to himself:
It seems that the cause is just, and that the total harm of the deaths that we would cause would be much smaller than the harm we would prevent by carrying through the plan. But I know that almost all the terrorists throughout history have drawn just this conclusion, and the vast majority of them have been incorrect. Thus, it is probably best if I don’t join.
Note that everything is conscious here. No self-deception is needed.
Thus, in experiments on PVs, it is worth giving subjects ways to express apparent PVs that are actually consistent with utilitarianism. Baron and Leshner (Reference Baron and Leshner2000, Experiment 2) included the following, among other nonexclusive options for responses to possible PV items such as “cutting all the trees in an old-growth forest”:
(1) I cannot imagine any situations in which this is acceptable. (38)
(2) I can imagine situations in which the benefits are great enough to justify this, but these situations do not happen in the real world. (7)
(3) There are situations in the real world in which the benefits are great enough, but people cannot recognize these situations, so it is best never to do this. (9)
(4) This is unacceptable as a general rule, but we should make exceptions to it if we are sure enough. (28)
The percentages of choices are shown in parentheses, so it seems that apparent PVs are not usually the result of a Mill-type explanation and are truly nonutilitarian principles. In this experiment, only the first response (with 38 percent) represented a true PV. Note that our claim here is that PVs exist with sufficient prevalence to matter; both subjects and items differed substantially in the prevalence of true PVs.
8.2.2.4 Parochialism and Self-interest
From a utilitarian perspective – as well as many other perspectives – a major bias in people’s reasoning is parochialism (Baron, Reference Baron, Goodman, Jinks and Woods2012a, Reference Baron2012b; also called “in-group bias”). The technical use of the term refers to a class of experimental social-dilemma games (Bornstein & Ben-Yossef, Reference Bornstein and Ben-Yossef1994). In a social dilemma, each player can help other players in the group at some cost to himself, and the total benefit to the group is greater than the cost. This is called “cooperation.” Examples in the real world are widespread, from doing one’s job without shirking, to following rules (e.g., rules for income taxes) even when there is no chance of getting caught breaking them, to contributing to charities. Parochialism arises when each player’s behavior can affect an in-group and an out-group, and some players are willing to help the in-group at some personal cost while hurting the out-group even more (perhaps as a result of ignoring the out-group).
Consider voting as an example. “Cooperation” means voting for the candidate or proposition that is best for those who are relevant to your vote, which could be you and your family, your compatriots, or everyone in the world. Defection in this example is not voting. Voting has a cost. It is well known but not well understood, that the probability of being the pivotal (decisive) voter is so low that, even if you gain a large amount of money from your side winning, the expected return of voting is, like that of a lottery ticket, not worth the cost.
However, if you care enough about other people, taking their utilities as part of your own, with some weight for each other person, then voting can be worth the cost (Edlin et al., Reference Edlin, Gelman and Kaplan2007). Given this mathematical fact, a situation could arise in which it is not worth voting if all you care about is yourself, not quite worth voting if you care about your nation, but well worth voting if you care about humanity. If you are rational, you would then vote for candidates or proposals that are best for humanity. Otherwise, voting is not worth the cost. The same applies to many other forms of political action. Possible current examples of policies that affect the world are climate change, refugees, migration, population pressure on natural resources, fisheries, biodiversity, world peace, world trade, and the strength of international institutions that attempt to regulate these matters. Nationalist policies often work to the detriment of adequate attention these issues. In its general form, parochialism is a candidate for the most harmful departures from utilitarian decision making.
“Cosmopolitanism” is sometimes used as a technical term used for the attitude of caring about the world. Although this attitude sounds as idealistic and fanciful as the John Lennon song “Imagine,” in fact it is fairly common in the modern world (Buchan et al., Reference Buchan, Brewer, Grimalda, Wilson, Fatas and Foddy2011; Buchan et al., Reference Buchan, Grimalda, Wilson, Brewer, Fatas and Foddy2009). Arguably, it could arise as a result of reflection (Singer, Reference Singer1982). What principle can justify caring about some people but ignoring others? Answers could arise, but when we reflect on them (without bias toward inherited opinions) they may seem weak. Surely this sort of reasoning was part of what has led people to oppose slavery and to promote women’s political and legal rights. The absence of it allows parochialism to exist.
Other sorts of reasoning lead to parochialism (Baron, Reference Baron, Goodman, Jinks and Woods2012a, Reference Baron2012b). People think they have a duty to support their nation because their nation has given them the vote, or in return for what their nation has done for them. (Of course, most nations do not tell their citizens, even naturalized citizens, that this is expected, and it is well known that some voters, especially in a nation of immigrants, are concerned with particular foreign countries to which they are tied in some way.)
8.2.3 Attending to Irrelevant Attributes or Ignoring Relevant Ones
Kahneman and Frederick (Reference Kahneman, Frederick, Gilovich, Griffin and Kahneman2002) proposed that many biases can be explained in terms of “attribute substitution.” Two options differ in terms of two or more attributes. Some attributes are normatively relevant and some are not, but the latter are easier to use and typically correlated (imperfectly) with the relevant ones. So people use the irrelevant ones and sometimes ignore the relevant ones completely.
8.2.3.1 Allocation
A great deal of research has examined how people think they should allocate benefits and burdens. Allocations can be local, such as the distribution of grades in a class or housework among those living together. But I focus here on policy. These issues include income, wealth, taxes, criminal penalties, tort fines, insurance, and compensation. Much of this research has examined the principles that people use for allocation decisions (e.g., Deutsch, Reference Deutsch1975). These include equality (everyone gets the same); contribution (to each according to their contribution, also called “equity”); need (to each according to need); and maximization (e.g., maximization of total wealth – economic efficiency or total utility). But punishment also raises questions about distribution. What principle should determine criminal or tort penalties? Likewise compensation for misfortune, whether at the hands of nature or a harmful act of someone else; compensation is provided by insurance, social insurance, or tort penalties.
Utilitarian theory implies that distributions of goods (e.g., of income or wealth) should be based on maximization of utility, but this principle implies two other criteria: declining marginal utility of most goods, and incentive. A given amount of money has more utility to the poor than to the rich (i.e., the utility of money is marginally declining, that is, the slope of the curve relating utility to money decreases as the amount of money increases). Hence, other things being equal, utility would be maximized if we took from the rich and gave to the poor until everyone is equal. However, this would prevent the use of income as an incentive for work (and has “transaction costs” of its own). Hence, maximization requires a compromise between equality and contribution. Such a principle is useless for psychology experiments. Even if it were possible to calculate the optimum trade-off, ordinary people would have no way of knowing the result. However, experiments can show deviations from any such model, even nonutilitarian models that incorporate similar assumptions. Such deviations can be explained in terms of simple heuristics such as equality, or demonstrated by framing effects, such as those described earlier.
People sometimes prefer equality over maximization that involves lives rather than money.Footnote 12 Several studies (e.g., Ubel & Loewenstein, Reference Ubel and Loewenstein1996) have presented subjects with allocation dilemmas of the following sort: Two groups of 100 people each are waiting for liver transplants. Members of group A have a 70 percent chance of survival after transplantation, and members of group B have a 30 percent chance. How should 100 livers – the total supply – be allocated between the two groups? The simple utilitarian answer is “all to group A,” but only a minority of the subjects chose this allocation. People want to give some livers to group B, even if less than half. Many want to give half. Many people are willing to trade lives for fairness to the two named groups. Surely there is some third group that is not in the scheme at all, so inequality is inevitable. (Such results are found even when group membership is unknown to anyone: Baron, Reference Baron1995.)
8.2.3.2 Compensation and Deterrence in Tort Law and Criminal Law
Compensation is justified by declining marginal utility. If you have a house fire that requires construction work, or an illness that requires expensive treatment, your utility for money increases. You have an immediate need for more of it. Insurance, including medical insurance and social insurance (such as unemployment compensation) is a scheme for transferring money from those who have a lower utility, those who pay insurance premiums or taxes, to those who have a higher utility. Like progressive taxes, compensation should be limited when its availability can provide incentive for reducing risks. For example, fire insurance could require installation of fire extinguishers. Health insurance may cost more for smokers, but this is consistent with utilitarianism only if this incentive effect actually causes people not to smoke.
Tort penalties and criminal penalties are justified by incentive effects, that is, by the principle of deterrence. If you know that you are likely to be punished or fined for some behavior (including omissions, in some situations), then you are less likely to engage in that behavior. Penalties “send a message” to the person penalized and to others: “Don’t do this.”Footnote 13
Experiments (e.g., Baron & Ritov, Reference Baron and Ritov1993) are hardly needed to demonstrate that these principles are not followed in the real world. Many people still advocate health insurance in which people pay premiums according to their individual “risk” even when that risk is beyond the individual’s control, hence not subject to incentive effects. (This practice is partially banned in the United States.) Compensation is often provided to relatives for “wrongful death” (but not for other deaths), even when the death in question reduces the utility of money for them. And tort penalties are often levied even when the incentive effect leads to more harmful behavior (e.g., a lawsuit for side effects of a beneficial vaccine with rare side effects causes the company to withdraw the vaccine entirely; see Baron & Ritov, Reference Baron and Ritov1993).
Likewise, criminal punishments are often inconsistent with the principle of deterrence (Carlsmith et al., Reference Carlsmith, Darley and Robinson2002). Preferences for penalties are based more on the heinousness of the offense than on factors that should affect deterrence. For example, by utilitarian theory, the severity of punishment should be higher when the probability of detection is lower; this way, potential offenders are risking a larger loss in the unlikely event that they get caught. But probability of detection plays little role. Littering is lightly penalized but rarely detected.
8.2.4 Comparison of Moral Judgments to Consequence Judgments
In some cases, such as the vaccination case described, the utilitarian answer is fairly clear. When the answer cannot be specified, a simple alternative approach for experimenters is to ask the subject which option, on the whole, has the best overall consequences for everyone affected. When the subject gives one answer to that question and a different answer to the question of what should be done, then we have pretty good evidence that the subject is giving a nonutilitarian answer, and we can go on to explore the reasons for this discrepancy.
Baron et al. (Reference Baron, Ritov and Greene2013) asked subjects what was best for their nation (or national group, in the case of Arab and Jewish Israelis), what was best on the whole, what was best for the other group (in Israel), and what their moral duty was. Many subjects thought it was their duty to go against their own judgment of what was best on the whole, in the direction of parochialism (in-group bias), and they indicated that they would do their duty in a real vote.
Baron and Jurney (Reference Baron1993) asked subjects if they would vote for various reforms. In one experiment, 39 percent of the subjects said they would vote for a large tax on gasoline (to reduce global warming). Of those who would vote against the tax, 48 percent thought that on the whole it would do more good than harm; this group was thus admitting that they were not following their own perception of overall consequences. Of those subjects who would vote against the tax despite judging that it would do more good than harm, 85 percent cited the unfairness of the tax as a reason for voting against it (for instance, the burden would fall more heavily on people who drive a lot), and 75 percent cited the fact that the tax would harm some people (e.g., drivers). The latter subjects were apparently unwilling to harm some people in order to help others, even when they see the benefit as greater than the harm. This effect may be related to “omission bias.” Unlike other results summarized here, the principle in question is nonutilitarian but is endorsed by other moral theories. Yet, its application in the real world can make things worse.
8.2.5 Isolation Effects
In “isolation” effects, people attend only (or primarily) to data or issues immediately before them (Camerer, Reference Camerer, Kahneman and Tversky2000; Kahneman & Lovallo, Reference Kahneman and Lovallo1993; Read et al., Reference Read, Loewenstein and Rabin1999). These effects are related to, or identical to, what others have called a focusing effect (Idson et al., Reference Idson, Chugh, Bereby-Meyer, Moran, Grosskopf and Bazerman2004; Jones et al., Reference Jones, Frisch, Yurok and Kim1998; Legrenzi et al., Reference Legrenzi, Girotto and Johnson-Laird1993). People know about indirect effects but do not consider them, or do not consider them enough. The idea came from the theory of mental models in reasoning (Legrenzi et al., Reference Legrenzi, Girotto and Johnson-Laird1993): People reason from mental models, and when possible they use a single, simple, model that represents just the information they are given. Other factors are ignored or underused.
McCaffery and Baron (Reference McCaffery and Baron2006) found apparent isolation effects in evaluation of taxes and other policies. For example, people prefer “hidden” taxes, such as a tax on corporations, without thinking about where the money comes from (employees, consumers, stockholders). If people are asked who actually pays, they realize that such taxes are not “free.” Caplan (Reference Caplan2007) reports similar effects for policies such as rent control, which have an immediate desirable effect on prices but an undesirable secondary effect on the supply of housing. Often people seem to evaluate policies (such as long prison sentences) in terms of their intended effects, even if those are not their main effects. These evaluations, working through the political system, affect real policies.
8.3 Moral Rules and Intuitions
Many demonstrations of nonutilitarian biases, or their cognitive bias cognates, seem to result from intuitive responses rather than any sort of reflection. At issue is whether these biases would be reduced if people engaged in more thinking, or more thinking of a certain sort.
Hare (Reference Hare1981; see www.utilitarianism.net/ for additional citations) proposed a related account. In defending utilitarianism, he proposed a two-level theory of moral thinking, with an intuitive and “critical” level. The critical level, which is a normative model in the sense discussed earlier, is utilitarian and is rarely approximated in human thinking but also rarely needed. Optimal decisions at this level are what would result if the decision maker could sympathetically represent to herself the preferences of all those affected and reach a decision as if the conflicts among their preferences were conflicts among her own preferences. At this level the distinction between case-by-case decisions and moral principles almost disappears, since each case specifies the decision for other cases that are similar in morally relevant ways and becomes a principle for just those cases (however few or many there may be). The principles and decisions accepted through such idealized reflection are those that would be rationally accepted by anyone, even if that person in real life would lose from application of the principle. The principles are thus universal, but each principle (unlike heuristics or intuitive rules) need not be simple. It includes all morally relevant features of a given case (those that could in principle affect the choice).
Hare argues that the term “moral” implies such universal agreement (an idea he attributes to Kant). Roughly, the idea is that we would balk at calling a principle “moral” if I applied it when you were in one position (e.g., the loser) and I was in another (the winner) but would not apply it if we switched positions (including with each position all its relevant features). This idea is embodied in the “veil of ignorance” (Rawls, Reference Rawls1971), which is a possible prescriptive intervention (Huang et al., Reference Huang, Bernhard, Barak-Corren, Bazerman and Greene2021).
Hare’s intuitive level, as I noted earlier, consists of intuitions that could be prescriptive principles worth following, or at least worth considering. But it also includes intuitions that may be the cause of harmful biases, such as “do no harm,” if it is understood as referring to actions but not omissions.
8.3.1 Intuitions and Dual Systems
Many approaches to reasoning have relied on the idea of dual systems, intuitive and reflective, with at least the intuitive level being similar to Hare’s. That system, by various accounts, is automatic (uncontrollable but also free of demands on cognitive resources), driven by emotion (or affect), and based on associations rather than rules. The reflective system is controllable and requires some effort. Because it is controllable, it may or may not become active after the intuitive system has begun its work. In principle, if the subject knows that reasoning is required, the reflective system could begin right away. Kahneman (Reference Kahneman2011) argues that a corrective version of this theory, in which reflection begins after results of intuition are available and can function to correct the intuition, is relevant to a variety of tasks studied in the heuristics-and-biases tradition. The corrective theory has also been proposed as an account of moral judgment by Joshua Greene (e.g., Reference Greene and Sinnott-Armstrong2008, p. 44, although elsewhere Greene is less specific about the ordering of events in time).
Several lines of evidence seem to support the dual system theory for moral judgment. First, response times (RTs) for “personal” dilemmas, those that involve direct killing, such as the footbridge dilemma, are longer, especially when subjects choose the utilitarian option.
A common finding in choice tasks is that RT is longer when the response is rarely made or when the options are similar in attractiveness (hence conflicting). These factors alone can explain RT differences found. Note that the corrective model implies that RT is longer for utilitarian than for deontological responses when their probability is equal (which is also where the two responses are maximally conflicting). Baron and Gürçay (Reference Gürçay and Baron2017), in a meta-analysis of 26 experiments, estimated this RT for each response by assuming that each subject had an “ability” to make utilitarian responses, and each dilemma had a “difficulty” for making that response. (Thus, the footbridge problem is more “difficult” than the simple trolley case.) The two choices would be equally likely when ability was equal to difficulty, according to our measures. A plot of RT for each choice as a function of ability minus difficulty indeed showed the longest RT when this difference was zero, but, at this point, the utilitarian responses took no longer than the deontological responses. Rosas et al. (Reference Rosas, Bermúdez and Aguilar-Pardo2019) also found that RTs were determined mainly by conflict. These results are inconsistent with any form of the corrective model.
Baron and Gürçay (Reference Gürçay and Baron2017) also noted that subjects who made more utilitarian responses had longer RTs on everything, a result consistent with the claim that reflection-impulsivity is correlated with utilitarian responding. Why this happens may depend on developmental processes that have already occurred before the experiment. For example, people who are generally reflective may come to favor utilitarian solutions over time.Footnote 14
Other results concern the effects of time pressure or cognitive load, which, in some studies, seem to affect utilitarian responses but not deontological responses. A general problem with these studies is that the effects vary for different dilemmas, not only in magnitude but also in direction (as also found by Gürçay & Baron, Reference Gürçay and Baron2017, despite finding no overall effect of time pressure vs. instructions to reflect). For example, to deal with load or time pressure, subjects may skip or skim the less salient parts of the printed description, and those may vary with how the dilemma is described. Researchers should at least test effects in ways that take into account the variance across dilemmas as well as across subjects, and most researchers have not done this (for an exception, see Patil et al., Reference Patil, Zucchelli, Kool, Campbell, Fornasier, Calò and Cushman2021), as well as trying different ways of ordering the information in the dilemma.
These sorts of results concerning time pressure and cognitive load have been difficult to replicate (e.g., Bago & De Neys, Reference Bago and De Neys2019). Rosas and Aguilar-Pardo (Reference Rosas and Aguilar-Pardo2020) found that utilitarian responses can occur under extreme time pressure. Moreover, studies that track the position of the pointer (mouse) during experiments with moral dilemmas do not show any tendency to switch from utilitarian to deontological responses during the time (usually 10–20 seconds) that the subject deliberates (Gürçay & Baron, Reference Gürçay and Baron2017; Koop, Reference Koop2013).
In sum, the most plausible account is that, when presented with dilemmas that pit utilitarian and deontological responses against each other, people are aware of the conflict as soon as they understand the dilemma. Yet more reflective people, for as yet unknown reasons, are more likely to favor the utilitarian resolution to the conflict. This kind of account in terms of individual differences in reflection is not far from Greene’s two-system account, but it does not assume any sequential effects involving suppressing an early response by a late one, so it is thus consistent with the known results, and with versions of dual system theory that assume that the systems work in parallel rather than sequentially (e.g., Sloman, Reference Sloman1996). It is clear by any account that people differ in some sort of reflectiveness, and these differences are related to differences in at least some moral dilemmas (Patil et al., Reference Patil, Zucchelli, Kool, Campbell, Fornasier, Calò and Cushman2021).
8.4 Future Directions
A lot remains to be known about moral reasoning. The reader who has gotten this far will not be surprised that I think this topic should be part of cognitive psychology, which has been studying reasoning more generally for well over a century. Many of the methods of psychology remain to be fully applied to moral reasoning. But moral reasoning is important for practical purposes too. It is tied up with politics and public policy.
Political judgments of citizens are often moral judgments (see Chapter 22, this volume). These merit special attention because the actions or omissions of citizens affect other citizens and noncitizens at home and abroad. Many of the world’s problems, within and among nations, can be traced to policies approved by citizens. The utilitarian argument I made earlier applies here. If citizens collectively follow nonutilitarian moral intuitions, then we should not be surprised if the final results they influence are deficient, for all those affected, everywhere.
Differences in thinking about politics arise in individual development (as studied by Adelson, Kohlberg, and many others; see, for example, Adelson, Reference Adelson1971, and Kohlberg, Reference Kohlberg1963) and in cultural evolution. Hallpike’s (Reference Hallpike2004) analysis, which is analogous to that of Kohlberg, suggests that something like developmental stages occurred over the course of cultural evolution, with the earlier stages still present. Early people, and those who still live as they did, and young children, do not distinguish morality, laws and social conventions, and etiquette. They are just “the way we do things.” With the growth of cities and writing, codified laws came to exist and were soon “written in stone” or in parts of what is now the Old Testament. Similar developments may occur in early adolescence (depending on culture; see Haidt et al., Reference Haidt, Koller and Dias1993). The development of a concept of morality, independent and outside of laws and conventions, came relatively recently in human history, possibly only a few thousand years ago, after writing became generally used. The concept of morality, like other concepts such as “science,” is not fully developed in human cultures. And the developments so far are not well understood by many people. For most who make the distinction between morality and convention now, it comes in adolescence. The existence of a concept of morality raises the possibility of rational thought about what it should be.
It is apparent that culture has a large effect not only on moral beliefs but also on how (or whether) people reason about them, or about other issues such as politics (Baron et al., Reference Baron, Isler, Yilmaz, Ottati and Stern2023). A question of interest is how cultural traditions persist over generations and over historical time (even within generations) and how they change. Attitudes toward homosexuality, for example, have changed enormously in the last 50 years, in some countries. And it is clear that there are cultural influences on beliefs about what good thinking is. One way to study cultural change over time is to examine written documents, both for their content and for the type of reasoning they exhibit. Some of this sort of work has been done (Suedfeld, Reference Suedfeld1985; Suedfeld et al., Reference Suedfeld, Guttieri, Tetlock and Post2003), but it has been confined to documented records of groups, such as legislators who make speeches, that are not representative of any larger cultural tradition.
It is clear that education can be designed to encourage rational thinking (e.g., Baron, Reference Baron1993). Liberal education at the university level is often explicit in its attempts to encourage questioning, consideration of diverse views, and understanding of the nature of expert knowledge. Many secondary schools do this too (e.g., Metz et al., Reference Metz, Baelen and Yu2020).
Several efforts have been made to teach moral thinking in a way that views it as a type of thinking rather than a set of rules. Kohlberg, in particular, encouraged widespread experimentation with moral discussion in high schools around the world (Snarey & Samuelson, Reference Snarey, Samuelson, Nucci and Narvaez2008). Much of this work disappeared with Kohlberg’s death and with claims that his ideas were biased against women (claims that were consistently shown to be unfounded, as Snarey and Samuelson point out).
Education is one important domain where people’s thinking can be influenced. Others, probably to a lesser extent, are journalism and politics itself. Ultimately, individuals and cultures change from a variety of influences, and we cannot expect applied research on one domain or another to provide the key. Change is slow, but the world would benefit if people’s moral thinking became more rational.
