2. Morality, Prosocial Payoffs and Endogenous Preferences

Game-theoretic models have often ascribed other-regarding or prosocial preferences to players, thus invoking an element of morality either implicitly or explicitly. For excellent surveys of related theoretical and experimental literature, please refer to Cooper and Kagel (Reference Cooper, Kagel, Kagel and Roth2015) and Capraro et al. (Reference Capraro, Halpern and Perc2023).

Several models and experiments in game theory have focused on the concept of fairness. Fehr and Schmidt (Reference Fehr and Schmidt1999) and Bolton and Ockenfels (Reference Bolton and Ockenfels2000) include players who are concerned not just about their own income but about the distribution of payoffs as well. Another approach, adopted by Levine (Reference Levine1998) and Andreoni and Miller (Reference Andreoni and Miller2002), has been to model altruism (or spitefulness) where players care about their own monetary reward as well as the monetary reward of others. This approach to modelling preferences is standard; players acquire their preference structures prior to the game and proceed to play strategically based on those predetermined preferences. As a result, players in this general set-up do not actively choose whether to be moral during the interactive game. To put it in the context of the plot device in the title of this paper, the predetermined preferences of the players fully specify whether to listen to the angel or whether to listen to the devil prior to the game, and, with that decision already made, the player then proceeds to play according to the standard concept of rational utility maximization.

Several alternative approaches to endogenizing moral preferences have been explored in the literature. One approach involves invoking evolutionary stability. Alger and Weibull (Reference Alger and Weibull2010, Reference Alger and Weibull2013, Reference Alger and Weibull2016, Reference Alger and Weibull2019), among others, adopt this approach. With this approach, the extent of a player’s morality is captured by the parameter $\kappa $ in Alger and Weibull (Reference Alger and Weibull2013), and is determined endogenously based on evolutionary stability. Guttman (Reference Guttman2000), Bar-Gill and Fershtman (Reference Bar-Gill and Fershtman2005) and Poulsen and Svendsen (Reference Poulsen and Svendsen2005) also apply an evolutionary approach where players with alternative types of moralities participate in some interactive game and evolutionary forces result in the prevalence of players with prosocial preferences. More generally, Heifetz et al. (Reference Heifetz, Shannon and Spiegel2007) show that in almost every strategic interaction, payoff maximization cannot be justified by appealing to evolutionary arguments. Thus, some distortions away from payoff-maximizing behaviour will persist if preferences were to evolve endogenously. These approaches are compelling because they allow preferences to be endogenously determined through an evolutionary process. However, even in the evolutionary approach, the preferences a player acts upon are ultimately exogenous to the player herself; since they embed morality solely in the preference structure, these approaches cannot model moral agency. Moreover, rather than allow for a philosophically rooted morality, the evolutionary approach suggests that morality emerges as a proximate concept solely to serve ultimate material interests either for the player or for some appropriately specified collective of which the player is a constituent.

Rotemberg (Reference Rotemberg1994) pioneers an approach in which players themselves choose their level of altruism using a model with a two-stage framework. In the first stage, players choose a level for an altruism parameter which features in their utility function in the second stage. However, in the first stage, players choose the altruism parameter based on their desire to maximize their own material payoffs. This approach is appealing, and Rotemberg (Reference Rotemberg1994) motivates it by alluding to evolution as the mechanism at play in the first stage, or the possibility of distinct inner and outer selves where the materialistic inner self chooses an altruism parameter for the outer self that ultimately leads to maximized material gain for the agent. In this regard, despite endogenizing altruism, Rotemberg (Reference Rotemberg1994), like the evolutionary approaches above, also constructs morality as a proximate concept that is chosen solely to serve materialistic ultimate interests.

Similarly, Sally (Reference Sally2001) constructs a model of sympathy where the extent of fellow-feeling experienced by a player towards another player depends on the physical and psychological distances between the two players and can evolve over time and through repeated interactions. Importantly, Sally (Reference Sally2001) allows players to choose the level of their sympathy based on the extent of fellow feeling between them. Since this approach endogenizes sympathy, it is similar to the one we adopt in this paper. However, agents in our approach can explicitly choose among distinct moral principles (with the possibility of mixed strategies over those principles), whereas agents in Sally (Reference Sally2001) choose a level for their sympathy without the possibility of mixed strategies over levels of sympathy. The use of mixed strategies allows our framework to feature agents who display ‘partial’ or ‘mixed’ moral behaviour without introducing parameters that determine the extent of an agent’s morality as in Sally (Reference Sally2001) or Rotemberg (Reference Rotemberg1994).

Another approach involves endogenizing preferences as cultural traits or learned influences on behaviour that coevolve with cultural and economic institutions. This approach is explored in depth in Bowles (Reference Bowles1998) and is insightful because it endogenizes not just preferences but also cultural and economic institutions which can be explored collectively as a co-evolving dynamic process. Although players’ preferences in Bowles (Reference Bowles1998) are directly related to their institutional environment, the focus is on the co-evolution of preferences and culture rather than the moral agency of players engaged in moral decision making. The approach we develop in this paper, on the other hand, allows us to explicitly model moral agency during gameplay.

In yet another approach, the psychic preferences of players are modelled as conditional on beliefs. Geanakoplos et al. (Reference Geanakoplos, Pearce and Stacchetti1989), a seminal paper in this literature, introduces psychological games in which the rewards depend not only on actions, but also on the beliefs of the players, including beliefs about the beliefs of other players. In this approach, as soon as a player determines her beliefs, her preference structure is fully established, and she proceeds to play rationally based on those preferences and beliefs. As a result, players do not actively choose whether to be moral during the interactive game, and the core moral quandary represented by the angels and devils metaphor no longer exists. In related literature, beliefs have been modelled as a choice variable for the agent. Rabin (Reference Rabin1994), among others, adopts this approach. He models players as having the ability to change their beliefs regarding what constitutes immoral activity in order to reduce their cognitive dissonance while engaging in immoral activity. While this approach is somewhat similar to ours since it allows for a sort of moral agency, we believe our modelling framework develops a viable alternative approach that focuses much more centrally on the role of moral agency during gameplay.

In another related approach, preferences are endogenized by allowing players to base their preferences on their chosen identities. This approach builds on Sen (Reference Sen1977) and Hirschman (Reference Hirschman1984) who make the case for distinguishing between preferences and meta-preferences (or preferences among preferences) and consider how a player may choose among preferences based on their meta-preferences. Building on this insight, McCrate (Reference McCrate1988) discusses how gender identities could result in differences between the endogenously determined preferences of men and women. This aspect of identity affecting preferences is central to the models explored by Akerlof and Kranton (Reference Akerlof and Kranton2000, Reference Akerlof and Kranton2005). In the models developed by Akerlof and Kranton (Reference Akerlof and Kranton2000, Reference Akerlof and Kranton2005), the preferences of players can change based on their identities, and an agent’s identity is itself an ex ante expected utility maximizing choice variable for the agent. Similarly, Bénabou and Tirole (Reference Bénabou and Tirole2011) model players as having the ability to invest in a moral identity which subsequently shapes individual and collective behaviour. As with Rabin (Reference Rabin1994), Rotemberg (Reference Rotemberg1994) and Sally (Reference Sally2001), this identity-based approach is related to ours since it too allows for a sort of moral agency. Consequently, our approach can be considered an alternative that directly highlights the moral autonomy of players.

It may be helpful to note that some authors have modelled morality by introducing alternative equilibrium concepts in place of Nash equilibrium. For example, Roemer (Reference Roemer2010) introduces the idea of a Kantian equilibrium, while Heap and Ismail (Reference Heap and Ismail2022) introduces the concept of a no-harm equilibrium. In our framework, too, the underlying game undergoes a transformation to include moral choices; however, we employ the standard concept of Nash equilibrium to solve the transformed game.

Rather than critiquing the standard solution of Nash equilibria, several approaches have challenged the very use of prosocial outcome-based preferences to model prosocial behaviour. Capraro et al. (Reference Capraro, Halpern and Perc2023), for instance, recommends the use of language-based preferences over outcome-based preferences. Another important and vast strand of the literature suggests that social norms offer a better explanation for prosocial behaviour than prosocial preferences. For a detailed survey of the literature on social norms, we refer the reader to Bicchieri et al. (Reference Bicchieri, Muldoon, Sontuoso, Zalta and Nodelman2023). While there are several theories on what constitutes a norm and how social norms originate, broadly, they can be defined as group- and game-specific rules of behaviour (Bicchieri et al. Reference Bicchieri, Muldoon, Sontuoso, Zalta and Nodelman2023). As such, they can be represented in terms of “norm-driven preferences.” For instance, Kimbrough and Vostroknutov (Reference Kimbrough and Vostroknutov2022, Reference Kimbrough and Vostroknutov2023a,b) show how different theories of moral reasoning (ranging from simple abstract moral principles to complex situation-specific psychologically founded principles) and insights from cooperative game theory can result in norms that can be expressed in terms of context-specific norm-dependent preferences. A third approach, presented by Smith and Wilson (Reference Smith and Wilson2019), formally incorporates Adam Smith’s insights into game theory’s analytical framework. This approach emphasizes that human actions are guided by social norms and the desire for mutual sympathy, rather than pure self-interest. Consequently, according to Smith and Wilson (Reference Smith and Wilson2019), utility functions based on outcomes are inadequate to explain human behaviour, even when they are augmented to incorporate standard behavioural economics assumptions. In order to successfully model human behaviour, Smith and Wilson (Reference Smith and Wilson2019) suggest that models need to consider two key aspects: (1) its crucial dependence on context and (2) humans’ tendency to judge the moral stance of others while caring about how their own moral stance would be judged, even in the absence of actual judgements. Our framework can be readily extended to accommodate all these approaches. Although we use the term “moral agency” and refer to preferences driven by “moral principles” in this paper, and although our examples use simple “morally motivated preferences”, our framework is amenable to other types of preferences, including language-based preferences, context-specific norm-dependent preferences, and preferences that account for moral judgements of and by others.

Ultimately, in this paper, our goal is to present a general framework for modelling moral agency and to explain the advantages and distinctive features of this general framework. Further, we present two example models to illustrate how the framework can be applied. In addition to the sample models we present in this paper, the model analysed in Studtmann and Gouri Suresh (Reference Studtmann and Suresh2021) could also be considered an application of our general framework. Studtmann and Gouri Suresh (Reference Studtmann and Suresh2021) study a model where agents choose between Kantian universalizing and selfishness in the context of a prisoner’s dilemma game. We too use prisoner’s dilemma in our first example, but we modify it to incorporate empathy as the source of moral reasoning. In our second example, we use Kantian universalizing, but we apply it in the context of the public goods game.

3. General Framework: Introducing Moral Principles as Decision Variables

In the standard version of game theory, players choose among the set of strategies available to them. Their choices in conjunction with the choices of other players determine their payoffs. In the transformation we are proposing, players choose simultaneously not just among the available strategies of the original game but also among a set of alternative moral principles through which to evaluate outcomes. With the transformation, a player’s payoffs depend not just on the specific strategies of the original game chosen by all players, but also on the specific moral principles chosen by all players.

Consider a standard normal-form representation of a non-cooperative game which specifies (1) the number of players in the game, (2) the strategies available to each player and (3) the payoff received by each player for each combination of strategies that could be chosen by the players. In the general version of this standard game, assume that there are $N$ players numbered from $1$ through $n$ . Let ${S_i} = \left\{ {{s_{1i}},{s_{2i}},{s_{3i}}, \ldots } \right\}$ denote the strategy space available to player $i$ and let ${s_i}$ denote an arbitrary member of this set. Let $\left( {{s_1}, \ldots, {s_n}} \right)$ denote a particular combination of strategies, one for each player, and ${u_i}$ denote player $i$ ’s von Neumann–Morgenstern utility levels associated with the outcome arising from strategies $\left( {{s_1}, \ldots, {s_n}} \right)$ . Collectively this standard game can be denoted by $g = \left[ {N;\left\{ {{S_1}, \ldots, {S_N}} \right\};\left\{ {{u_1}\left( \cdot \right), \ldots, {u_n}\left( \cdot \right)} \right\}} \right]$ .

Any standard game such as this can be transformed to allow for moral agency. In order to do this, a morality space needs to be defined for each player which describes the set of alternative moral principles that the player can choose amongst. Let ${M_i} = \left\{ {{m_{1i}},{m_{2i}},{m_{3i}}, \ldots } \right\}$ denote the morality space or set of alternative moral principles available to player $i$ and let ${m_i}$ denote an arbitrary member of this set. Recall that with this transformation, players choose not just their strategy but also their moral principle. The transformed strategy space available to player $i$ will therefore be the Cartesian product of the morality space and the original strategy space. For purposes of distinguishing between the transformed strategy space and the original strategy space, we will refer to the original strategy space, ${S_i}$ , as the action strategy space and its elements, $\left\{ {{s_{1i}},{s_{2i}},{s_{3i}}, \ldots } \right\}$ as action strategies. Let the transformed strategy space be denoted by ${T_i} = {M_i} \times {S_i}$ . Let ${t_i}$ denote an arbitrary member of this set. Let $\left( {{t_1}, \ldots, {t_n}} \right)$ denote a particular combination of transformed strategies, one for each player, where each transformed strategy ${t_i}$ includes both an action strategy ${s_i}$ and a moral principle ${m_i}$ . With the transformation, the morally motivated payoffs for each player will depend on the transformed strategies of all players which will entail not just the action strategies of all players but also the moral principles of all players. Let ${v_i}$ denote player $i$ ’s morality-dependent von Neumann–Morgenstern utility levels associated with the outcome arising from transformed strategies $\left( {{t_1}, \ldots, {t_n}} \right)$ . Here, ${v_i}$ should be interpreted as a function that encodes the moral considerations used to produce the morally motivated payoffs that form the basis for decision-making by agents with moral agency. For each moral principle, ${m_i}$ , the function ${v_i}$ encodes a value for each situation described by the agent’s own action strategy, counterparty action strategies, and counterparty moralities. Suppose there are two different situations, where the first situation is described by agent $i$ action strategy ${s_i}$ , counterparty moralities ${m_{ - i}}$ , and counterparty action strategies ${s_{ - i}}$ while the second situation is described correspondingly by $s{{\rm{'}}\!_i}$ , $m{{\rm{'}}_{\!\! - i}}$ and $s{{\rm{'}}_{\!\! - i}}$ . If the former situation, $\left( {\left( {{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ , is better than the latter situation, $\left( {\left( {s{{\rm{'}}\!_i}} \right),\left( {m{{\rm{'}}_{\!\! - i}},{s{\rm{'}}_{\!\! - i}}} \right)} \right)$ , from the perspective of moral principle ${m_i}$ , then ${v_i}\left( {\left( {{m_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right) \gt {v_i}\left( {\left( {{m_i},s{{\rm{'}}\!_i}} \right),\left( {m{{\rm{'}}_{\!\! - i}},s{{\rm{'}}_{\!\! - i}}} \right)} \right)$ . The function ${v_i}$ also enables the inter-morality comparisons facilitated by our framework. For instance, if ${m_i}$ and $m{{\rm{'}}_i}$ be two moral systems considered by agent $i$ , inter-morality comparisons between ${m_i}$ and $m{{\rm{'}}_{\!\!i}}$ for some situation $\left( {\left( {{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ can be made by comparing the values of ${v_i}(\left( {{m_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)$ and ${v_i}\left( {\left( {m{{\rm{'}}_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ . If ${v_i}(\left( {{m_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right) \gt {v_i}\left( {\left( {m{{\rm{'}}_{\!\!i}},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ , then agent $i$ perceives morality ${m_i}$ to be superior to morality $m{{\rm{'}}_i}$ for the situation given by $\left( {\left( {{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ . It is worth emphasizing that the construction of ${v_i}$ therefore requires assigning morally motivated payoffs that are appropriately comparable across all the moral principles under consideration in ${T_i}$ . Since the construction of ${v_i}$ is of central importance in the application of our framework, we discuss it further in section 4 which addresses inter-morality comparisons in more detail.

The transformed game can be denoted by $\tau = \left[ {N;\left\{ {{T_1}, \ldots, {T_N}} \right\};\left\{ {{v_1}\left( \cdot \right), \ldots, {v_n}\left( \cdot \right)} \right\}} \right]$ . Since the transformed game $\tau $ has the same algebraic structure as $g$ , all standard properties, theorems and proofs for standard games can be extended to transformed games as well. In particular, the transformed game can feature mixed transformed strategies. Moreover, if $N$ is finite and ${T_i}$ is finite for every $i$ , then there exists at least one Nash equilibrium, possibly involving mixed transformed strategies. It might be helpful to note that the framework suggested here is very general and can accommodate several classes of models with multiple moral principles. While the examples in this paper demonstrate the framework in the context of one-shot perfect information simultaneous games and use Nash equilibrium as the solution concept and outcome-based preferences reflecting pure moralities, the framework can be extended to sequential games, Bayesian games, repeated games, etc. as well as alternative solution concepts and alternative approaches to preferences, including preferences with parameterized levels of morality, language-based preferences and norm-driven preferences.

The structure of this framework allows us to construct a formal definition for moral agency. A player has moral agency if the set of moral principles they can choose from is non-trivial (i.e. when $\left| {{M_i}} \right| \ge 2$ ).Footnote ² We will refer to players with $\left| {{M_i}} \right| \ge 2$ as moral agents since such players have moral agency.

In general, note that the payoffs for a moral agent could depend on the action strategy and moral principle chosen by the agent as well as the action strategies and moral principles chosen by other moral agents. In other words, in our general framework, the payoff for moral agent $i$ is given by ${v_i}\left( {{t_i},{t_{ - i}}} \right)$ which can be rewritten as ${v_i}\left( {\left( {{m_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ . The mathematical representation of our framework allows one to characterize different types of moral agency, which have played significant roles in the history of ethical theorizing. Moral philosophers have often treated an ideal moral agent as having intentions to act morally that are independent of how other agents act. This is, for instance, one of the implications of Kant’s view that an agent ought to act out of reverence for the moral law. Our framework allows one to characterize two different types of independence. First, an agent may adopt a moral principle that is independent of the moral principles other agents adopt. We call such a moral principle unconditional.Footnote ³ If a moral agent were to choose a moral principle that is unconditional, the moral motivations of all other players become irrelevant to her morally motivated payoffs. In other words, a moral agent choosing an unconditional morality transcends reciprocity, since her own morally motivated payoffs are unaffected by the moral motivations of other agents. Second, and more strongly, an agent may adopt a moral principle that is independent of both, the moral principles and the action strategies that other agents adopt. We call such a moral principle strictly deontological. If a moral agent were to choose a strictly deontological moral principle, both moral principles and action strategies adopted by other players become irrelevant to her morally motivated payoffs. The formal characterizations of these two types of independence are as follows.

• • Unconditional Morality: If a moral principle, ${c_i}$ , satisfies the property of unconditional morality, then ${v_i}\left( {\left( {{c_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ , will necessarily be independent of ${m_{ - i}}$ , for all ${s_i}$ and ${s_{ - i}}$ .Footnote ⁴

• • Strictly Deontological: If a moral principle, ${d_i}$ , is strictly deontological, the payoffs associated with it, ${v_i}\left( {\left( {{d_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ , will necessarily be independent of ${m_{ - i}}$ and ${s_{ - i}}$ for all ${s_i}$ .Footnote ⁵

While moral agents get to choose from a non-trivial set of moral principles, we would like to emphasize that the set of moral principles is itself exogenous and will depend on higher-order considerations. Likewise, the values of the function, ${v_i}$ , used to encode these moralities also depend on higher-order considerations. This set of moral perspectives and the specific functions used to encode them could depend on culture, evolution, the salient identities of the moral agent and other players, the relationships between them, the emotional state of mind of the moral agent, the precise framing and context of the game, etc. Consequently, we believe that researchers should be judicious in choosing the appropriate exogenous set of moral principles based on the specific context of the game being modelled.Footnote ⁶ As an example, in a prisoner’s dilemma game being played by two moral agents who know and care for each other, the moral tussle in the agents’ minds could be between empathy (“what impact would my action have on my friend?”) and selfishness (“what’s in it for me?”). On the other hand, in the public goods game, where other players are a larger group of unknown individuals, the moral perspective might be founded on Kantian universalizing (“what if no one contributed to the public good?”) with selfishness (“what’s in it for me?”) being the alternative.

In section 5 of the paper, we develop and analyse an example of a symmetric prisoner’s dilemma where the morality space of both players includes two alternatives: selfishness and empathy. In the unique symmetric Nash equilibrium of this game, the extent of cooperation between moral agents depends on the payoffs of the game, such that when the benefit of cooperation to the other player is greater, cooperation increases, and when the cost of cooperation to the self is greater, cooperation decreases. In section 6 we study a standard $n$ -player public goods game transformed such that all players are moral agents with the ability to choose between selfishness and Kantian universalizing. In the unique symmetric Nash equilibrium of this game, moral agents contribute to the public good with a positive probability increasing in the multiplication factor and decreasing in the number of players.Footnote ⁷

In addition to exercising care while determining the set of moral principles available to moral agents, researchers using our framework should also be judicious in how they operationalize those moral principles. The following section delves into this topic.

4. Inter-morality Comparisons

While our framework enables endogenous inter-morality comparisons, these comparisons presuppose that a moral agent has (i) identified the relevant moral principles for a given context and (ii) determined an appropriate scaling for evaluating them in that context. In other words, although inter-morality comparisons are conducted endogenously within our framework, they rely on higher-order considerations that guide the selection and relative weighting of moral principles. Consequently, our framework requires agents to have moral principles that can be conceptualized as functions that take into account the actions of all agents (which, in turn, yield the material outcomes of the game) and the morality of the other agents to provide real numbers based on the content of the moral principle itself along with scaling considerations across moral principles. Every moral principle ${m_i}$ that can be modelled using this framework should yield a real number for each situation $\left( {\left( {{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ as encoded by the function ${v_i}\left( {\left( {{m_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ and these real numbers should be comparable across the other moral principles being considered by the agent. Since several philosophically important moral principles such as Kantian deontology, empathy, average-consequence utilitarianism, selfishness, etc. can be similarly conceptualized as functions, our framework can be used for inter-morality comparisons among any of them. In Appendix A, we discuss these moral principles in particular as well as entire classes of such moral principles.

Ensuring that morally motivated payoffs are meaningfully comparable across different moral principles requires careful attention to scaling. This requires thoughtfully scaling payoffs across moralities. Just as models involving interpersonal utility comparisons must establish a consistent scale across individual utilities, our framework demands a systematic approach to scaling morally motivated payoffs. While this might seem as problematic as interpersonal utility comparisons, we argue that inter-morality comparisons are more justifiable because they occur within the minds of individual moral agents rather than across the minds of different people. It is one thing to say that Ann can prefer moral outcomes to selfish outcomes; it is quite another to compare Ann’s preference for apples to Bob’s preference for apples. Nonetheless, our framework does require a careful scaling of payoffs across moral principles based on higher-order considerations. We discuss scaling issues in more detail in Appendix A.

When applying this framework, it is also important to recognize that while the set of appropriately scaled morally motivated payoffs described by ${v_i}$ across the entire transformed strategy space ${T_i} = {M_i} \times {S_i}$ is robust to positive affine transformations, such transformations must be applied consistently across the entire transformed strategy space rather than selectively to a subset of strategies associated with certain moral principles. In standard game theory, a game remains unchanged when all payoffs for a player are adjusted by the same positive affine transformation across all available strategies. Similarly, in our framework, since the transformed strategy space includes both moral principles and action strategies, any positive affine transformation must be applied uniformly across the entire transformed space to preserve the structural integrity of the model and ensure meaningful inter-morality comparisons.

It might help to illustrate the construction of the function ${v_i}$ through an example. Suppose the situation under consideration is based on a standard game of prisoner’s dilemma where Agent $i$ is cooperating while her counterparty, Agent $j$ is defecting. The material outcome in this situation is given by $S$ , the sucker’s payoff, for Agent $i$ , and $T$ , the temptation payoff, for Agent $j$ . Now suppose Agent $i$ evaluates this situation through two moral principles: empathy (defined as focusing entirely on the counterparty’s material payoff) and selfishness (defined as focusing entirely on one’s own material payoff). When Agent $i$ considers the moral principle of empathy, her morally motivated payoff equals $T$ , because empathy, by its very definition, requires focusing on the counterparty’s material payoff. However, when Agent $i$ views the same situation through the moral principle of selfishness instead, her morally motivated payoff equals $S$ , because selfishness, by its very definition, requires an agent to focus on one’s own material payoff. Since $T \gt S$ in prisoner’s dilemma, ${v_i}\left( {\left( {empathy,cooperate} \right),\left( {{m_j},defect} \right)} \right) \gt {v_i}\left( {\left( {selfishness,cooperate} \right),\left( {{m_j},defect} \right)} \right)$ .Footnote ⁸ Thus, Agent $i$ who is cooperating while her counterparty Agent $j$ is defecting gets a higher payoff when she adopts the moral principle of empathy relative to when she adopts the moral principle of selfishness. We develop the full version of this game and solve for all Nash equilibria in section 5.

Our approach to inter-morality comparisons is not substantively different from a common approach in the literature, where prosocial preferences are parametrically combined with self-interest, e.g. Levine (Reference Levine1998), Fehr and Schmidt (Reference Fehr and Schmidt1999), Alger and Weibull (Reference Alger and Weibull2013) and Schneider and Shields (Reference Schneider and Shields2022). While inter-morality payoff comparisons, like interpersonal utility comparisons, can be problematic, the standard parametric approach also relies on such comparisons, as it requires morally motivated prosocial payoffs and selfish payoffs to be placed on a common scale before they can be meaningfully combined mathematically. In other words, the very act of assigning relative weights to prosociality and self-interest presupposes that these moral considerations can be evaluated within a shared mathematical framework. Our approach differs in structure but not in substance: rather than assuming a parametric weighting of moral principles within a utility function, we embed the comparison process within the transformed strategy space, allowing moral agents to make these comparisons endogenously as part of their strategic decision-making. In both cases, payoffs across different moral principles must be made comparable before they can be meaningfully incorporated into game-theoretic analysis.

To illustrate the considerations involved in constructing our example above, we note the payoffs for both moral principles are tied directly to the material payoffs of the game. However, putting these together in the function ${v_i}$ implicitly assumes that neither morality offers any additional morally motivated value beyond the associated material payoffs, which is itself a higher-order consideration. More generally, morally motivated payoffs may reflect a range of higher-order considerations. For instance, if an agent holds that “empathy” has intrinsic value beyond the material payoff of the counterparty, then, in our example above, the payoff for ${v_i}\left( {\left( {empathy,cooperate} \right),\left( {{m_j},defect} \right)} \right)$ could be set to a value greater than $T$ , to accommodate this consideration. In a more extreme case, an agent might consistently privilege empathy over selfishness across all possible game situations. If these considerations lead to ${v_i}\left( {\left( {empathy,{s_i}} \right),\left( {{m_j},{s_j}} \right)} \right) \gt {v_i}\left( {\left( {selfishness,{s_i}} \right),\left( {{m_j},{s_j}} \right)} \right)$ for all combinations of ${s_i}$ , ${m_j}$ and ${s_j}$ , then such an agent would effectively eliminate selfishness from their feasible set of moral principles during gameplay. This situation would be analogous to an agent adopting the moral principle “always be empathetic” prior to gameplay.

We believe, however, that agents typically compare moral principles anew in each decision context, especially when choosing among alternative actions that entail different values under different moral principles. Thus, the examples we present to illustrate our framework avoid such pre-commitment scenarios and instead allow moral agents to choose jointly and simultaneously among alternative actions and moral principles at each instance of decision-making. We hope that the “angels and devils” metaphor successfully conveys the nature of this simultaneous choice among actions and moral principles, which in turn requires situation-specific inter-morality comparisons based on the content of those moral principles combined with higher-order considerations across them.

A comprehensive discussion of all the higher-order considerations required for inter-morality comparisons is beyond the scope of this paper. Instead, our goal is to develop a framework that enables endogenous comparisons of moral principles, assuming that the morally motivated payoffs have already been constructed based on the content of those moral principles and associated higher-order considerations. Beyond these foundational moral elements, agents may also be influenced by two other factors in their decision-making. First, adhering to certain decisions may confer additional psychic benefits or costs beyond their morally motivated values. Second, decision-making may be shaped not only by moral reasoning but also by context- and group-specific social norms rather than morals. We acknowledge the importance of these arguments and consider them in turn.

• • Psychic utility: It is certainly possible that agents could experience additional psychic joy (or sorrow) for choosing a certain moral principle. However, in our basic framework we assume that any additional psychic joy (or sorrow) experienced by an agent beyond the morally motivated payoffs does not factor into how the agent makes moral decisions. Therefore, in our basic framework, any such additional joy (or sorrow) is completely disregarded in the transformed game as we assume it to be irrelevant for moral decision-making. Consequently, for the moral principles we consider in our examples in the paper (empathy, Kantian universalizing and materialistic selfishness) and the appendices (average-consequence utilitarianism and total-consequence utilitarianism), we assume that morally motivated payoffs depend solely on the core values implied by those moral principles and disregard any considerations of additional psychic utility or disutility associated with those moral principles beyond the values assigned directly by those moralities.

  As a straightforward extension of our framework, if agents do factor in some additional psychic joy (or sorrow) of adopting a particular moral principle into their decision-making, that could be appropriately featured in the transformed game involving that moral principle by combining morally motivated payoffs and any additional psychically obtained payoffs as needed. However, we abstain from doing so in this paper as we restrict our attention to moral decision-making.

• • Social norms: If norms can be viewed as rules represented by norm-driven preferences that guide strategic interactions, they can be incorporated as “moral principles” in a straightforward extension of our framework. In this context, we would like to highlight the working papers by Kimbrough and Vostroknutov (Reference Kimbrough and Vostroknutov2022, Reference Kimbrough and Vostroknutov2023a, b), which explicitly derive norm-driven preferences, and Bicchieri et al. (Reference Bicchieri, Muldoon, Sontuoso, Zalta and Nodelman2023), which also permits a similar understanding of norms. This perspective of social norms allows them to be seamlessly incorporated in an extension of our framework, thereby allowing it to model scenarios where individuals choose between adhering to a norm or other moral principles, such as selfishness (which we regard as a form of morality), empathy, Kantian deontology, or others.

  A key feature of social norms that makes our framework particularly suitable for modelling them is their dependence on others’ choice of norms. As Bicchieri (Reference Bicchieri2005) argues, social norms are often conceptualized as being contingent, at least in part, on normative expectations about other agents’ adoption of similar norms. This contingency aligns well with our framework’s ability to model conditional moralities, where the value of adopting a principle can depend on others’ choices of moralities. For instance, our framework can readily incorporate Adam Smith’s idea of mutual sympathy, which Smith and Wilson (Reference Smith and Wilson2019) describe as involving judgments of others’ moral stances and concern for how one’s own stance would be judged. Similarly, Hankins and Thrasher (Reference Hankins and Thrasher2022: 644) characterize Smith’s view of mutual sympathy as the ‘independent pleasure we get when our sentiments concord with those of our fellows’. These norm-dependent principles can be modelled in our framework by allowing ${v_i}\left( {\left( {{c_i},{s_i}} \right),\left( {{m_{ - i}},{s_{ - i}}} \right)} \right)$ to depend on ${m_{ - i}}$ , representing how the value of adopting a norm varies based on others’ norm adherence.Footnote ⁹ Consequently, our framework could be used to model social norms similar to those suggested in Bicchieri (Reference Bicchieri2005) and allow agents to choose both their personal norms and how much to accommodate others’ normative positions within the model. To implement this, each combination of a personal norm and degree of consideration for others’ norm adherence could be treated as a distinct “moral” rule. Agents could then select among these combinations, effectively choosing both their personal norm and their level of responsiveness to others’ behaviour.

Although straightforward extensions of our framework can allow many diverse ideas to be modelled, in this paper we focus on modelling the calculations involved in choosing among moral principles for agents who take morality seriously. These agents compare moral principles based on the values they entail, combined with relevant higher-order considerations. With this foundation of inter-morality comparisons established, we now turn to two examples that illustrate how our framework can be applied.

5. Empathetic Prisoner’s Dilemma

We begin with a standard representation of a prisoner’s dilemma (PD) with two players, each of whom can choose to either cooperate $\left( c \right)$ or defect $\left( d \right)$ , i.e. the strategy space of each player is given by $S = \left\{ {c,d} \right\}$ . The payoffs associated with this standard PD are given in Table 1. In this game, if two cooperators interact, each gets the payoff $R$ , the “reward for mutual cooperation”. If a cooperator meets a defector the cooperator gets $S$ , the “sucker’s payoff”, while the defector gets $T$ , the “temptation of defection”. If two defectors interact, each obtains the payoff $P$ , the “punishment” of mutual defection. The game is a prisoner’s dilemma if $T \gt R \gt P \gt S$ .

Table 1.

Prisoner’s Dilemma (PD)

Table 1 Long description

A matrix with two players, each choosing between cooperation (c) or defection (d). The matrix has two rows and two columns, labeled with Player 1's and Player 2's choices. The cells contain payoff values: R for mutual cooperation, T for temptation of defection, S for sucker's payoff, and P for punishment of mutual defection. The payoffs follow the condition T > R > P > S.

For our example, we will assume that both players share a relationship that extends beyond mere acquaintance and therefore feel empathy towards one another. This view is somewhat aligned with the original framing of prisoner’s dilemma where both players are accomplices who have jointly engaged in a criminal activity. We will assume that both players are also capable of thinking selfishly. In terms of the metaphor in the title of this paper, each player hears their shoulder-angel recommending absolute empathy and their shoulder-devil recommending absolute selfishness.

The concept of empathy has deep roots in moral philosophy. In general, this moral approach is closely associated with the golden rule and is a prominent feature of several religious and ethical systems of thought. It can be summarized pithily as follows, “put yourself in the other person’s shoes”. Empathy is also closely related to the notion of sympathy as described in the works of Adam Smith and David Hume. Hankins and Thrasher (Reference Hankins and Thrasher2022: 640) presents a key differentiating element between Smith’s and Hume’s perspectives: ‘Among the significant differences between Hume and Smith on sympathy, is that the latter’s account, but not the former’s, allows us to sympathize with others while holding onto the idea that what we sympathetically feel is not what they in fact feel.’ Our modelling of empathy is more in line with the Smithian perspective than the Humean perspective.Footnote ¹⁰ In our model, an agent choosing empathy considers the material payoff of the counterparty rather than the modified (morally motivated) payoff experienced by the counterparty. What the counterparty actually feels, and therefore their affective experience (which motivates Hume’s version of sympathy, according to Hankins and Thrasher Reference Hankins and Thrasher2022) does not feature in the considerations of a moral agent choosing empathy in our model.

To illustrate our framework with a simple model, we treat empathy as absolute, i.e. the moral agent cares only about the other without any regard to self. In the case where there is only a single other player, the payoff for a moral agent with absolute empathy simply equals the material payoff of the other player. While this moral principle is typically conceptualized in the context of two-person interactions, it can be theoretically extended to multi-person situations too where moral agents choosing empathy receive a morally motivated payoff equal to the average of the material payoffs of all other players. Further, we model absolute empathy as being unconditional, i.e. for someone choosing absolute empathy as their moral principle, the moral motivations of other players are irrelevant for their own morally motivated payoffs.

The morally motivated payoffs for a moral agent choosing absolute empathy can be expressed as follows:

(1) $${v_i} \equiv {1 \over {N - 1}}\sum\limits_{j \ne i} {{u_j}} $$

In words, ${v_i}$ , the morally motivated payoff of an agent with absolute empathy equals the average of the material payoffs, ${u_{j,j \ne i}}$ , of all the other agents in the game. It may be worth highlighting the absoluteness of empathy here; agents choosing absolute empathy completely disregard their own material payoffs since their morally motivated payoff utility is based solely on the material payoffs of others.Footnote ¹¹

On the other hand, when selfishness is absolute, the agent cares only about herself without any regard to others. It may be helpful to note that we assume that players with moral agency have the ability to choose a ‘moral’ principle in which they are entirely materialistic or self-centred. In terms of the metaphor in our title, we assume that the shoulder-devil recommends the principle of selfishness. If a moral agent chooses this principle, their ‘morally motivated’ payoffs depend exclusively on their own material payoff. While calling this a moral principle might be contentious, we do so because our analysis in this paper is positive rather than normative with respect to alternative moralities. Moreover, selfishness is often recommended as a guiding principle by parents, teachers, as well as society at large through the advice that one must look out for oneself. Selfishness can be pithily expressed in terms of the famous observation, “each man for himself”, as popularized by Chaucer in the Knight’s Tale (Chaucer and MacKaye Reference Chaucer and MacKaye1914).

The morally motivated payoffs for a moral agent choosing absolute materialistic selfishness can be expressed as follows:Footnote ¹²

(2) $${v_i} \equiv {u_i}$$

In words, ${v_i}$ , the morally motivated payoff for an agent adopting absolute materialistic selfishness equals their own material payoffs, ${u_i}$ . Agents choosing absolute materialistic selfishness completely disregard the payoffs of all other players since their morally motivated payoff is based solely on their own material payoffs.

To create the first example model based on our framework, we modify the prisoner’s dilemma game between these two players to allow for moral agency such that each moral agent can choose between materialistic selfishness and empathy. We call this transformed game Empathetic Prisoner’s Dilemma (EPD). To transform the game, first a morality space needs to be defined for each moral agent which describes the set of alternative moral principles that the moral agent can choose amongst. Let $M = \left\{ {{m_1},{m_2}} \right\}$ denote the morality space or set of alternative moral principles available to each moral agent where ${m_1}$ is empathy $\left( E \right)$ and ${m_2}$ is materialistic selfishness $\left( {MS} \right)$ . With this transformation, moral agents choose not just their action strategy of cooperate or defect from the action strategy space, $S = \left\{ {c,d} \right\}$ , but also their moral principle of materialistic selfishness or empathy from the morality space $M = \left\{ {E,MS} \right\}$ . The transformed strategy space available to each moral agent will be the Cartesian product of the morality space and the action strategy space. The transformed strategy space is $T = \left\{ {\left( {E,c} \right),\left( {E,d} \right),\left( {MS,c} \right),\left( {MS,d} \right)} \right\}$ . We assume that the payoffs in the original prisoner’s dilemma (as depicted in Table 1) are material payoffs. Correspondingly, in the transformed game, the morally motivated payoffs for a moral agent choosing $MS$ will be the same as those in the original prisoner’s dilemma. The morally motivated payoffs for a moral agent choosing $E$ , on the other hand, will equal the material payoffs of the other player. In this example, we have chosen to model empathy as an unconditional moral principle, i.e. we assume that moral agents choosing empathy are empathetic with the other player regardless of whether the other player is empathetic or not. Table 2 depicts the payoffs for EPD.

Table 2.

Empathetic Prisoner’s Dilemma (EPD)

Table 2 Long description

The table presents the payoffs for the Empathetic Prisoners Dilemma (EPD) between two moral agents. It consists of four rows and four columns. The rows are labeled with the strategies of Moral Agent 1: (E, c), (E, d), (MS, c), and (MS, d). The columns are labeled with the strategies of Moral Agent 2: (E, c), (E, d), (MS, c), and (MS, d). Each cell in the table shows the payoff outcomes for the combinations of strategies chosen by both agents. For example, when both agents choose (E, c), the payoff is (R, R). When Moral Agent 1 chooses (E, d) and Moral Agent 2 chooses (MS, c), the payoff is (S, T). The table illustrates the different outcomes based on the choices of cooperation or defection and the moral principles of empathy or materialistic selfishness.

To analyse this game, it may be helpful to first eliminate strictly dominated strategies. Note that the strategy of $\left( {E,d} \right)$ is strictly dominated by the strategy of $\left( {E,c} \right)$ . This is intuitive because if a moral agent chooses to be empathetic, cooperation is strictly superior to defection. Similarly, the strategy of $\left( {MS,c} \right)$ is strictly dominated by the strategy of $\left( {MS,d} \right)$ . This too is intuitive because if a moral agent chooses to be materialistically selfish then, as in the standard result from PD, defection strictly dominates cooperation. After eliminating strictly dominated strategies, EPD reduces to the following game depicted in Table 3.

Table 3.

Empathetic Prisoner’s Dilemma (EPD), Reduced

Table 3 Long description

A table with two rows and two columns compares outcomes of moral agents in the Empathetic Prisoners Dilemma. The rows are labeled Moral Agent 1 and the columns are labeled Moral Agent 2. The table cells contain pairs of outcomes: (E,c) and (MS,d) for Moral Agent 1, and (E,c) and (MS,d) for Moral Agent 2. The outcomes are R,R; T,T; T,T; and P,P. The table illustrates the reduced strategies after eliminating strictly dominated strategies.

This game has two types of equilibria: an asymmetric pure-strategy equilibrium in which one moral agent plays $\left( {E,c} \right)$ and the other moral agent plays $\left( {MS,d} \right)$ , and a symmetric mixed-strategy equilibrium in which each moral agent plays a mixture of $\left( {E,c} \right)$ and $\left( {MS,d} \right)$ .

The asymmetric pure strategy equilibrium depicts a common scenario in which a moral agent who is capable of empathy acts altruistically even toward someone who is defecting in a materialistically selfish way. Similarly, a moral agent capable of materialistic selfishness exploits empathetic altruists. Note that this interaction can only be considered an exploitation in terms of material payoffs for the moral agents since the defector receives $T$ while the cooperator receives $S$ . In material terms, this result broadly matches some of those in Bernheim and Stark (Reference Bernheim and Stark1988) who show how altruism can often entail exploitability in the context of resource allocation within families. In terms of morally motivated payoffs, on the other hand, the asymmetric equilibrium results in both moral agents receiving a payoff of $T$ , the highest possible payoff in this game.

From our perspective, the symmetric mixed-strategy equilibrium is more interesting. The symmetric equilibrium is the unique equilibrium where each moral agent believes that the other moral agent will play the same strategy he does. Within the symmetric equilibrium the probability that a moral agent playing optimally plays the strategy $\left( {E,c} \right)$ is given by the following equation:

(3) $${\rm{Pr}}\left( {E,c} \right) = {{\left( {T - P} \right)} \over {\left( {T - P} \right) + \left( {T - R} \right)}}$$

As can be seen in Equation 3, ${1 \over 2} \lt Pr\left( {E,c} \right) \lt 1$ because $T \gt R \gt P$ . When $P$ approaches $R$ , ${\rm{Pr}}\left( {E,c} \right)$ approaches ${1 \over 2}$ , and when $R$ approaches $T$ , ${\rm{Pr}}\left( {E,c} \right)$ approaches $1$ . Hence, moral agents acting optimally in the symmetric equilibrium of EPD are more likely than not to cooperate. As can be seen from this equation, such moral agents are not immune to temptation; because $T \gt R$ , they defect with some probability. Yet, no matter how large the temptation to defect is, moral agents who play the mixed strategy equilibrium of EPD optimally are more likely to cooperate than to defect. And as the temptation to defect approaches $0$ , i.e. when $T$ approaches $R$ , the probability that a moral agent defects approaches $0$ .

Granting players moral agency therefore allows them to choose to be empathetic in a probabilistic manner where the probability depends on the material payoffs of the game. Moreover, even though the two moral choices were absolute (absolute empathy and absolute selfishness), through the use of mixed strategies, agents are able to arrive at an optimal intermediate level of morality that is contingent on the specific payoffs of the game.

It may be helpful to contrast this result with the unique equilibrium for prisoner’s dilemma games where moral agency is no longer permitted. The typical version of prisoner’s dilemmas as depicted in Table 1 can be considered identical to a transformed game where the morality space for both players is restricted to materialistic selfishness. This game (FMSPD) is depicted in Panel (A) of Table 4. In this game, the unique equilibrium involves both players defecting regardless of the specific values of the payoffs (as long as $T \gt R \gt P \gt S$ , i.e., the underlying game is indeed prisoner’s dilemma). Similarly consider a situation in which empathetic behaviour is forced on players (FEPD). This would be the situation if the morality space for both players is restricted to empathy. Panel (B) of Table 4 presents this game. In this game, the unique equilibrium involves both players cooperating regardless of the specific values of the payoffs (as long as $T \gt R \gt P \gt S$ , i.e. the underlying game is indeed prisoner’s dilemma).

Table 4.

Forced Prisoner’s Dilemmas

Table 4 Long description

The table presents two versions of the Prisoner's Dilemma game. Panel A shows the Forced Materialistically Selfish Prisoner's Dilemma (FMSPD) with Player 1 and Player 2 each having two strategies: cooperate (c) and defect (d). The payoff matrix includes combinations of (MS, c) and (MS, d) for Player 2, and (MS, c) and (MS, d) for Player 1. The payoffs are R, R; S, T; T, S; and P, P. Panel B shows the Forced Empathetic Prisoner's Dilemma (FEPD) with Player 1 and Player 2 each having two strategies: empathize (E) and defect (d). The payoff matrix includes combinations of (E, c) and (E, d) for Player 2, and (E, c) and (E, d) for Player 1. The payoffs are R, R; T, S; S, T; and P, P.

Allowing players to have moral agency therefore enables players to choose how moral they wish to be, in a probabilistic sense where the probabilities are endogenously determined in terms of the material payoffs of the game being played. Once agents are allowed moral agency, their cooperation is probabilistic too; the same player may choose to cooperate on one occasion while choosing to defect on another (in keeping with the mixed-strategy equilibrium), reflecting how moral agents sometimes listen to the shoulder-angel and sometimes listen to the shoulder-devil.

We should acknowledge certain limitations of our modelling choices in EPD, particularly the binary nature of the choice between absolute empathy and absolute materialistic selfishness, the idealized nature of these moral principles, and our focus on the one-shot game rather than the repeated game.Footnote ¹³ These limitations could be relaxed in future research, for example, by modelling choices among multiple moral principles, introducing more nuanced forms of empathy, or considering repeated games. Since our choices were made for simplicity and to illustrate key features of our framework, we do not make strong empirical claims about EPD’s definitive accuracy or superiority over parametric approaches, given the differences between our model’s assumptions and typical experimental setups.

Nevertheless, we find the results from EPD are aligned with those from several studies in experimental game theory that find the rates of cooperation in prisoner’s dilemma to be dependent on payoff values. In a well-known study widely considered a classic in the field, Rapoport and Chammah (Reference Rapoport and Chammah1965: 48) argue that: “If the payoffs are varied singly, common sense suggests that the frequency of cooperative responses ought to increase with the rewards parameter ${\rm{R}}$ , ought to decrease with the temptation parameter ${\rm{T}}$ , and ought to increase as the magnitude of the (negative) punishment parameter ${\rm{P}}$ increases.” Based on experimental results conducted using seven different prisoner’s dilemma games, Rapoport and Chammah (Reference Rapoport and Chammah1965) corroborate these common-sense conjectures. Each of these common-sense conjectures is also borne out in the theoretical predictions of EPD. As can be seen from Equation 3, ${{\partial {\rm{Pr}}\left( {E,c} \right)} \over {\partial R}} = {{T + P} \over {{{[\left( {T + P} \right) - \left( {T - R} \right)]}^2}}} \gt 0$ , entailing that cooperation increases when reward increases. Similarly, ${{\partial {\rm{Pr}}\left( {E,c} \right)} \over {\partial T}} = {{P - R} \over {{{[\left( {T + P} \right) - \left( {T - R} \right)]}^2}}}{\rm{ \lt }}0$ , entailing that cooperation decreases when temptation increases. Finally, ${{\partial {\rm{Pr}}\left( {E,c} \right)} \over {\partial P}} = {{R - T} \over {{{[\left( {T + P} \right) - \left( {T - R} \right)]}^2}}}{\rm{ \lt }}0$ , entailing that cooperation decreases when punishment increases. The core findings from Rapoport and Chammah (Reference Rapoport and Chammah1965) have been supported by more recent studies as well. Engel and Zhurakhovska (Reference Engel and Zhurakhovska2016) find that cooperation rates fall monotonically with increases in $P$ while Charness et al. (Reference Charness, Rigotti and Rustichini2016) and Schneider and Shields (Reference Schneider and Shields2022) find that cooperation rates increase monotonically with increases in $R$ .

While EPD’s theoretical predictions regarding the relationship between cooperation rates and changes in payoffs align very well with observed behaviour in experimental studies of prisoner’s dilemma, a direct comparison is infeasible because of differences in experimental setups and the assumptions of EPD. For instance, EPD assumes that both players know each other and are capable of feeling empathy towards one another. The experiments, on the other hand, are typically conducted among anonymous participants who may therefore be unable to experience empathy with the other players. Moreover, while $T$ , $R$ , $P$ and $S$ in experiments typically refer to monetary payoffs, in EPD they refer to the non-moral component of psychic payoffs that may scale non-linearly due to factors such as diminishing marginal utility. Nevertheless, if we were to assess the predictive power of EPD using simple correlations between its theoretical predictions and experimental observations of how cooperation rates change with payoffs, we find compelling results. Charness et al. (Reference Charness, Rigotti and Rustichini2016) examine four one-shot games with varying payoffs. The correlation between observed cooperation rates across those four games and EPD’s theoretical predictions is 0.98. Rapoport and Chammah (Reference Rapoport and Chammah1965) test seven games using several conditions among which the ‘block matrix’ condition most closely resembles the setup assumed in EPD. Across these seven games in the ‘block matrix’ condition, the correlation with EPD’s theoretical prediction is 0.84. In Schneider and Shields (Reference Schneider and Shields2022), only four out of the six games tested are symmetric and therefore directly comparable to EPD. Schneider and Shields (Reference Schneider and Shields2022) examine these six games using several different experimental settings involving sequential and simultaneous play. Among all the varied experimental settings, the one most closely matching EPD yields cooperation rates that have a correlation of 0.97 with the theoretical predictions of EPD. Appendix B provides more details on how we computed these correlations.

Several key features of the framework proposed in this paper can be identified by comparing EPD with the models explored in Schneider and Shields (Reference Schneider and Shields2022). To explain their observed data, Schneider and Shields (Reference Schneider and Shields2022) consider four alternative models of preferences. The first alternative, pure selfishness, is non-parametric and is inconsistent with observed rates of cooperation. The remaining three alternatives parametrically combine pure selfishness with inequity aversion, utilitarianism and Rawlsian Maximin, respectively. For each of these three parametric approaches, Schneider and Shields (Reference Schneider and Shields2022) assume that the parameter can vary over a normal distribution (or a beta distribution) and estimate the mean and variance of the distribution using experimentally observed cooperation rates across the different games. EPD, built using the framework described in this paper, uses an entirely different approach. Table 5 highlights the differences between Schneider and Shields (Reference Schneider and Shields2022) and EPD.

Table 5.

Comparing Schneider and Shields (Reference Schneider and Shields2022) with EPD

Table 5 Long description

A table comparing Schneider and Shields 2022 with EPD across four rows and two columns. The table highlights key differences in how selfishness and prosocial preferences are combined, the balance of selfishness and prosociality, degrees of freedom in model parameters, and the explanation of cooperation variability. Row 1: Schneider and Shields 2022 combines selfishness and prosocial preferences parametrically, while EPD combines them through mixed strategies. Row 2: In Schneider and Shields 2022, the relative balance of selfishness and prosociality is exogenous, whereas in EPD, it is endogenous and depends on the payoffs of the game being played. Row 3: Schneider and Shields 2022 permits two degrees of freedom beyond modeller's choice of core preferences, while EPD has extreme parsimony with zero degrees of freedom beyond modeller's choice of core preferences. Row 4: Schneider and Shields 2022 requires parameters to be drawn from a probability distribution to explain cooperation variability, whereas EPD uses mixed strategy equilibria to explain and predict cooperation variability even if preferences are homogeneous.

Although Table 5 focuses on Schneider and Shields (Reference Schneider and Shields2022), it is important to note that this approach of parametrically combining prosocial and selfish preferences is the dominant approach in the literature and common to many papers, including, for instance, Levine (Reference Levine1998), Fehr and Schmidt (Reference Fehr and Schmidt1999), Bolton and Ockenfels (Reference Bolton and Ockenfels2000), Andreoni and Miller (Reference Andreoni and Miller2002), Alger and Weibull (Reference Alger and Weibull2013), etc.Footnote ¹⁴ While we acknowledge that there could be advantages to modelling morality with parametric combinations of selfish and prosocial preferences, we present an alternative approach in our framework that allows for preferences to be combined endogenously through mixed strategies such that moral agency can get modelled explicitly.

6. Kantian Universalizing in the Public Goods Game

For a second example of a model based on our framework, we analyse a public goods game (PGG) with $n$ moral agents, each of whom can choose to either contribute $\left( c \right)$ a single unit to the public good or withhold $\left( w \right)$ their contribution and keep their unit with themselves. The action strategy space of each moral agent is therefore given by $S = \left\{ {c,w} \right\}$ . All contributed units are added together and then multiplied by a multiplication factor $r$ to obtain the total quantity of the public good. This total quantity of the public good is then distributed equally across all $n$ players regardless of whether they contributed or not. For PGG, $1 \lt r \lt n$ .

For this game, we believe that the relevant moral principle is Kantian universalizing, $\left( {KU} \right)$ . In prisoner’s dilemma, the example developed in section 5 of the paper, we assumed that the two players have a close relationship between them that could evoke empathy. In PGG, on the other hand, the setup involves a large collection of $n$ players who are unlikely to know each other. Contributions to public goods might therefore arise from a sense of duty to the collective, i.e. in response to a shoulder-angel whispering, “what would happen if no one paid into the public good?”.

In general, if a moral agent were to choose Kantian universalizing as their moral principle, the morally motivated payoff she receives from an action matches the material payoff she would have received if all other players were to choose the same action. Culturally, this moral approach is often propagated by parents and teachers through the common admonition, “what if everyone did that?”.

Kantian ethics have been modelled in several economics papers, including, for instance, in Alger and Weibull (Reference Alger and Weibull2013), Roemer (Reference Roemer2010) and Studtmann and Gouri Suresh (Reference Studtmann and Suresh2021). The version of morality we use in this paper is identical to Studtmann and Gouri Suresh (Reference Studtmann and Suresh2021) and can be considered an absolute form of the version described by Alger and Weibull (Reference Alger and Weibull2013) among others. Alger and Weibull (Reference Alger and Weibull2013) define the utility function of homo moralis (their term for agents with parametrically varying degrees of Kantian morality) as follows:

(4) $${v_\kappa }\left( {x,y} \right) \equiv \left( {1 - \kappa } \right)u\left( {x,y} \right) + \kappa u\left( {x,x} \right)$$

In words, the morally motivated payoff of an agent, ${v_\kappa }\left( {x,y} \right)$ , with a morality level $\kappa \in \left[ {0,1} \right]$ who chooses an action $x$ while her symmetric counterparty chooses an action $y$ equals the weighted average of her actual material payoff, $u\left( {x,y} \right)$ , and the material payoff, $u\left( {x,x} \right)$ , she would have received if her counterparty had also chosen the same action $x$ as her. Alger and Weibull (Reference Alger and Weibull2013) refer to agents with $\kappa = 1$ as homo kantiensis.

In our example we assume that the shoulder-angel recommends absolute Kantian universalizing; in other words, an agent choosing the moral principle of Kantian universalizing receives the same morally motivated payoff as homo kantiensis in Alger and Weibull (Reference Alger and Weibull2013):

(5) $$v\left( {x,y} \right) \equiv u\left( {x,x} \right)$$

In words, the morally motivated payoff, $v\left( {x,y} \right)$ , for an agent who is an absolute Kantian universalizer equals the material payoff she would have received if her counterparty had also chosen the same action $x$ as her, regardless of the actual action $y$ chosen by her counterparty. It might be worthwhile to note that Kantian universalization is strictly deontological. That is, if a moral agent chooses Kantian universalizing, her morally motivated payoffs will not just be independent of the other player’s payoffs, they will also be independent of the actions and moral motivations of other players.

Thus, we include $\left( {KU} \right)$ as a member of the morality space for agents in PGG. We model players as also having the moral agency to choose materialistic selfishness, $\left( {MS} \right)$ , as their other alternative. The morality space for each agent is therefore assumed to be $M = \left\{ {MS,KU} \right\}$ . The transformed strategy space for each moral agent will be the Cartesian product of the morality space and the action strategy space. The transformed strategy space is given by $T = \left\{ {\left( {MS,c} \right),\left( {MS,w} \right),\left( {KU,c} \right),\left( {KU,w} \right)} \right\}$ .

The morally motivated payoffs for a moral agent choosing $\left( {MS} \right)$ will depend on how many total players contributed and whether the agent herself contributed. Since $\left( {KU} \right)$ is strictly deontological, the morally motivated payoffs for a moral agent choosing $\left( {KU} \right)$ depend solely on whether she contributes. If the moral agent who chooses $\left( {KU} \right)$ contributes, her morally motivated payoff equals what her material payoff would be if all players were to contribute = ${{\left( {r \times n} \right)} \over r} = r$ . If the moral agent who chooses $\left( {KU} \right)$ withholds, her morally motivated payoff equals what her material payoff would be if no players were to contribute, i.e. $1$ , the withheld unit she gets to keep for herself. Table 6 depicts the payoffs for this game for an arbitrary moral agent, $i$ .

Table 6.

Public Goods Game (PGG)

Table 6 Long description

The table presents payoffs for a moral agent in a Public Goods Game, focusing on different contributions and motivations. It has four rows and five columns. The rows are labeled with Moral Agent i and different moral choices: (MS,c), (MS,w), (KU,c), and (KU,w). The columns represent the number of other players contributing, ranging from 0 to n-1. Each cell contains a payoff value based on the agent's choice and the number of contributors. For (MS,c), payoffs vary with the number of contributors, incorporating a fraction of the total contributions. For (MS,w), payoffs are calculated based on the number of contributors and a fixed value. For (KU,c), the payoff is consistently r if the agent contributes. For (KU,w), the payoff is consistently 1 if the agent withholds. The table illustrates how different moral choices and contributions affect the payoffs in the game.

Since $1 \lt r \lt n$ , notice that $\left( {MS,c} \right)$ is strictly dominated by $\left( {MS,w} \right)$ and $\left( {KU,w} \right)$ is strictly dominated by $\left( {KU,c} \right)$ . Upon eliminating strictly dominated strategies, the game reduces as shown in Table 7.

Table 7.

Public Goods Game (PGG), Reduced

Table 7 Long description

The table presents the number of other players contributing in a Public Goods Game, focusing on moral agents and their strategies. It has two rows labeled Moral Agent i with strategies (MS, w) and (KU, c), and columns indicating the number of other players contributing, ranging from 0 to n-1. The table shows the payoff values for each strategy under different conditions. For strategy (MS, w), the payoff is 1 when no other players contribute and increases linearly with the number of contributors. For strategy (KU, c), the payoff is r when no other players contribute and remains constant across all columns.

While this game has many asymmetric Nash equilibria,Footnote ¹⁵ it has a unique symmetric Nash equilibrium that involves a mixed strategy between $\left( {MS,w} \right)$ and $\left( {KU,c} \right)$ . Solving for the unique symmetric Nash equilibrium, moral agents play the strategy $\left( {KU,c} \right)$ with the following probability:

(6) $${\rm{Pr}}\left( {KU,c} \right) = {{n \times \left( {r - 1} \right)} \over {\left( {n - 1} \right) \times r}}$$

Note that $0 \lt Pr\left( {KU,c} \right) \lt 1$ because $1 \lt r \lt n$ . When $r$ approaches $1$ , ${\rm{Pr}}\left( {KU,c} \right)$ approaches $0$ . When $r \le 1$ , the game is not PGG; contributing to the public good does not create any benefit because the multiplicative factor is less than or equal to one. When $r$ approaches $n$ , ${\rm{Pr}}\left( {KU,c} \right)$ approaches $1$ . When $r \ge n$ the game is not PGG; contributing to the public good creates so much benefit that it would be optimal for each agent to do so even if no one else contributes.

The behaviour of moral agents playing the symmetric equilibrium is dependent on the multiplicative factor and the number of players. Since ${{\partial {\rm{Pr}}\left( {KU,c} \right)} \over {\partial r}} = {n \over {\left( {n - 1} \right) \times {r^2}}} \gt 0$ and ${{\partial^2 {\rm{Pr}}\left( {KU,c} \right)} \over {\partial r}^2} = - {{2 \times n} \over {\left( {n - 1} \right) \times {r^3}}} \lt 0$ , when the multiplicative factor increases (for a given number of players), a moral agent is more likely to contribute, but at a diminishing rate for further increases in the multiplicative factor. It may be helpful to note that this is qualitatively consistent with empirical evidence suggesting that cooperation increases at a diminishing rate for increases in the multiplicative factor (van den Berg et al. Reference van den Berg, Dewitte, Aertgeerts and Wenseleers2020). On the other hand, since ${{\partial {\rm{Pr}}\left( {KU,c} \right)} \over {\partial n}} = - {{\left( {r - 1} \right)} \over {r \times {{(n - 1)}^2}}} \lt 0$ , when the public good is distributed across a greater number of agents (for a given multiplicative factor), a moral agent is more likely to withhold their contribution. Empirical evidence on group size is mixed. Some studies such as Isaac et al. (Reference Isaac, Walker and Williams1994) find that contributions increase with group size while other studies such as Nosenzo et al. (Reference Nosenzo, Quercia and Sefton2013)Footnote ¹⁶ find that under some circumstances, contributions decrease with group size. While our result predicts that moral agents will decrease their contributions as the number of players increases, it is important to note that the negative effect of group size on the likelihood of contribution diminishes with further increases in group size since ${{\partial^2 {\rm{Pr}}\left( {KU,c} \right)} \over {\partial n}^2} = {{2 \times \left( {r - 1} \right)} \over {r \times {{(n - 1)}^3}}} \gt 0$ . Moreover, moral agents continue to contribute with positive probability even when $n$ approaches infinity; ${\rm{li}}{{\rm{m}}_{n \to \infty }}{\rm{Pr}}\left( {KU,c} \right) = {{r - 1} \over r} \gt 0$ . Granting players moral agency therefore allows players to choose to be Kantian universalizers in a probabilistic manner where the probability depends on the multiplicative factor and the number of players in PGG.

As with EPD in the previous section, we acknowledge that our modelling choices in this public goods game have limitations. The binary choice between Kantian universalizing and materialistic selfishness, and the idealized nature of these principles, are simplifications made for illustrative purposes. Similar to our discussion of EPD, these limitations could be addressed in future research by exploring more nuanced moral principles or a wider range of choices. Once again, while we caution against overstating the model’s empirical accuracy or claiming superiority over parametric approaches, it’s worth noting that our predictions align well with empirical observations.

7. The Case for Incorporating Moral Principles as Decision Variables

There are several compelling reasons for modellers to consider adopting the framework we have proposed in this paper.

First and foremost, our framework allows modellers to ascribe moral agency to players. From a Kantian philosophical perspective, moral autonomy is a necessary condition for an agent’s behaviour to be considered moral; i.e. an agent can only be considered to be acting morally if they are actively choosing to be moral. In standard game-theoretic models, agents are typically ascribed a utility function that is a parametric combination of some prosocial preference (chosen by the modeller) and selfishness. The value of the parameter determines the extent of an agent’s morality. The autonomy of agents within a game in standard models is therefore limited to their ability to choose among action strategies. Our approach, on the other hand, extends the autonomy of agents to the domain of moral principles too. Agents can choose not just their action strategy, but also from among a set of moral preferences (the set, however, is chosen by the modeller). Moreover, while exogenous parameter values determine the extent of an agent’s morality in standard models, our approach allows agents to choose the extent to which they wish to be moral through the use of mixed strategies.

It is important to note that several papers do ‘endogenize’ the extent of prosociality or morality by describing how it can come about through evolution, culture, identity relationships, etc. While these approaches are valuable, they do not allow agents to have any moral autonomy within the game; the morality of agents in these approaches is determined pre-game. We believe that since our framework allows morality to be determined within the game rather than pre-game, it can prove to be a valuable addition to the toolset of game theorists. In this regard, our approach builds on Rotemberg (Reference Rotemberg1994) and Sally (Reference Sally2001) since they also model agents who can choose their level of morality within the game and do so based on the material payoffs of the game. However, we believe that our framework offers an approach that is distinct from prior approaches because it allows more explicit and more general models of moral autonomy. Rather than choosing the level of a morality parameter, agents in our framework can choose among moral rules. Moreover, rather than a binary version of morality, our approach can simultaneously accommodate multiple alternative moral principles.Footnote ¹⁷ However, it is important to reiterate that the set of moral principles will still need to be exogenous for the models in our framework and therefore cannot be chosen by the agents within a game. As such, the set of moral principles and their associated morally motivated payoffs will need to be determined by the game theoretician modelling the game based on the specific context of the game and the players, just as economic theoreticians, more generally, need to be judicious in their choice of utility functions based on the context of their model.

On a closely related note, when moral principles are not offered as a choice to agents, rationality (in terms of optimizing game-theoretic behaviour) extends only to how agents choose their strategies. In our framework, agents use rationality (again, in terms of optimizing game theoretic behaviour) even while choosing among the set of moral principles available to them. Similarly, equilibrium outcomes in standard models involve mutual consistency of beliefs only with regard to strategies whereas equilibrium outcomes in our framework involve mutual consistency of beliefs regarding both, strategies and moral principles.

In terms of the shoulder-angels and shoulder-devils metaphor, through mixed strategies, our framework can describe how a moral agent might sometimes choose to listen to the devil and at other times choose to listen to the angel. While such mixed strategy equilibria may suggest some kind of internal inconsistency, we prefer to describe them as reflecting inner conflict. In our everyday experience, it is common to feel pulled in different directions. We believe that this feeling of inner conflict also reflects several important ideas in philosophy.

• • Philosophical traditions, such as Plato’s concept of the appetitive soul (Plato 2003) and Freud’s notion of the Id (Freud Reference Freud2018), depict humans as inherently conflicted, with a selfish aspect and a drive towards what is perceived as “the good” or the requirements of civilization. Similarly, Kant’s philosophy also resonates with our depiction of agents in conflict. According to Kant, autonomous agents must choose between acting according to selfish inclinations or moral laws (Kant Reference Kant2015). Our model aims to capture this inner conflict, where agents must decide between competing visions of how to navigate the world while navigating the world. This contrasts with parametric views where conflicting motives are pre-mixed, thus eliminating the need for agents to resolve this conflict through reasoning during gameplay.

• • Mixed strategies in our framework arguably also embody Leibniz’s idea that reasons incline but do not necessitate action (Clarke and Leibniz Reference Clarke and Freiherr von Leibniz1956). In EPD, factors like stakes and temptation influence an agent’s inclination towards cooperation or defection, though because the equilibrium is probabilistic, these factors do not entail a decision one way or another. For instance, if empathizing and cooperating results in a larger benefit to the other player (for a given cost to self), a moral agent is more likely to empathize and cooperate. However, if empathizing and cooperating imposes a larger cost to self (for a given benefit to the other player), a moral agent is more likely to be materialistically selfish and defect. Similarly, in PGG, the rate of return influences an agent’s inclination to contribute to the public good. Thus, the extent to which an agent adopts a particular moral stance is a function not only of their predetermined set of underlying alternative moral principles (possibly acquired through evolution, culture, identity relationships, etc.), but also of the payoffs specific to the game. While one may reject Leibniz’s view, we believe that the nature of the mixed strategy equilibria in our models represents this view and thus offers a valuable contribution to game theoretic accounts of moral agency.

• • Finally, there is one other source of interest in the mixed strategy, namely that it accommodates the intriguing possibility that agents can exhibit stable characters while acting differently in similar situations. For example, an individual might choose to give money to a panhandler on one occasion but not on another, despite no apparent change in circumstances, while still maintaining a consistent character, which can be understood as the result of the fact that a single probability distribution governs the equilibrium and hence the agent’s actions within that equilibrium.

Our framework also typically results in a greater number of possible equilibria because the transformed strategy space is larger than the original strategy space. The presence of multiple equilibria, especially mixed-strategy equilibria as noted above, could be used to explain heterogeneous behaviour within a player without resorting to time-varying, random, arbitrary or inconsistent individual-specific moralities. This framework can therefore explain why studies such as Blanco et al. (Reference Blanco, Engelmann and Normann2011) find that models of other-regarding preferences have predictive power at the aggregate level but not at the individual level. Moreover, the presence of multiple equilibria, especially mixed-strategy equilibria, can also explain heterogeneous outcomes within a population without assuming heterogeneous agents.

In addition to the theoretical advantages mentioned above, we believe that human experience also suggests the validity of our framework. In our own lived experience, we have often grappled consciously with moral quandaries and considered multiple moral perspectives in order to determine our eventual course of action. These considerations have not been merely at the level of action strategies (e.g. should I defect, or should I cooperate?), but also at the level of deeper principles (e.g. should I act on the dictum of ‘each man for himself’, or should I ‘put myself in the shoes of the other person’?). The universality of our own lived experiences is evident from the ubiquity of the shoulder-angels and shoulder-devils plot device in literature, theatre, television and film. Moral agents don’t just choose action strategies, they also simultaneously choose moral principles. We therefore believe that in order to reflect this experience, game theory models should also explicitly allow players to simultaneously choose action strategies and moral principles.

In the context of empathy specifically, several empirical studies suggest that humans do have agency over whether to be empathetic. In the experimental economics literature, Andreoni et al. (Reference Andreoni, Miller, Rao and Trachtman2017) provide evidence that people actively choose to avoid empathetic situations in certain circumstances, but once empathy is triggered the same people are capable of considerable generosity. In the psychology literature, Zaki (Reference Zaki2014: 1634) writes, ‘Scientists have long recognized empathy’s simultaneous automaticity and context dependency, but they have struggled to reconcile these countervailing findings. Highlighting empathy’s motivated nature resolves this tension. As with other affective states, motives guide our willingness to empathize and shape the structure of empathic responding.’ We believe that a comprehensive modelling of moral agency requires a framework such as the one we propose – our framework allows for both an exogenous component to moral preferences (i.e. the elements of the set, ${M_i}$ ) and an endogenous, situation-specific component (i.e. the moral choices made by moral agents during gameplay).

Our final justification for the framework is related to the role of assumptions in economic theory. Friedman (Reference Friedman1953) recommends readily adopting assumptions through an ‘as-if’ approach as long as the resulting theory is simple and fruitful. Friedman (Reference Friedman1953) defines simplicity as being able to predict at least as much as an alternate theory, although requiring less information. We believe our proposed framework is simple, intuitive and very general. Unlike standard approaches which typically represent moral ideas through parameterized combinations of prosocial and selfish preferences, our framework endogenously combines such preferences with a parameter space of zero. This is a substantial improvement in terms of parsimony. Friedman (Reference Friedman1953) defines the fruitfulness of a theory in terms of the precision and scope of its predictions as well as its ability to generate additional research lines. While models using our framework have not yet been evaluated in terms of their predictions through targeted empirical research, we find that theoretical predictions from our two sample models, EPD and PGG, are broadly consistent with experimental literature. Most importantly though, we are confident that our framework will be fruitful in terms of its ability to successfully generate important additional lines of research.