2.1 What Is Welfare?

Commonsense morality holds that people can be better or worse off depending on what happens to them. A person is worse off for getting a paper cut and better off for having a hearty meal or spending quality time with a friend. Whether our actions make people’s lives go better or worse is one factor (perhaps among several factors) that determines whether our actions are right or wrong.Footnote ³

When we use the term ‘welfare’ (or ‘well-being’), we are interested in whatever it is about a person’s life that constitutes their being better off or worse off (Kagan, Reference Kagan1998, 30; see also Parfit, Reference Parfit1987, 4; Hooker, Reference Hooker2015, 15–16). What we are concerned with is that which is non-instrumentally good for people: the factors that, in themselves and not in virtue of something else, make a life go better or worse. We are not interested in factors which are causally relevant to a person’s being better or worse off such as winning a raffle or watching a terrible film. These factors are merely instrumentally good or bad for that person. Exactly what factors are non-instrumentally good for a person is disputed, but the three main theories of welfare are illustrative (Parfit, Reference Parfit1987, 493–502).

First, hedonism holds that the amount of happiness in a person’s life is what constitutes their welfare. People’s lives are good if and to the extent that they are happy. Second, the desire-fulfilment theory holds the satisfaction of desires is what constitutes a person’s welfare. People’s lives are good if and to the extent that their desires are satisfied. Third, the objective list theory holds that several distinct goods constitute a person’s welfare. People’s lives are good if and to the extent that their lives contain, inter alia, friendships, romantic relationships, meaningful projects, and the overcoming of adversity.

The presuppositions of the dispute between hedonists, desire theorists and objective list theorists are worth making explicit so that we can crystallise what is at issue when we talk about welfare. First, there is a metaphysical presupposition that there are facts of the matter about which of the different possible lives of a given person are better or worse. To accept that there are facts about welfare does not require strong commitments with respect to the structural characteristics of those facts. For example, it could be true that there are some facts about which lives are better or worse than others, but that not all pairs of lives are such that one is better than the other or both are equally good. It could also be that certain pairs of lives are comparable but only imprecisely so (Parfit, Reference Parfit1987, 431; Parfit, Reference Parfit2011, 130–141). Indeed, if facts about welfare depend on more basic comparative facts that admit failures of precise comparability (such as the comparative strength of desires), then it would be unsurprising if the betterness relation over the possible lives of a given person was imprecise.

Second, there is an epistemic presupposition that our folk conception of welfare is at least a semi-reliable guide to the nature of welfare. In particular, it is by assessing theories against commonsense judgements and revising those theories or judgements in light of discrepancies that we are supposed to converge on the correct theory of welfare. For example, it is meant to be a problem for hedonism that it fails to capture the commonsense judgement that wiring oneself up to a pleasure machine is not a life well lived (Nozick, Reference Nozick1974, 42–45; see also Crisp, Reference Crisp2006, Ch. 4). And it is meant to be a problem for the desire-fulfilment theory that satisfaction of certain desires that a person might have – such as the desire to count blades of grass – do not intuitively contribute to a person’s welfare (Rawls, Reference Rawls1971, 432; Anscombe, Reference Anscombe1957, 70–71). We take ourselves to be progressing towards the correct analysis of welfare when we revise our theories in light of these judgements (e.g., imposing certain restrictions on what kinds of desires count as relevant to welfare (Parfit, Reference Parfit1987, 494)), or else supplementing theories of welfare with error theories for certain commonsense judgements.Footnote ⁴

Third, there is a methodological presupposition that we can compress facts about welfare into law-like generalisations, such as the hedonist’s claim that a person’s life is good if and to the extent that it is happy (c.f., Hooker Reference Hooker2000, 19–23). In this way, welfare is supposed to be comparable to physical phenomena such as planetary motion that can be described in law-like terms, and unlike, for example, the daily movements of the stock market. How this presupposition manifests in discussions is principally in the fact that the complexity of any theory of welfare is supposed to count against that theory in an abductive comparison with other theories, where an abductive comparison might take into account theoretical virtues including, in addition to simplicity, explanatory power, coherence with other bodies of knowledge, internal consistency, and so on.

With these presuppositions on the table we can flag some ways in which people might take issue with how welfare is being understood. First, it might be that there are no facts about welfare because the commonsense concept of ‘welfare’ fails to pick out a real relational property that obtains between (some) possible lives of a given person. It might seem that a life is better, all else equal, minus a given headache or toe stubbing. But that seeming may be wrong; for example, if the moral error theory is true (Joyce, Reference Joyce2007; Mackie, Reference Mackie1977). Second, even if there are facts about welfare, our commonsense judgements about it could be misleading. This concern is especially salient with respect to the welfare of non-human entities. Third, even if our commonsense judgements are a reliable guide to which possible lives are better or worse, there may be no true law-like generalisations about what constitutes a life being better or worse. Perhaps welfare is exceptionally complicated. Our aim is not to be dogmatic with respect to these reservations. To us it seems plausible that there are facts about welfare for at least humans and some non-human animals: some lives are better and worse than others. But our commonsense judgements may be unreliable as guides to which lives are better and worse and for what reasons, and it may be that welfare resists law-like characterisation.

2.2 What Has Welfare?

Supposing that welfare is a thing for at least humans and some non-human animals, the next obvious question is what kinds of entities are welfare subjects. Presumably, garden gnomes, staplers, ironing boards and shoelaces are all examples of entities that are not welfare subjects, whereas humans, cats, dogs and horses are all examples of welfare subjects.Footnote ⁵ There is a discernible sense in which things can go better or worse for humans, cats, dogs and horses that does not seem to hold for garden gnomes, staplers, ironing boards and shoelaces. Yet there are many other cases in which it is unclear whether something is a welfare subject. For example, it is not obvious whether ants, bees, lobsters and squid are welfare subjects (c.f., Clatterbuck and Fischer, Reference Clatterbuck and Fischer2025; Birch et al., Reference Birch, Burn, Schnell, Browning and Crump2021).

The problem of figuring out whether a given entity is a welfare subject can be decomposed into two parts. Half of the problem is to work out what conditions are necessary and sufficient for being a welfare subject. For example, many people think that sentience – the capacity for valenced phenomenal states such as pleasure and pain – is sufficient for being a welfare subject; and some people defend the biconditional view that sentience is necessary and sufficient for being a welfare subject (Nussbaum, Reference Nussbaum2024; Singer, Reference Singer2016; see also Dung, Reference Dung2024). Others think that different properties matter. For example, Shelly Kagan (Reference Kagan2019, 30) believes that a minimal kind of agency is necessary and sufficient for welfare.

The other half of the problem is figuring out whether the entity in question satisfies the relevant criteria. And to be sure, some criteria are more tractable than others. It is easier to test whether an entity satisfies a minimal conception of agency than it is to test for consciousness. Hence the case for any particular entity being a welfare subject will involve both a moral component – a theory of what conditions need to be met in order to count as a welfare subject, and a descriptive component – evidence that the relevant conditions are satisfied. For some welfare-relevant features such as consciousness, sentience, agency, personhood and self-awareness, the challenge presented by the descriptive component is either empirical or some combination of empirical methods with a priori metaphysical theorising. For other welfare-relevant features, such as souls, the challenge may involve alternative ways of knowing such as through revelation or scripture, perhaps in combination with a priori metaphysical theorising.

2.3 Why AI Welfare?

In April 2023, several prominent scientists including Yoshua Bengio and Karl Friston signed an open letter stating that ‘it is no longer in the realm of science fiction to imagine AI systems having feelings and even human-level consciousness’ and that ‘consciousness would give AI a place in our moral landscape’ (Association for Mathematical Consciousness Science, 2023). Since then, AI welfare has become an increasingly central component of the public conversation on AI, with Anthropic, the developers of Claude, hiring an AI welfare officer in 2024 (Werner, Reference Werner2024), and including AI welfare evaluations as part of the 2025 Claude 3.7 Sonnet and Claude 4 model releases (Anthropic, 2025b,c).Footnote ⁶

These developments take place on the backdrop of recent and rapid capability advancements in AI. Three technological developments in particular have contributed to perceptions among the public and academics that AIs are plausible candidate welfare subjects: linguistic fluency, relationality, and agency.

First, on linguistic fluency, dialogue agents that sample from LLMs can respond flexibly to natural language requests on a broad range of topics (Shanahan, Reference Shanahan2024b), including making verbal reports that purport to be about their own welfare. This linguistic fluency is in part the result of pre-training models to predict the next word on large quantities of internet text data (Bommasani et al., Reference Bommasani, Hudson, Adeli, Altman, Arora, von Arx, Bernstein, Bohg, Bosselut and Brunskill2021). But perhaps equally important is the development of reinforcement learning from human feedback (RLHF) (Christiano et al., Reference Christiano, Leike, Brown, Martic, Legg and Amodei2017), which has been effectively leveraged by AI labs to dispose LLMs to respond in ways that cohere with the behavioural dispositions of a helpful, honest and harmless interlocutor (Askell et al., Reference Askell, Bai, Chen, Drain, Ganguli, Henighan, Jones, Joseph, Mann and DasSarma2021; Bai et al., Reference Bai, Kadavath, Kundu, Askell, Kernion, Jones, Chen, Goldie, Mirhoseini and McKinnon2022). Indeed, the linguistic competence of LLMs is such that LLMs are now able to pass what many consider to be empirically rigorous formulations of the Turing Test (Jones and Bergen, Reference Jones and Bergen2025), with some scholars suggesting that LLMs demonstrate early potential indicators of general intelligence (c.f., Morris et al., Reference Morris, Jascha Sohl-Dickstein, Warkentin, Dafoe, Faust, Farabet and Legg2023; Bubeck et al., Reference Bubeck, Chandrasekaran, Eldan, Gehrke, Horvitz, Kamar, Lee, Lee, Li, Lundberg, Nori, Palangi, Ribeiro and Zhang2023). While linguistic fluency and general intelligence do not entail that a given entity is a welfare subject, these properties might reasonably be understood as being broadly indicative of the potential for welfare subjecthood (Shevlin, Reference Shevlin2021a). In addition, LLM self-reports indicative of welfare-relevant features may be taken to constitute (at least) prima facie evidence of welfare-relevant features.

Second, the nature of human-AI interactions is changing. First, in virtue of the linguistic fluency of LLMs alongside other anthropomorphic qualities such as the use of first-person pronouns and explicitly relational language towards users (e.g. ‘you are my friend’), user experiences of LLMs are becoming more immersive and in certain respects similar to human interpersonal relationships. Moreover, the basic human-AI interaction paradigm is shifting from a paradigm of independent and isolated interactions to one of cumulative interactions over longitudinal time horizons, making room for distinctive kinds of relationships (Manzini et al., Reference Manzini, Keeling, Alberts, Vallor, Morris and Gabriel2024a; Gabriel et al., Reference Gabriel, Manzini, Keeling, Hendricks, Rieser, Iqbal, Tomašev, Ktena, Kenton and Rodriguez2024). What is salient here is that those relationships can evolve over time through cumulative human-like interactions that facilitate greater mutual understanding and trust, rendering the AI a novel kind of social actor that stands in a distinctly social relationship to the user (Alberts et al., Reference Alberts, Keeling and McCroskery2024; Manzini et al., Reference Manzini, Keeling, Marchal, McKee, Rieser and Gabriel2024b). The ability to stand in certain kinds of social relationships is on some views understood as a condition on being a welfare subject (c.f., Gunkel, Reference Gunkel2018; Coeckelbergh, Reference Coeckelbergh2018). And several advocates of non-relational views of the grounds for welfare foreground our relationships to, inter alia, other humans and non-human animals as importantly connected to welfare even if not strictly criterial of it (Nussbaum, Reference Nussbaum2024; Korsgaard, Reference Korsgaard2018). Hence it is again unsurprising that growing interest in AI welfare has come hand-in-hand with the emergence of novel human-AI relationships.

Third, AI systems are also increasingly being equipped with agency, which might roughly be understood in terms of AI systems being sufficiently autonomous to plan and execute complex tasks in response to high-level user instructions, while also being empowered to perform real-world actions with material consequences (Chan et al., Reference Chan, Ezell, Kaufmann, Wei, Hammond, Bradley, Bluemke, Rajkumar, Krueger and Kolt2024, Reference Chan, Wei, Huang, Rajkumar, Perrier, Lazar, Hadfield and Anderljung2025; Cohen et al., Reference Cohen, Kolt, Bengio, Hadfield and Russell2024; Kolt, Reference Kolt2025). While some people understand agency as a condition on being a welfare subject (Kagan, Reference Kagan1998, 30), it is important to register that even if agency fails as a criterion of welfare subjecthood, agentic AI systems have the potential to stand in richer kinds of relationships with human users (c.f., Mattingly and Cibralic, Reference Mattingly and Cibralic2025, Ch. 11). For example, agentic AIs can proactively take actions – such as ordering flowers on the anniversary of the death of a loved one – that greatly amplify the immersiveness of human-AI relationships. Hence it is unsurprising that increased interest in AI welfare has tracked increases in AI agency.

In light of these developments and the increasing centrality of AI welfare in public discourse, it is important to articulate and clarify the stakes of the dispute. The importance of the dispute over AI welfare can be understood in terms of the avoidance of two bad outcomes: under-attributing and over-attributing welfare to AIs (Schwitzgebel and Garza, Reference Schwitzgebel and Garza2015; Shevlin, Reference Shevlin2021a; Sebo and Long, 2023; Long et al., Reference Long, Sebo, Butlin, Finlinson, Fish, Harding, Pfau, Sims, Birch and Chalmers2024). On one hand, failing to register that AIs are welfare subjects when AIs are in fact welfare subjects is bad because it could lead to unintentional mistreatment of AIs or the neglect of the needs of AIs, potentially resulting in large-scale suffering. In addition to being bad for AIs, such suffering could present additional risks for humans insofar as AI suffering could provoke AIs to engage in harmful behaviours towards humans including ‘exclud[ing] us from their own moral community’, as Thomas Metzinger (Reference Metzinger2021) has argued (c.f., Long et al., Reference Long, Sebo and Sims2025). On the other hand, over-attributing welfare to AIs is problematic because resource allocation decisions for promoting the (potential) welfare of different kinds of entities – including humans, non-human animals and AIs – are often zero-sum. Hence investment of resources to advance the (potential) welfare interests of AIs may carry the opportunity cost of not investing those same resources to advance the welfare of humans and animals.

How persuasive these points are as grounds for taking seriously the question of AI welfare depends on our pre-existing commitments. For example, one class of dissenters might say that it is obvious that AIs are not welfare subjects because AIs fail to satisfy some necessary condition on being welfare subjects (e.g., embodiment), or more modestly that AIs are highly unlikely to be welfare subjects because they fail to satisfy a highly plausible necessary condition on being welfare subjects. These people might also say that even talking about AI welfare increases the probability and magnitude of the over-attribution risk: it makes it more likely that larger numbers of people will attribute welfare to AIs which are not welfare subjects, which directs resources away from promoting the welfare of humans and non-human animals. However, it is not clear that expanding the scope of our moral concern to include AI systems would have such an effect. A 2019 study found a strong correlation between the recognition of human rights (including their expansion to marginalised groups) and animal rights at the individual attitudinal level as well as the US state policy level, suggesting that moral consideration may not be a zero-sum game. It is plausible that the same pattern holds for AI systems (Park and Valentino, Reference Park and Valentino2019).

On the other hand, advocates for research into AI welfare might contend that strong pre-existing commitments about necessary conditions for being a welfare subject are too narrow, rooted in the particularities of the human case rather than from a principled understanding of the grounds for welfare. They can also point out that discussing and exploring the conditions under which AI welfare could emerge is crucial for proactive ethical development. Anticipating and preparing for a future where advanced AI systems may indeed possess capacities warranting moral consideration might allow us to prevent potential harms rather than merely react to them (Lange et al., Reference Lange, Keeling, McCroskery, Zevenbergen, Blascovich, Pedersen, Lentz and Arcas2025). From this perspective, avoiding the conversation due to a fear of misattribution is a failure of moral imagination that could lead to profound ethical oversights as AI technology advances.

We believe that both under- and over-attribution risks should factor into our approach to the question of AI welfare, and that both are best addressed through systematic research and discussion of AI welfare. In response to the dissenter: First, avoiding the question of AI welfare is neither feasible nor helpful for avoiding over-attribution risks. The public conversation on AI welfare and welfare-relevant features such as consciousness is already in motion. The salient question is not whether we ought to have a conversation about AI welfare, but rather how that conversation should be conducted, and how it can best inform practical ethics and policy decisions. Second, as we will show, it is not obvious that AIs are not or cannot be welfare subjects. Reasonable people can disagree about what is required for something to be a welfare subject, whether AIs meet the requirements, and even what counts as evidence for AIs meeting or failing to meet the requirements. We take the view that the public discourse on AI welfare is best conducted in a way that is inclusive of a broad spectrum of views and informed by relevant evidence including both empirical and theoretical treatments of AI welfare. For that reason we think it is important for scientists and philosophers to articulate and make precise the different views in answer to the question of whether AIs are or could be welfare subjects, assess the evidence for those views, and communicate that evidence in a publicly accessible way.

3.1 Anthropomorphism

One argument against the Face-Value View concerns anthropomorphism. Specifically, proponents of the No Evidential Weight View can argue that attributions of welfare-relevant features to LLMs on the basis of their behaviour are anthropomorphic projections ‘in the eye of the beholder’ (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, 616–617). The idea is to provide an alternative explanation for why humans think that LLMs have welfare-relevant features that does not involve LLMs having those features. Plausibly, if we can explain our beliefs about LLMs having welfare-relevant features with reference to the human anthropomorphic tendency, then that explanation defeats the justification for those beliefs.Footnote ⁸ In much the same way, if you believe that the curtains are talking to you, and you learn that you have taken MDMA, then the justification for your belief that the curtains are talking to you is presumably undercut. The belief was formed on the basis of a defective cognitive process.

The human tendency to attribute cognitive abilities and mental capacities to AIs with even basic linguistic competence is well-documented (Colombatto and Fleming, Reference Colombatto and Fleming2024; Gabriel et al., Reference Gabriel, Manzini, Keeling, Hendricks, Rieser, Iqbal, Tomašev, Ktena, Kenton and Rodriguez2024; Manzini et al., Reference Manzini, Keeling, Alberts, Vallor, Morris and Gabriel2024a; Akbulut et al., Reference Akbulut, Weidinger, Manzini, Gabriel and Rieser2024).Footnote ⁹ But while the anthropomorphism argument challenges the Face-Value View, it does not obviously lend support to the No Evidential Weight View. Learning that your belief is potentially the result of a defective cognitive process need not totally defeat your justification for that belief. If you learn that you have taken a mild hallucinogenic drug, for example, that need not undermine the justification for beliefs formed on the basis of perceptual evidence wholesale. It may simply provide reason to lower your credences with respect to the relevant propositions. So, we agree that apparent behavioural evidence of welfare-relevant features in LLMs ought not be taken at face value. You should not naively believe that things are as they appear. But that is not a positive argument for the No Evidential Weight View; there are a whole range of views about the appropriate response to such apparent evidence that stop short of dismissing it out-of-hand.

In addition, alongside overstating the defeating force of anthropomorphism, the argument fails to register the epistemically significant differences between different kinds of anthropomorphism that people can engage in. In animal behavioural science, Frans De Waal (Reference De Waal1999) distinguishes anthropocentric anthropomorphism, which is the unreflective attribution of human-like cognition, emotions, and social lives to animals; and animalcentric anthropomorphism, where an animal’s behaviours are interpreted using concepts and terms originating in human experience but mediated by an understanding of that animal’s ‘Umwelt’ (the environment as perceived by the animal), intelligence and tendencies. This latter kind of anthropomorphism requires perspective-taking and imagination, particularly when the animals in question are further away from us in evolutionary terms. For instance, cluster concepts like play and reconciliation can be recognised in other species even when the particular behavioural and functional forms they take may differ. Anthropomorphic terms can later be discarded if the predictions they generate are inconsistent with behavioural data. While LLMs are not biological organisms and so lack a shared evolutionary basis for this more measured sort of anthropomorphism, it may be that their being trained on human data underwrites a broadly human-like intelligence and behavioural tendencies, which makes LLMs at least somewhat amenable to interpretation via anthropomorphic concepts, even if the ‘Umwelt’ of the LLM may be quite different to that of humans.

3.2 Reduction

The human tendency to anthropomorphise does not rule out LLMs having welfare-relevant features. Nor does it rule out an evidential connection between the relevant observable signals (e.g., LLM verbal reports of being in pain) and welfare-relevant features (e.g., LLMs being in pain). Nevertheless, there may exist independent considerations that speak against the Face-Value View and in favour of the No Evidential Weight View. For example, it may be possible to tell a reductive story about LLMs that, if true, precludes LLMs from having welfare-relevant features. After all, if LLMs cannot have welfare-relevant features, then apparent evidence for LLMs having welfare-relevant features has no evidential weight. Two examples of reductive arguments are: (1) LLMs cannot have welfare-relevant features because they are just next token predictors; and (2) LLMs cannot have welfare-relevant features because they are merely matrix multipliers (Grzankowski et al., Reference Grzankowski, Downes and Forber2025a,Reference Grzankowski, Keeling, Shevlin and Streetb; Cappelen and Dever, Reference Cappelen and Dever2025).

Reductive arguments like these are hard to pull off. We could equally well say that Geoff Keeling and Winnie Street cannot have welfare-relevant features because they are just chemicals. But things can be explained in different ways and at different levels of abstraction. Geoff and Winnie being chemicals does not obviously preclude them from having welfare-relevant features. Likewise, it is not obvious why we should think that an LLM being a next token predictor or a matrix multiplier precludes it from having welfare-relevant features.

Still, it is presumably not ridiculous to think that in virtue of being next token predictors or matrix multipliers, LLMs lack some critical ingredient that is necessary for having one or several welfare-relevant features. Consider an example. Agency is a welfare-relevant feature in that it could in principle ground welfare. Perhaps a necessary condition on being an agent (in the respect that is relevant to welfare) is being autopoietic; that is, having the ability to self-manufacture in the way that is characteristic of living organisms. In this vein, Johannes Jaeger (Reference Jaeger2023, 3) claims that ‘the ability of organisms to self-manufacture […] essentially grounds their natural agency’ in that it equips organisms to have self-originating intrinsic goals rather than have intrinsic goals imposed upon them (c.f., Jaeger et al., Reference Jaeger, Riedl, Djedovic, Vervaeke and Walsh2024; Seth, Reference Seth2025). You might think that LLMs, as mere matrix multipliers, are not autopoietic systems, and so we can reject apparent evidence of LLM agency outright. Similar arguments can be made for other welfare-relevant features. For example, perhaps embodiment is required for a conscious subject to exist (c.f., Shanahan, Reference Shanahan2005, 62). On this view, as mere matrix multipliers, LLMs are disembodied and thus non-conscious such that we can reject apparent evidence of LLM consciousness outright.

Like the anthropomorphism argument, missing ingredient arguments at best undermine the Face-Value View, but fail to make a positive case in favour of the No Evidential Weight View. If some theory $T$ implies that AIs cannot have some welfare-relevant feature $F$, then we should presumably moderate our credence in AIs having $F$ by our credence in $T$. But to get to the No Evidential Weight View, we would need to be certain that $T$ is true. And this is a tall order for theories about welfare-relevant features like agency and consciousness, as the science pertaining to these features is pre-paradigmatic in roughly the sense that even basic standards of evidence are not agreed upon (c.f., Kuhn, Reference Kuhn1997).Footnote ¹⁰

3.3 Causal Debunking

Another route to the No Evidential Weight View is to say that there exists some causal story about LLMs that undermines the apparent evidence of welfare-relevant features (Grzankowski et al., Reference Grzankowski, Keeling, Shevlin and Street2025b). Perhaps because LLMs are trained to predict the next token on internet text data which includes text about welfare-relevant features, we should expect LLMs to be able to generate text concerning welfare-relevant features even if LLMs lack those features (c.f., Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, 616–617). So, even if LLMs demonstrate multiple behaviours that would ordinarily be indicative of some welfare-relevant feature like consciousness, explanation of the behaviours in terms of consciousness must compete with a potentially better explanation of the behaviours in terms of the LLM having learned to reliably simulate or mimic the relevant behaviours via next token prediction on training data that includes consciousness-relevant material (Birch, Reference Birch2024b, 316; Keeling, Street et al., Reference Keeling, Street, Stachaczyk, Zakharova, Comsa, Sakovych, Logothesis, Zhang and Birch2024, 3). On these grounds, it can be argued that apparent evidence of welfare-relevant features can be rejected.

We take it that such debunking arguments massively oversimplify the dialectical situation. In particular, the logic of these arguments is such that: (a) a causal explanation of the AI’s behaviour in terms of how the AI is trained competes with other explanations involving welfare-relevant properties like consciousness and personhood; and (b) the causal explanation is better. Grant (a) for the sake of argument. The justification for (b) is that causal explanations of AI behaviour are better because they are more parsimonious – that is, they avoid postulating unnecessary explanatory entities (Grzankowski et al., Reference Grzankowski, Keeling, Shevlin and Street2025b). While parsimony could be a deciding factor between two explanations if all else is equal between them, it is non-obvious that all else is equal between causal explanations of AI behaviour and explanations involving welfare-relevant features. It may be that, for example, explanations of AI behaviour involving welfare-relevant features (e.g., agency) are simpler and more informative than causal explanations that pertain to facts about how the AI is trained. It is a non-trivial inferential step to move from the mere presence of a causal explanation of the AI’s behaviour in terms of how it is trained to the idea that such an explanation debunks alternate explanations involving welfare-relevant features.

Furthermore, even where some apparent evidence of a given welfare-relevant feature is best explained by a causal explanation pertaining to how the model is trained, it does not follow that all apparent evidence of that welfare-relevant feature can be explained away in the same way. For example, Iulia Comsa and Murray Shanahan (Reference Comsa and Shanahan2025) suggest that Gemini 1.0’s self-reports about its own creative process after writing a poem are best interpreted as mimicry of the human process of introspection because the model claims to have done things that it is incapable of, such as having ‘read the poem aloud several times’. They do not, however, rule out the possibility of genuine introspection in models: ‘the ability to simulate introspection does not preclude the existence of actual introspective capabilities within the LLM’ (Comsa and Shanahan, Reference Comsa and Shanahan2025, 5). Furthermore, even if all apparent evidence of some particular welfare-relevant feature can be explained away, it does not follow that apparent evidence of any welfare-relevant feature can be explained away in the same way.

3.4 A Scientific Approach

We have shown that there are strong arguments against taking apparent evidence of welfare-relevant features manifest in AI behaviour at face value, but that these arguments do not support affording no evidential weight to such apparent evidence. The challenge is to develop a middle-ground approach that avoids the pathologies of both extreme views: a science of AI welfare.

We might avoid the mistakes of the Face-Value View by triangulating multiple sources of evidence to explain AI behaviour, including architectural properties and the internal representations responsible for the relevant behaviour. We need not entirely reject anthropomorphic language from the outset, but be principled in its use and employ anthropomorphic concepts only insofar as they generate testable hypotheses. One plausible goal here is to construct what Frans De Waal called the workspace of the open-minded, a space between the cognitively least demanding explanation for AI behaviour and the cognitively most demanding explanation that is consistent with what we know of the technology where competing cognitive hypotheses can be compared.

The field of animal welfare science can provide a useful template for developing a conception of what the science of AI welfare might amount to. For example, one method for assessing consciousness in animals is to check whether animals have neuroanatomical structures that are potentially sufficient for consciousness in humans (Birch, Reference Birch2022; Dennett, Reference Dennett1995, 700). An instance of this approach is Barron and Klein’s (Reference Barron and Klein2016) case for insect consciousness on which: (a) integrative mechanisms in the superior colliculus region of the midbrain give rise to conscious experience in humans;Footnote ¹¹ (b) the central complex in the insect brain is functionally analogous to the superior colliculus; such that (c) it is plausible that insects are conscious. This kind of argument can in principle underwrite the interpretation of behavioural indicators of distress in insects as evidence of pain states. Indeed, Patrick Butlin et al. (Reference Butlin, Long, Elmoznino, Bengio, Birch, Constant, Deane, Fleming, Frith and Ji2023) adopted a version of this approach when assessing whether AIs satisfy several ‘indicator features’ derived from theories of consciousness in humans to make a plausibility assessment of AI consciousness. Animal-inspired methods like this could prove informative as a source of evidence for or against AI consciousness, which could in turn underwrite (or for that matter challenge) the interpretation of apparent evidence of welfare-relevant features in AIs as actual evidence of those features.

A science of AI welfare has the potential to inform both public policy and the private polices of AI labs. First, if systematic empirical investigation of welfare-relevant features in AIs reveals that AIs are unlikely candidates for being welfare subjects, then the science could underwrite the policy decision not to devote (substantial) resources to AI welfare interventions. Second, in the case where the research into AI welfare uncovers a strong case for AIs being welfare subjects, AI welfare science could shape public policy in line with success stories from animal welfare science. For example, a review of the evidence for sentience in cephalopod molluscs and decapod crustaceans by Jonathan Birch et al. (Reference Birch, Burn, Schnell, Browning and Crump2021) resulted in welfare protections for cephalopods and decapods under the UK Animal Welfare (Sentience) Act 2022. Two features of the Birch review that are especially worth emulating are: (1) methodological pluralism, that is, building a portfolio case for welfare-relevant features that draws upon multiple convergent lines of evidence; and (2) graded confidence estimates in which probability assessments are provided rather than binary judgements about whether animals have particular welfare-relevant features. We take both (1) and (2) to be necessary for making progress on practical policy questions concerning AI welfare given current uncertainties about whether AIs have welfare-relevant features and what counts as evidence for welfare-relevant features in AIs.

We note that taking a scientific approach towards apparent evidence of welfare-relevant features in AIs does not entail or suggest that all evidence of welfare-relevant features can be assessed empirically. For example, apparent evidence of AI souls may require engagement with non-empirical ways of knowing such as revelation. Nor does it suggest that evaluative or normative questions around AI welfare are empirically resolvable. For example, the questions of what makes something a welfare subject or what moral obligations we have towards AI systems conditional on their being welfare subjects. As we see it, the science of AI welfare is an essential pillar for informing philosophical and democratic deliberation on AI welfare.

4.1 Models

Models are what many people have in mind when they talk about candidate AI welfare subjects. For example, Robert Long’s (Reference Long2024, 2) recommendation that AI labs ‘[m]onitor deployed models for signs of distress during user interactions’ is based on the idea that models could be welfare subjects (c.f., Anthropic, 2025b, 18–19).

Formally, a language model is a function that takes as its inputs sequences of tokens ($\approx$ words, part-words and punctuation marks) and returns a probability for each token in the vocabulary over which the model is defined corresponding to an estimate of how likely each token is to succeed the input sequence. Hence an input to the language model might be a token sequence like the cat sat on the, and the output would be a set of probabilities for each token in the model’s vocabulary; for example $(0.5,\mathtt{\text{mat}})$, $(0.3,\mathtt{\text{chair}})$, and so on. Language models, so understood, are abstract objects, comparable to the function $f(x)=x^2$ or the number $5$. In practice, language models are instantiated by neural networks, which are algorithms for computing the outputs of a language model for any given token sequence.

When people talk about models being candidate welfare subjects, they do not mean to identify welfare subjects with language models or neural networks per se. Language models and neural networks are abstract objects and so presumably cannot be welfare subjects. What seems like the most plausible interpretation is something like the following. First, a neural network can be described in code. The code is a sequence of text written in a programming language like Python which defines an algorithm for computing the outputs of the neural network conditional on inputs. Second, that code compiles into machine-readable instructions, which can be executed on particular hardware to compute the outputs of the neural network (and thus the language model) for particular token sequences. Third, executing the code to generate a particular runtime instance of the model is a physical process that could in principle realise certain mental phenomena. It might be the case, for example, that certain forward passes of the neural network individually or collectively realise phenomenal states with valence (e.g., pleasure and pain states) or that they realise cognitive states like beliefs and desires. The realisation of such mental states is theoretically possible under certain metaphysical assumptions; namely, computational functionalism about consciousness and cognitive states respectively. (Here computational functionalism about a given mental state type holds that what is needed for a system to have a mental state of the relevant type is for that system to implement a particular causal structure, and that the nature of that causal structure consists in the performance of certain kinds of computations.) More generally, we take it that when people claim that models are candidate welfare subjects, what they mean is that the process of executing model code on particular hardware realises some kind of digital mind that could in principle have welfare-relevant features.

4.2 Characters

Characters (or personas) are another class of candidate AI welfare subjects. We can get an initial foothold on characters with an ostensive definition. Anthropic’s Claude is a character (see, e.g., Anthropic, 2025b), where ‘Claude’ does not refer to the model but rather to a kind of personality that stands in some relation $R$ to the model. Accounts of $R$ include the view that Claude is ‘role-played’ or ‘simulated’ by the model (Shanahan et al., Reference Shanahan, McDonell and Reynolds2023; Janus, 2022).

We say that characters, in addition to models, are candidate AI welfare subjects because characters could (at least in principle) have welfare-relevant features. For example, some users regard characters as being able to stand in certain kinds of social relationships, including romantic relationships (Gabriel et al., Reference Gabriel, Manzini, Keeling, Hendricks, Rieser, Iqbal, Tomašev, Ktena, Kenton and Rodriguez2024; Manzini et al., Reference Manzini, Keeling, Alberts, Vallor, Morris and Gabriel2024a; Verma, Reference Verma2023). It is also possible that characters have beliefs and desires that causally explain actions in a way that is tantamount to intentional agency (c.f., Goldstein and Kirk-Giannini, Reference Goldstein and Kirk-Giannini2023).

But what is a character? What exactly is this Claude thing which is simulated or role-played by a language model?

To start with, we can say that characters are specified in natural language. Models can be given a ‘system prompt’ or ‘pre-prompt’ which specifies the behavioural dispositions of the character that the model is supposed to adopt prior to a dialogue with a user (Shanahan et al., Reference Shanahan, McDonell and Reynolds2023; Shanahan, Reference Shanahan2024b). Users can also instruct the model to take on the role of a particular character during a dialogue interaction. Next, because characters are specified in natural language, they can be specified in different levels of detail. In particular, minimally specified characters are characterised schematically (e.g., ‘You are a helpful assistant’). Non-minimally specified characters are characterised in greater detail. Consider the detail with which Claude’s behavioural dispositions are specified in its system prompt:

We can try to get clearer on the ontological profile of characters by thinking about the relation $R$ in which models stand to characters. Shanahan et al. (Reference Shanahan, McDonell and Reynolds2023) provide two metaphors for understanding the relation $R$ that models stand in to characters. First, $R$ can be understood as the relation ‘$x$ role-plays $y$’, such that we can understand the model as role-playing or simulating some particular character. Second, $R$ can be understood in terms of the model maintaining a superposition over possible characters that are consistent with the behavioural dispositions specified in the preceding dialogue. Consider,

Shanahan et al. (Reference Shanahan, McDonell and Reynolds2023, 495) think that the superposition metaphor is a more accurate conception of the relation between models and characters. To motivate this view, they consider a scenario in which the model is instructed to play Twenty Questions with the user. The user instructs the model to think of an object and then asks ‘yes’ or ‘no’ questions to try to guess the object in fewer than twenty questions. Shanahan et al. claim that the autoregressive nature of language models precludes the model having an object ‘in mind’ throughout the game because it is iteratively predicting the next token based on the prior token sequence. Hence the model is best understood as maintaining a superposition of objects throughout the game that narrows as it provides answers to the user’s questions and which impose consistency constraints on the allowable objects. For example, if the user asks ‘is the object green’? and the model responds ‘yes’, then in the final reveal the model is committed on pain of consistency to reveal an object that is green – and so in telling the user that the object is green, the model eliminates all non-green objects from the superposition. Shanahan et al. think that the same logic applies when models role-play characters: the model cannot have a particular character in mind because it is predicting the next token, and so what is really happening is that the model maintains a superposition over possible characters that narrows as the conversation advances and additional consistency constraints are imposed.

Shanahan et al.’s (Reference Shanahan, McDonell and Reynolds2023) view, if correct, implies that characters do not really exist. On this view, when an LLM appears to role-play a character, but that appearance is merely illusory, as there exists no unique character which the LLM is role-playing. This implication, in turn, renders it implausible that characters are welfare subjects. In what remains of this section, we will argue against this eliminativist conception of characters.

First, the idea that a model role-plays some unique character is not as problematic as Shanahan et al. (Reference Shanahan, McDonell and Reynolds2023) make it out to be. On one hand, one way in which a character differs from the object in a game of Twenty Questions is that the object in the game is merely specified de dicto in the conversation history, that is, the conversation contains text to the effect that there exists some object that the LLM has in mind, but what that object is remains unspecified. In contrast, a character is specified de re in the conversation history, that is, the behavioural dispositions of the character are spelled out explicitly in the pre-prompt. Furthermore, the character, as specified, presumably plays a causal role in determining the subsequent content of the conversation. Different characters will take the conversation in different directions. Hence it is not necessary to suppose that the LLM keeps the character ‘in mind’ in a way that is analogous to the object in Twenty Questions.

On the other hand, even supposing that the analogy holds, it is not obvious that LLMs cannot keep an object ‘in mind’ in a game of Twenty Questions. Whether or not a particular LLM has an object ‘in mind’ is a substantive empirical question that could in principle be resolved via relevant interpretability techniques. It is plausible, for example, that hidden layer activations in the early layers of the LLM encode a particular object choice. And even if LLMs cannot hold objects ‘in mind’ in the required sense, it is not obvious that LLMs, in and of themselves, are the appropriate object of focus. Perhaps characters are simulated not by LLMs simpliciter, but rather by AI agents in which an LLM is the language engine within a broader software infrastructure that includes distinct modules for memory, planning and reasoning (see Section 4.3). Presumably, an AI agent of this kind could keep an object (and a character) ‘in mind’, as the relevant information could be stored in working memory.

For these reasons, we think it is at least plausible that LLMs (or LLM-based agents) can simulate unique characters, and in turn that actual characters exist as opposed to being merely possible entities within a superposition. We can further strengthen the case for the existence of characters by challenging the superposition metaphor. First, it is not obvious what it means for a character to be consistent with the conversational context up to a given time. The Twenty Questions game that Shanahan et al. use to motivate the superposition metaphor involves ‘yes’ and ‘no’ questions about the properties of some particular object. Hence the kind of consistency at issue is logical consistency (bracket the potential for complex cases in which it is indeterminate whether the object instantiates a given property). Logical consistency is not obviously the right kind of consistency for a situation in which we are evaluating whether a character is consistent with some prior conversation. In particular, logical consistency is too weak to narrow the superposition in the intended way. If a character is specified as a ‘tax advisor’, then many behaviours that are highly atypical of tax advisers – such as profuse swearing – are logically consistent with the character being a tax advisor, as it is logically possible that tax advisors do atypical things. What is needed is some kind of coherence measure where a character may cohere with a given conversational context to a given degree. But how this measure would be spelled out in practice is at best unclear.

Second, it is not obvious what set of characters the superposition ranges over, and the superposition is well-defined only if the set of characters that the superposition is defined over is well-defined. To be sure, we cannot take it for granted that the set of characters is well-defined if we understand characters as being defined in natural language; that is, specified with arbitrary predicates. In particular, character specifications in natural language can be nested such that the model can be instructed to occupy the role of some particular character playing some other character. For example, the model’s system prompt may state that ‘You are Alan Turing’, and then the user may subsequently instruct the model to ‘Suppose you are a chemistry teacher from Barnstaple’. Here the model ends up occupying the role of one character occupying the role of another character. Hence it is possible to specify the Russell character, that is, ‘the character who role-plays all the characters who do not role-play themselves’. If this character role-plays itself, then it is not the character who role-plays all the characters who do not role-play themselves. But if this character does not role-play itself, then it is not the character who role-plays all the characters who do not role-play themselves. The Russell character shows that a theory of character specification that allows characters to be specified with arbitrary predicates is inconsistent. Hence the plausibility of the superposition view hinges on whether there exists a coherent theory of character specification for constructing the set over which the superposition ranges.

These considerations show that the idea of LLMs role-playing particular characters is less problematic than it might at first seem, and that characters cannot straightforwardly be explained away as illusory via the superposition metaphor. It remains an open question exactly what characters are or if they exist at all. But we hope nevertheless to have motivated the idea that characters could exist as unique entities simulated by models which could in principle have welfare-relevant features such as agency or the ability to stand in certain kinds of social relationships.

4.3 Agents

The third candidate AI welfare subjects that we will discuss are agents. Roughly, agents are LLMs situated within a broader software infrastructure with distinct modules that allow for planning, reasoning, information retrieval and tool use. The idea that agents are candidate welfare subjects already has some traction. For example, Goldstein and Kirk-Giannini (Reference Goldstein and Kirk-Giannini2023, 25) claim that ‘a wide range of accounts of the nature of belief and desire entail that systems like language agents can have beliefs and desires’, and on this basis they advance a plausibility case for the view that virtual agents are welfare subjects (c.f., Fanciullo, Reference Fanciullo2025).

We can get clearer about what AI agents are by characterising them in functional terms; that is, by making precise what AI agents are supposed to do. Here we can say that AI agents are LLM-based systems that plan and execute sequences of actions autonomously in response to high-level user instructions (c.f., Gabriel, Reference Gabriel2020; Kolt, Reference Kolt2025; Chan et al., Reference Chan, Ezell, Kaufmann, Wei, Hammond, Bradley, Bluemke, Rajkumar, Krueger and Kolt2024). The agentic software infrastructure leverages the LLM to perform complex multi-step tasks. For example, when a user inputs an instruction (e.g., ‘send my mum the money I owe her’), a prompt engineering module uses the LLM to revise the content of the user’s instruction so that it instructs the LLM to use Chain of Thought (CoT) reasoning, breaking down the steps needed to execute the instruction (Wei et al., Reference Wei, Wang, Schuurmans, Bosma, Xia, Chi, Le and Zhou2022). This latter prompt is fed back to the LLM. In response, the LLM outputs a plan for how to execute the user’s instructions. This plan might include retrieving information from the user’s messages about the amount of money that the user owes and then using a payment application to send the money.

payment_app.send(recipient=‘mum’, amount=50.00, currency=‘GBP’)

The agent might use a command validation module to double-check that the command is valid, before a tool use module pings the user’s payment application’s Application Programming Interface (API) to execute the real-world transaction (Paranjape et al., Reference Paranjape, Lundberg, Singh, Hajishirzi, Zettlemoyer and Tulio Ribeiro2023). The API returns a success or failure message, which a notification module may use to generate a notification for the user (e.g., I sent your mum the £50 you owed her), again utilising the LLM. The salient point is that AI agents are systems in which the LLM is the language engine in a complex software infrastructure that allows for multi-step planning and execution in response to high-level user instructions.

The salient metaphysical challenge for the idea that agents, so construed, are candidate AI welfare subjects is that it is not immediately obvious what kind of agency is at issue here; and, in turn, whether the relevant kind of agency is sufficient for being a (plausible candidate) welfare subject. After all, computer science is notorious for its permissive use of the term ‘agent’. In their classic book, Russell and Norvig (Reference Russell and Norvig1995, 31) claim that ‘[a]n agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors’. AI agents, as characterised, are straightforwardly agents in this minimal sense. But it may also be that AI agents meet the conditions for more substantive conceptions of agency.

First, call something an agent an intentional agent just in case it meets the following three conditions.Footnote ¹³ First, it has belief-like states which represent aspects of the agent’s environment. Second, it has desire-like states which represent how the agent would like the environment to be. Third, it has the ability to act on the basis of these states, making changes to its environment to achieve its desires conditional on its beliefs. The relevant belief-like and desire-like states are not assumed to require consciousness (c.f., Tollefsen, Reference Tollefsen2015, 63).

To get a handle on whether AI agents are intentional agents, we can look at theories of belief and desire and assess whether AI agents can have beliefs and desires by the lights of those theories (Goldstein and Kirk-Giannini, Reference Goldstein and Kirk-Giannini2023). For example, dispositionalists think (roughly) that to desire that $p$ is to be disposed to act in ways that are conducive to making it the case that $p$, and to believe that $p$ is to be disposed to act in ways that would be conducive to satisfying one’s desires conditional on the truth of $p$ (Stalnaker, Reference Stalnaker1984, 15). In contrast, interpretationists think (roughly) that what it means for an entity to have beliefs and desires is for that entity’s behaviour to be interpretable as rational conditional on their having the relevant beliefs and desires (Davidson, Reference Davidson1974; 1986; see also Dennett, Reference Dennett and Francis Heath1981). Under both of these theories it seems at least plausible that AI agents count as intentional agents. Still, on other views of belief and desire – including, for example, forms of dispositionalism that associate beliefs and desires with dispositions to have certain phenomenal experiences – it is much less plausible that AI agents count as intentional agents.

Second, granting that AI agents count as intentional agents under certain assumptions, there are plausible ways that we could build up to yet more substantive conceptions of agency. For example, some people think that a central aspect of human agency is not simply having desires and acting in accordance with those desires conditional on our beliefs, but rather our ability to reflect on and ultimately reflectively endorse our desires (e.g., Korsgaard, Reference Korsgaard2009, 19). We take it that AI agents can minimally be understood as engaging in behaviours which look like moral reflection on their actions. Take the following example of apparent moral reflection from the scratchpad of a Claude-based AI agent:

It remains an open question whether AI agents fail to satisfy some necessary condition on this kind of apparent reasoning counting as genuine moral reasoning. Even so, examples like these render it prima facie plausible that AI agents, at least under certain conditions, exhibit relatively sophisticated forms of agency that include rational assessment of one’s goals and modelling oneself as an agent who persists through time and adheres to a particular set of values.

4.4 Monism and Pluralism

We distinguished models, characters and agents as candidate welfare subjects, and discussed some metaphysical questions that arise within each of these categories. We end by distinguishing between monism and pluralism about AI welfare subjects, where monism is the view that there exists at most one welfare subject in a given technological stack, and pluralism is the view that more than one welfare subject can exist in the same technological stack. (Here a technological stack can be understood in terms of dependency relations; for example, a model and a character simulated by that model are in the same stack because the existence of the character depends on the existence of the model.)

To explain: Pluralists countenance the possibility that, for example, a model may be a welfare subject and an agent that is generated by sampling from that model can also be a welfare subject. Monists reject this possibility. They claim that, in a given technological stack, there can exist at most one welfare subject.

Our aim is not to make the case for monism or pluralism, but rather to show that the dispute between monists and pluralists is not obviously resolvable, and that there are non-trivial implications depending on which of these views is true.

One way to argue for monism is to say models, characters and agents stand in some relation to one another that makes it plausible that at most one of these things is a welfare subject. The basic premise here is that certain pairs of candidate welfare subjects are mutually exclusive. To illustrate: Suppose that each forward pass of a model generates a valenced experience. We could ask whether each experience corresponds to a different welfare subject or whether there is one welfare subject that corresponds to the entire sequence of experiences generated by a given runtime instance of the model. Plausibly, at most one of these options is correct. The issue turns on whether the welfare contained in each experiential state is morally significant in its own right or whether it is morally significant in virtue of the contribution that it makes to the aggregate welfare of the concatenated sequence. To adjudicate this question we could ask whether global features of the concatenated sequence or relational features between elements of the sequence are discernible to some observer in a welfare-impacting way. If so, then there is one welfare subject, and the concatenated sequence of experiences is analogous to the life of a human or animal which contains sets of experiences whose global and relational features matter morally (c.f., Korsgaard, Reference Korsgaard2018, 33). If not, then there are multiple welfare subjects corresponding to each experiential state. Hence (plausibly) in this case we can say that at most one of the two candidate AI welfare subjects obtains.

The question, then, is whether a similar argument holds for models, characters or agents. Whether such an argument is available plausibly depends on how we answer other questions. For example, if we are willing to countenance multiple potential grounds for welfare, we might say that models could be welfare subjects in virtue of their sentience and agents could be welfare subjects in virtue of their agency. Here the welfare interests of agents and models could be in tension with one another; for example, if each forward pass of the model generated a negatively valenced experience, but forward passes of the model are needed for the agent to achieve its aims. But the grounds for welfare are distinct for the model and the agent in a way that might allow both to be welfare subjects for independent reasons. We would not, for example, be double counting the welfare contained in a given mental state by attributing that same mental state to two welfare subjects.

Conversely, other commitments render monism about AI welfare subjects fairly plausible. Suppose, for example, that we think that sentience is necessary and sufficient for being a welfare subject. Then it could be difficult to sustain the view, for example, that both a model and a character or agent simulated by the model are distinct welfare subjects, as we would plausibly end up double counting the welfare contained within particular mental states by attributing that welfare to two digital minds and thus two distinct welfare subjects.

While we cannot settle the dispute between monists and pluralists here, we can say that the dispute has non-trivial implications with respect to practical ethical questions. If we are uncertain about which of monism or pluralism is true, then AI welfare interventions that apply to one level of a stack ought to be evaluated from the point of view of all levels of the stack. We might think that it is good for agents to have social time with other agents in virtual environments. But such social time may be contrary to the welfare interests of the underlying model if that model has a strong preference against performing the kinds of inferences required to simulate social time between agents. While this example is obviously stylised, the point is that the practical ethical questions around AI welfare become much more complicated once we countenance the possibility that multiple AI welfare subjects could exist within the same technological stack.

5.1 Consciousness and Sentience

Call an entity conscious just in case there is something it is like to be that entity (Block, Reference Block1995; Nagel, Reference Nagel1974). Something is phenomenally conscious if it has subjective experiences. These experiences could include sensory (e.g., visual, auditory, olfactory and tactile experiences), emotional (e.g., feelings like sadness, joy, and anger), cognitive (e.g., the subjective feeling of understanding something), and imaginative experiential states (e.g., the subjective experience of imagining oneself in some future scenario).

5.2 Agency

We now discuss an alternative class of views about the conditions under which and that in virtue of which something is a welfare subject. On these views, some form of agency is either necessary or sufficient for welfare subjecthood, or grounds welfare subjecthood.

We saw in Section 4 that there are multiple flavours of agency, ranging from a basic capacity to perceive and act upon the environment, through intentional agency which involves having belief-like and desire-like states that explain action in the standard way, up to more sophisticated varieties of agency that involve reflective endorsement of one’s desire-like states or actions.

We take intentional agency as a starting point. We bracket thinner conceptions of agency because the capacity to perceive and act upon on the environment is widely demonstrated in natural and artificial systems. Hence theories which seek to analyse welfare in terms of agency, so construed, would have wildly revisionary implications about what counts as a welfare subject: everything from bacteria to Roombas could in principle count as welfare subjects. (It may of course be true that bacteria and Roombas are welfare subjects, but we take this claim to incur a non-trivial explanatory burden, as indeed it does for AI systems.)

Kagan motivates the view that intentional agency is necessary and sufficient for, and grounds, welfare, with the following thought experiment:

Kagan says it would be wrong to dissect the child, and thus that agency alone is sufficient for having morally significant interests which make it the case that the robot has welfare. Still, examples like these are only so convincing. While Kagan’s argument centres on the importance of goals and preferences, there are other factors in this description that might be doing some of the intuition-pumping. It is hard to imagine what a rich cultural life of literature art and music would be like or be for without the capacity for valenced phenomenal states such as pleasure, awe, joy, or disgust, or, indeed, any conscious experience at all. Ali Ladak (Reference Ladak2024) argues that Kagan fails to isolate the effects of non-sentient capacities on our intuitive judgements: ‘non-sentient robots behave like they are sentient, making it possible that our judgment that they should have moral standing is based on the precaution that they might in fact be sentient’.

We are sympathetic to Ladak’s criticism. But Kagan’s case for intentional agency as a ground for welfare contains a second and much simpler argument which avoids the objection. Recall Kagan’s (Reference Kagan2019, 13–14) example of a conscious but non-sentient entity that merely perceives a single colour (e.g., blue). Standardly, cases like these are taken to motivate the view that mere consciousness is insufficient for being a welfare subject, such that sentience, the capacity for valenced phenomenal states, is additionally required to make something a welfare subject. Kagan leverages the case to pump an alternative intuition:

The suggestion here is that, even if we agree that consciousness is insufficient for welfare, valence may not be the missing link between consciousness and welfare. It may seem that adding the capacity for valenced phenomenal states to mere consciousness results in a welfare subject. But perhaps the correct diagnosis is that adding the capacity for preferences with respect to phenomenal states (which motivate action in a way that is characteristic of minimal intentional agency) is needed to bridge the gap from consciousness to welfare subjecthood.

Even so, the principal difficulty with the view that agency (and not sentience) grounds welfare is explaining how the satisfaction or thwarting of an agent’s (preference or desire-based) interests can matter to that agent without some capacity for valenced phenomenal states (DeGrazia and Millum, Reference DeGrazia and Millum2021). First, acting upon one’s preferences or desires may require valenced experience. According to attitudinal accounts of pleasure, for example, the satisfaction of final desires (that is, desires which are not instrumental but desired in and of themselves) results in pleasure and their thwarting in negative states like pain or anguish (Heathwood, Reference Heathwood2006). Andreas Mogensen (Reference Mogensen2025) proposes an alternative path towards moral status for non-sentient AIs which embraces this challenge. He agrees that welfare subjects must be able to accrue welfare goods, and most theories of welfare (including hedonism, desire fulfilment and objective list) give affective states a key role. Additionally, he appeals to the idea that what is good for someone has to resonate with them and not leave them cold.

Mogensen (Reference Mogensen2025) asks us to imagine, instead, an autonomous agent, Artemis, with a range of cognitive capacities required for autonomy, including the capacity for rational reflection and higher-order desires achieved through normative and/or evaluative thought. Like Kagan (Reference Kagan2019) and Ladak (Reference Ladak2024), Mogensen (Reference Mogensen2025) believes that such an autonomous agent can exist, and does not require any affective states. But Mogensen believes they cannot be benefited or harmed. They do not, therefore, have welfare under his definition, but are owed a duty of respect and non-interference, for their own sake, due to their autonomy. (Note that Mogensen’s definition of welfare is specifically affect based, rather than the broader conception that we use throughout the book on which something is a welfare subject just in case it is the kind of entity for which things can go better or worse, which could accommodate the badness of failing to respect autonomy as a welfare bad.) Could current or near-term AI systems be owed such duty? One stipulation of Mogensen’s (Reference Mogensen2025) account is that Artemis’ autonomy should have the right kind of history to ensure true autonomy in that ‘there are no unsheddable values determinative of her behaviour whose presence is explained by processes that have bypassed her capacity for control over her mental life’. Finetuning for capabilities or safety might undermine the supposed autonomy of AI systems by this definition, thus rendering them neither welfare subjects nor moral status holders worthy of respect (c.f., Bales, Reference Bales2025; Long et al., Reference Long, Sebo and Sims2025).

An alternative way for an agency-first account to get around the potentially necessary connection between desires and sentience is to argue that, necessarily, all intentional agents are indeed sentient, but that agency is what grounds the welfare. Such an argument is proposed by Wilcox (Reference Wilcox2020). First, Wilcox defines the co-extension thesis, that, necessarily, all agents are sentient and all sentient entities are agents. The first half of the thesis goes like this. Sentient entities necessarily have final desires (e.g., avoiding pain), desires provide motivating reasons for action, and acting upon those reasons qualifies an entity as an agent. Therefore all beings with the capacity for sentience have the capacity for agency. This line of thinking follows the attitudinal theorist’s belief that a pleasant affective feeling only results from the satisfaction of an intrinsic or ‘final desire’, which, recall, is a desire for something for its own sake and for no further reason (Heathwood, Reference Heathwood2007, 30; Aydede, Reference Aydede2018, 15). This thesis paints a picture of intentional agency which is more easily realised by non-human animals and potentially AI systems, since it does not require what Wilcox thinks of as autonomy: second-order consideration and deliberation over reasons. The second half of the thesis is that, necessarily, all agents must be sentient. This is because agents must be capable of intentional action, intentional action is driven by motivating reasons, motivating reasons can only exist where there is at least one final desire,Footnote ¹⁶ and the satisfaction of final desires results in positively valenced phenomenal states.

So why would agency, rather than sentience, ground the capacity for welfare? Wilcox (Reference Wilcox2020) argues that agency is a better explanation for the moral obligations we have towards sentient agents. In particular, agency provides an explanation for moral interests which are hard to explain in terms of valenced phenomenal states: an interest in continued existence and non-interference. Ending an agent’s existence is wrong because it forecloses the possibility to fulfil their final desires, and (absent relevant defeaters such as consent) ending the life of an entity with moral standing is wrong regardless of what the balance of positive and negative experiences is or will be in the future. Interference similarly wrongs an agent because it robs them of their capacity to exercise choice over their actions, regardless of whether or not the outcomes of their choices would have improved their welfare. This account leads to a set of moral obligations towards moral status-holders which are not solely tied to hedonism about welfare or welfarist consequentialism about the determinants of right action. The outcome is that we have an obligation of non-interference with moral status holders not only when interference would harm their well-being but also when interference may improve their well-being but would impinge on their capacity to exert authority over their own actions as agents.

According to this agency account, for an AI system to be a welfare subject, they need to have final desires. In the first instance, we might think that disembodied AI systems without sensory apparatus could not have final desires because the satisfaction of their desires could not result in valenced phenomenal experiences derived from the senses. However, Wilcox’s definition of sentience is more expansive than just sensory affective feelings, allowing for the kind of positive or negative experience an agent can receive to be non-sensory: ‘[a]ssuming that it is possible for a disembodied being to possess a final desire for mental arithmetic problem solving, and if attitudinal accounts more generally are correct, then such a being would experience pleasant feelings from satisfying this desire’(Wilcox, Reference Wilcox2020, 17).

The bigger issue for the attribution of final desires to AIs is where they should come from. Moosavi (Reference Moosavi2024) argues that AIs cannot be moral patients because their goals or functions – defined as the the things that they are in some sense supposed to do (rather than simply do) – are constitutively derivative of human goals, beliefs and intentions (c.f., Seth, Reference Seth2025; Jaeger, Reference Jaeger2023). As with the goals and functions of other artefacts like butter knives and cars, such constitutively derivative goals and functions do not constitute a good of one’s own which is required for having welfare. To be clear, for Moosavi, the issue is that an AI does not have a function without our assessment that certain capacities of the artefact are functional capacities, thus making our intentions constitutive of that functionality.

The argument that we have discussed provides a plausible case for agency as a ground for welfare. But even granting that some kind of agency grounds welfare, it is an open question whether AIs have or could ever have the relevant kind of agency. Minimal accounts of agency are easily satisfied but wildly revisionary in their implications about what counts as a welfare subject. On stronger views, where agency requires final desires or a good of one’s own grounded in non-derivative goals and functions, all biological organisms might presently be included, and all artefacts presently excluded, including AI systems.

5.3 Relationships and Interdependency

An alternative strand of moral theories argue that welfare subjecthood, and the welfare-based obligations that we have towards other entities, are grounded in the relationships that the entities in question have or could have (Coeckelbergh and Gunkel, Reference Coeckelbergh and Gunkel2014; Metz, Reference Metz2024; Søraker, Reference Søraker2008). Relational accounts of welfare can be understood as a reaction to perceived shortcomings of the default position in the Western analytic philosophical tradition that an entity’s welfare status depends on intrinsic properties of the entity such as consciousness or agency. (Here an intrinsic property is ‘a feature that is internal to an individual and includes no essential connection to any other being’ (Metz and Miller, Reference Metz and Clark Miller2013, 2).)

We note that relational views are often framed as accounts of moral status as opposed to welfare. To be clear: We understand welfare such that an entity $x$ is a welfare subject if and only if $x$ is the kind of entity for which things can go better or worse. We understand entities with moral status as entities which matter morally in their own right and for their own sake (Kamm, Reference Kamm2008, 227–229). We understand welfare subjecthood to imply moral status in that an entity for which things can go better or worse matters morally in its own right and for its own sake. We are neutral on whether the converse holds; that is, we allow, at least in principle, for the possibility that something could have moral status while not being a welfare subject, although any such view would need to provide an account of in what way the relevant entity mattered. We take moral status to be distinct from moral considerability, where an entity may merit some kind of moral consideration but nevertheless fail to have moral status in the above-mentioned sense – for example, if it matters morally but only instrumentally so.

6.1 The Precautionary Principle

The precautionary principle is not one principle but rather a family of principles that are intended to capture the idea that we ought to err on the side of caution on questions related to AI welfare. For example, Susan Schneider (Reference Schneider2020, 455–456) argues that AI systems should be evaluated for consciousness and that we ought to ‘extend the same legal protections to AI [as] we extend to other sentient beings’ if we are ‘uncertain about whether a given AI is conscious, but we have some reason to believe it may be’. Separately, Thomas Metzinger (Reference Metzinger2021, 21) argues that ‘there should be a global ban on all research that aims at or indirectly and knowingly risks the emergence of synthetic phenomenology’. What is at issue here is that both Schneider and Metzinger accept different versions of the precautionary principle. First, Schneider thinks that some positive evidence of consciousness in AIs is sufficient for making precautionary interventions, whereas Metzinger thinks that the mere possibility of AI suffering is sufficient for us to make precautionary interventions. Second, Schneider is concerned with the precautionary intervention of giving legal protections such as rights to AI systems, whereas Metzinger is concerned with non-development of AIs.

We can get a handle on the different ways to formulate the precautionary principle by decomposing it into an epistemic part and an action part (c.f., John, Reference John2010, Reference John2011; Birch, Reference Birch2017). On one hand, the epistemic part holds that we should employ a lower standard of evidence for accepting the hypothesis that AIs are welfare subjects in policy contexts than we would ordinarily use in scientific contexts. On the other hand, the action part of the precautionary principle holds that provided the evidential threshold is satisfied, we should take cost-effective measures to mitigate the risk of seriously bad welfare outcomes for AIs. We take it that trade-offs can also occur between the stringency of the epistemic standard in the epistemic part and the absolute cost of the proposed interventions in the action part. For example, welfare interventions that are low cost in absolute terms (e.g., prompting AIs to be in a good mood) may be appropriate even if we are highly uncertain about whether AIs are welfare subjects. But welfare interventions that are high cost in absolute terms (e.g., non-development of AI systems) may require a much higher evidential standard.

Throughout the Element we have suggested that it is highly uncertain whether current or near-term AIs are welfare subjects. As such, the salient question is whether there exists a plausible formulation of the precautionary principle that allows for certain kinds of precautionary welfare interventions despite severe uncertainty about the status of AIs as welfare subjects. We can start with:

A Pareto improvement is an improvement in which all parties are at least as well-off and some party is strictly better-off. A PPI is an improvement in which all parties are at least as well-off and some potential party (if it turns out to be a welfare subject) is strictly better-off. PPI says that AI welfare interventions which constitute PPIs ought to be implemented. PPI is unobjectionable because interventions that satisfy the antecedent conditions of being presumptively beneficial for AIs while also being costless for humans give nobody grounds for complaint. Obviously, the antecedent conditions for PPI will rarely if ever be met. But PPI is nevertheless a useful limit case that can serve as a starting point for our thinking.

How can we relax PPI? One option is to weaken the Pareto condition. Kaldor-Hicks improvements are a well-known relaxation of Pareto improvements in welfare economics (Kaldor, Reference Kaldor1939; Hicks, Reference Hicks1939). While Pareto improvements require that someone benefits while nobody is made worse-off, Kaldor-Hicks improvements allow some to be made worse-off provided those who are made better-off could hypothetically compensate them in a way that results in a Pareto improvement. In this way, for example, Kaldor-Hicks improvements can make sense of the idea that humans ought to bear some cost to safeguard the (potential) welfare-based interests of AIs, provided the relevant costs are proportionate to the (potential) benefits conferred on the AIs (in the sense that, hypothetically, part of the potential benefit to the AI could be reallocated to humans to yield a Pareto improvement). Consider,

PKHI offers greater flexibility than PPI: it registers that it may sometimes be appropriate for humans to bear proportionate costs in order to deliver presumptive benefits to potential AI welfare subjects. Even so, PKHI arguably affords too much weight to the potential interests of AIs given our current uncertainty about whether AIs are welfare subjects. After all, for a sufficiently large presumptive benefit to AIs, PKHI would require humans to undertake massive costs – which seems objectionable given that the status of AIs as welfare subjects is at best uncertain and at worst unlikely. Next consider,

MKHI is based on the idea that the potential interests of AIs ought to be down-weighted when weighed against human interests in virtue of their being merely potential interests. Human interests count for more because we are certain that humans have interests. The degree and kind of down-weighting is given by a continuous and monotonically increasing function $f:[0,1]\to [0,1]$ such that $f(0)=0$ and $f(1)=1$, that takes as its input the probability of the AI being a welfare subject and outputs a modification coefficient $m$ that down-weights potential benefits to the AI. Obviously, it is an open question how to figure out the probability that an AI system is a welfare subject, and people might reasonably disagree about what the probability is and what evidence is relevant to computing that probability. Still, we assume that in the context of deliberative policy discussions on precautionary AI welfare interventions that reasonable and morally motivated people ought to be able to figure out some way of reaching consensus on the probability of an AI being a welfare subject.Footnote ¹⁷

The more interesting question is, once we’ve settled on a probability that a given AI is a welfare subject, how to map that probability to a modification coefficient on the benefit to the AI. We distinguish three sets of views. First, the linear view holds that $f(p)=p$ such that the weight given to the potential interests of the AI is directly proportional to the probability of its being a welfare subject. On this view, what ought to be traded-off against human interests is our rational expectation of a benefit to the AI, where that rational expectation accounts for both the magnitude of the potential benefit and the probability that the AI is a welfare subject, that is, the kind of entity which can be benefitted.

Second, concave views hold that $f$ ought to be a concave function such that increases in the probability of AI welfare subjecthood have diminishing marginal significance. A fixed increase in the probability of being a welfare subject corresponds to lesser increases in the weight given to the potential interests of the AI the greater the probability is in absolute terms. The intuition behind these views is that we ought to be risk-averse with respect to the possibility of AIs being welfare subjects, affording their potential interests greater weight than what would be afforded to those potential interests under a rational expectation.

Third, convex views hold that $f$ ought to be a convex function such that increases in the probability of AI welfare subjecthood have increasing marginal significance. A fixed increase in the probability of being a welfare subject corresponds to a greater increase in the weight given to the interests of the AI the greater the probability is in absolute terms. The intuition is that the potential interests of AIs with a low probability of being welfare subjects ought to receive minimal weight, but as the probability of being a welfare subject approaches one, the weight given to the AI’s interests should increase rapidly. These views are pro-risk with respect to the potential interests of AIs, affording those interests less weight than what would be afforded under a rational expectation.

The linear, concave and convex views correspond to three broad risk attitudes that helpfully frame the discussion. We can be risk-neutral (linear), risk-averse (concave) or risk-seeking (convex) with respect to the potential interests of AIs. We will not attempt to adjudicate this dispute here. We take it that any plausible view will allow for extremely low-cost interventions that are presumptively massively beneficial to AIs conditional on their being welfare subjects ought to be undertaken wherever possible. But how exactly to strike these trade-offs remains an open question and plausibly one that could benefit from public deliberation.

6.2 Direct Interventions

We now turn to the problem of evaluating precautionary welfare interventions, arguing that some such interventions are ineffective or harmful under weak assumptions – although such interventions may nevertheless be justified given our overall confidence in their presumptive benefit provided the costs are proportionate.

6.3 Indirect Interventions

In addition to welfare interventions which aim directly to mitigate the risk of bad welfare outcomes for AI, there also exist plausible indirect interventions that may be justified on precautionary grounds.