The term ‘Artificial Intelligence’ (AI) was introduced in 1956 by American mathematician and computer scientist John McCarthy, who used it in a funding proposal for a summer research project at Dartmouth College (McCarthy et al., Reference John McCarthy, Rochester and Shannon1955). AI is myriad technologies that mutate and evolve, yet both the academic field devoted to its study and the industry that aims to realize it in its multiplicity are often said to have the nebulous aim, intimated already in McCarthy’s proposal, of making machines capable of displaying human-level, intelligent activity in all or most domains.Footnote 1 The history of AI is narrated as a sequence of alternating ‘springtimes’ and ‘winters’ in which an approach is supposedly made to this vague goal, then falters (e.g., Bostrom Reference Bostrom2014, pp. 6–8; Kurzweil, Reference Kurzweil2005, 263–266). In 2022, the year chatbots based on large language models (LLMs) seized the world’s attention, and OpenAI’s ChatGPT became for many identical to all AI, a massive springtime broke out, with a small cohort of companies securing unprecedented funding while their leaderships told stories about how general machine intelligence at the human level or, to use the current jargon, artificial general intelligence (AGI), was just around the corner.
This familiar narrative, which has continued to be woven by AI billionaires – and has perhaps only recently started to subsideFootnote 2 – serves to obscure how, in competing to build ever-larger LLMs, big AI companies became responsible for social injustices and environmental damage. Members of a high profile group of philosophers, ‘Effective Altruists focused on the long-term future’ or longtermists, aided the obfuscation by echoing the idea that AGI is likely close and arguing that trying to realize it is, morally, the most important and perhaps the most dangerous endeavour for all humankind. By boosting the hype around AGI, longtermists became complicit in serious harms. This article critically examines the longtermist tradition as a means of understanding what is being done to the world in AGI’s name and of recovering the ability to philosophize and act in resistance.
1. Longtermism’s Ancestry in AI
Longtermism emerged from conversations about the promise and perils of AI that took place primarily among AI researchers. This fact can be missed because the tradition gets described as a future-oriented variant of the utilitarianism-based philanthropic program Effective Altruism (EA). EA, which got its titular start in 2011 at the hands of two young Oxford philosophers, Toby Ord and William MacAskill, aims to make charitable gifts maximally effective per hour spent or dollar donated, and its original cause areas were poverty in the global South and animals in industrial agriculture.Footnote 3 Its originators are mostly seen as impressed by a famous 1972 argument of philosopher Peter Singer’s about practical implications of applying utilitarian principles to the suffering of the world’s poor (Singer, Reference Singer1972) as well as by Singer’s (Reference Singer2009) proposal for developing his argument into a charitable movement (e.g., MacAskill Reference MacAskill2015, p. 12). This origin-story leaves out EA’s roots in AI circles.
During the aughts, autodidact computer scientist and co-founder of the Berkeley-based Machine Intelligence Research Institute (MIRI) Eliezer Yudkowsky started (or co-started) two blogs devoted to a tradition he founded and dubbed rationalism. Rationalism credits decision theory with an apt image of reason, and it aims to ‘improv[e] human reasoning and decision-making’ so there are enough smart people around to create and control intelligent machines.Footnote 4 Some in Yudkowsky’s online community wanted to use rationalist ideas to determine how to do the most good, and one of his own posts about this implicitly utilitarian enterprise mentions ‘effective altruism’.Footnote 5 That was in 2007, four years before Ord and MacAskill introduced the phrase as the name of a movement and two years before they founded the organization Giving What We Can and began practicing effective altruism avant la lettre. Even if Singer’s work influenced online rationalist conversations about these issues,Footnote 6 Ord, who was active on the more significant of Yudkowsky’s two blogs, LessWrong, already in 2009,Footnote 7 would have been aware of routes to EA ideas, not only via arguments among philosophers about moral theory, but via debates among AI researchers about moral applications of rationalist notions. Ord would also have been aware that, in the heavily tech-populated LessWrong community, the main aim of rationality training was control of emerging AI systems. These matters would have been salient for Ord, who, starting in 2006, worked at Oxford’s Future of Humanity Institute (FHI), an institute partly concerned with risks of AGI. In 2005, Ord was already collaborating with Swedish philosopher Nick Bostrom, who would found FHI that year and give it this orientation (see Bostrom & Ord, Reference Bostrom and Ord2006). Although, as officially announced, EA wasn’t an AI-oriented movement, it stemmed partly from AI-researchers’ discussions about the AGI-related topics that would preoccupy longtermists, and it derived its soon-evident fundraising-virtuosity from its AI ties. Longtermism is, as investigative researcher Mollie Glieberman argues, better seen as the parent than the child of EA.Footnote 8
In 2017, when EA was taking off – developing new offshoots, directing many millions in grants, and securing billions in pledgesFootnote 9 – MacAskill proposed the monicker ‘longtermism’ for an EA-related ethical view that had long been championed at Bostrom’s FHI, Ord’s home institution, and that Ord was researching.Footnote 10 The view, which Ord defended in a 2020 book and MacAskill laid out in his bestselling 2022 What We Owe the Future, is that humankind is at a stage of technological development at which we could annihilate ourselves or go on to a radiant future, and that we should prioritize confronting threats to such a future. It is part of the meaning of ‘longtermism’ that a future that qualifies as suitably radiant is one in which, facilitated by advanced technology, immense numbers of humans or, rather, our digital descendants live on for billions or trillions of years, past the sun’s heat death, by colonizing other star systems. Longtermists may in theory differ about the biggest threats to this purported utopia, but most agree that anthropogenic threats are likelier than natural ones (such as asteroid impacts) and that, among the former, the biggest threat is likely AI and, specifically, machines with above-human-level intelligence lacking values aligned with ‘ours’ (MacAskill, Reference MacAskill2022, pp. 80–120 & 113, and Ord, Reference Ord2020, p. 165). In their respective books, Ord and MacAskill present themselves as breaking from standard EA and arriving at longtermism by applying a future-indexed strain of utilitarianism (MacAskill, Reference MacAskill2022, pp. 5 & 188–189, and Ord, Reference Ord2020, pp. 7–8 & 47–49). These accounts of longtermism’s provenance omit the fact that views that not only anticipate longtermism in essentials but provide the immediate inspiration for it were circulating in Silicon Valley decades earlier.
The first formulation of what today is called longtermism is in articles Bostrom published in the early aughts (esp. Bostrom, Reference Bostrom2002, Reference Bostrom2003a & Reference Bostrom2005c). In 1996, just before starting PhD work in philosophy, Bostrom became persuaded that technology would render many traditional areas of inquiry obsolete (Khatchadourian, Reference Katchadourian2015). He discovered a like-minded, if undisciplined, intellectual community on an email list managed by futurists who, evoking an entropy-opposing drive for order, called themselves Extropians (Bostrom, Reference Bostrom2005b, and Torres, MS). Extropians were advocates of a modern strain of transhumanism, and their email list, which included many engineers and computer scientists, exposed Bostrom to transhumanist ideas influential in AI circles. Modern transhumanism’s guiding conviction is that genetic engineering, AI, and molecular nanotechnology will enable us not only to enhance ourselves but to transcend the human condition, realizing the eugenic project of augmenting our intelligence and overcoming disease and aging. A small number of modern transhumanists hope to achieve extreme longevity while keeping their physical bodies. But most believe we will need to merge with machines, say, through processes of mind-uploading, and leave Earth to colonize exoplanets (see Thomas, Reference Thomas2025).Footnote 11 Bostrom’s enthusiasm for transhumanism was such that in 1998 he co-created the World Transhumanist Association (Bostrom et al., Reference Bostrom2005) and started translating transhumanist ideas into the idiom of analytic philosophy. In a 2002 article, he introduced the category, now used widely in AI and policy settings, of existential risk for dangers that could permanently destroy the ‘potential of humankind to develop’ into the sort of post-humanity transhumanists envision (Bostrom, Reference Bostrom2002, p. 5) In a 2003 article entitled ‘Astronomical Waste’, he presented a utilitarian case for holding that the expansion of the human population through space colonization is such a great good that reducing risks to its attainment should be our top moral priority as a species (Bostrom, Reference Bostrom2003a). At the time of these early articles, Bostrom regarded non-aligned superintelligent machines as a significant but not primary existential risk (Bostrom, Reference Bostrom2002, p. 7). By 2014, he regarded such machines as the biggest existential risk and had argued the point in a book, Superintelligence, that became a best-seller (Bostrom, Reference Bostrom2014). At that point, having run FHI at Oxford for almost a decade, Bostrom was an establishment figure. He had come close to converting speculative and academically disreputable, Silicon Valley transhumanism into mainstream Oxford analytic philosophy. Ord and MacAskill completed this project by writing their acclaimed book-length treatments of longtermism without mentioning transhumanism once.
This made it harder to see that longtermism’s tenets came directly from the tech-world while also making it seem serendipitous that tech leaders were so admiring. Microsoft co-founder Bill Gates declared he would ‘highly recommend’ Bostrom’s Superintelligence;Footnote 12 CEO of Tesla and xAI Elon Musk boosted the book’s message in a tweet in 2014 (see Robins-Early, 2024), and OpenAI’s Sam Altman wrote on his blog that the book was ‘the best thing I’ve seen’ on relevant topics and ‘well worth a read’ (Altman, Reference Altman2015). In 2022, Musk reposted a link to Bostrom’s article ‘Astronomical Waste’, commenting that it was ‘likely the most important paper ever written’Footnote 13 and also reposted promotional copy for MacAskill’s What We Owe the Future, declaring it ‘a close match for my philosophy’.Footnote 14 MacAskill reportedly had a massive publicity budget for the book due partly to support from Facebook co-founder Dustin Moskowitz (Torres, Reference Torres2022).
This adulation for longtermism might have been anticipated because it is directed at the output of researchers at institutes, in a growing international network, for which AI billionaires and centimillionaires themselves provide generous funding. Oxford’s FHI was created with part of a $100 million gift from IT consultant and engineer James Martin. Skype co-founder Jaan Tallin helped found Cambridge University’s Centre for the Study of Existential Risk and the longtermist Future of Life Institute (FLI), the latter of which received $14 million from Musk and got most of its funding from co-creator of the cryptocurrency Ethereum, Vitalik Buterin, who gave it $650 million in 2021 (Becker, Reference Becker2025, p. 21). Muskowitz and his wife Cari Tuna were among the co-founders of Open Philanthropy, which gives hundreds of millions annually to EA and longtermist projects,Footnote 15 including the Berkeley Existential Risk Initiative and Berkeley’s Center for Human-Compatible AI.Footnote 16 The funding of such think tanks leverages not just intellectual credibility but influence at the world’s seats of power. Longtermists like Ord and MacAskill advise US, UK and other national governments, as well as international organizations like the World Bank and World Economic Forum, and, in 2019, Open Philanthropy gave $55 million to create Georgetown University-based Center for Security and Emerging Technology (CSET),Footnote 17 first directed by one-time FHI researcher Jason Matheny, later a senior appointee in the Biden Administration, who, since 2022, is CEO of the RAND Corporation, which on his watch has received tens of millions from Open Philanthropy.Footnote 18 And so on.Footnote 19
These cashflows are noteworthy not only because longtermism originates in the world of AI but because it rehearses a worldview common in AI circles. Longtermism stresses concern for the wellbeing of the trillions of posthumans who may live in a distant techno-future and, construing the enormity of their numbers to outweigh other moral issues, treats overcoming impediments to this future as a, or even the, moral priority.Footnote 20 There seem to be no limits to the resources we should devote to rogue robots and other perceived ‘existential risks’, and we seem obliged to downplay the importance of actual social problems such as, say, structural racist and ablest bias, that are deemed ‘non-existential’, even if they cause great suffering and mortality. Among the ills longtermism shockingly dismisses are those caused by failures to properly regulate the building and running of the AI systems it promotes. Among the false dangers it outrageously implies must be combatted as ‘existential’ are social justice movements that call for limiting growth in the name of sustainable, equitable forms of life and that threaten growth-fuelled techno-utopias that depend on increasing energy use to maximize the cosmic spread of digital intelligences (see MacAskill, Reference MacAskill2022, Ch. 7).
The scope and nature of the AI-longtermism relationship have rarely intruded into general public awareness. There was a brief clamour, in 2022, when the crypto exchange FTX declared bankruptcy (e.g., Conroy, Reference Conroy2022) and, a year later, when its CEO Sam Bankman-Fried was convicted of fraud and conspiracy (e.g., McBain, Reference McBain2023), since it was known that McAskill counselled Bankman-Fried in 2012 to ‘earn to give,’ that MacAskill went on to advise FTX’s charitable fund, and that the fund in turn pledged millions of dollars to several of the EA and longtermist organizations MacAskill led and advised.Footnote 21 The scandal of the 2023 unearthing, by philosopher and critic of longtermism Émile Torres, of a racist email Bostrom wrote in 1996 on the Extropian email list (Torres, Reference Torres2023b), an occasion followed a year later by the shuttering of Oxford’s FHI, got relatively little attention, though the retention of the categories of fraudulent ‘racial science’ in Bostrom’s supposed ‘apology’ ought to have drawn attention to racism and eugenic thinking in the AI community.Footnote 22 With the money flowing, AI companies and their longtermist allies mostly sustain their self-serving narrative about being humanity’s best hope for a ‘vast and glorious’ future (Ord, Reference Ord2020, p. 217).
The situation is dire, because, as critics have shown, AI tools that aren’t properly scoped and tested do real damage and because Big AI is doing great environmental harm. Forging a commensurate political response is urgent for everyone committed to justice, equity, and a liveable planet. Since philosophers have helped forge the destructive path, this is an occasion for philosophical self-scrutiny and for a philosophical practice able and willing to challenge the status quo.
2. Springtime for AI
The question of who can best build ‘safe’ AGI is the framework in which major AI companies’ jockey for position in the era of LLMs. The story starts with DeepMind, co-founded by Demis Hassabis, Shane Legg, Mustafa Suleyman in 2010, acquired by Google in 2014, and initially funded by $1.85 million from venture capitalist, PayPal and Palantir Technologies co-founder, and tech right leader Peter Thiel. DeepMind was the first AI company explicitly dedicated to AGI (Torres, MS). Musk invested in DeepMind and became convinced that Hassabis was not taking the proper steps to make AGI safe (Hao, Reference Hao2025, p. 34). When, in 2015, Musk, Altman, Ilya Sutskever, and six others founded the non-profit OpenAI, with funding from Thiel as well as Altman and Musk (Hao, Reference Hao2025, p. 60), their declared goal was ‘artificial general intelligence for the benefit of humanity’.Footnote 23 When Musk left OpenAI in 2018, he claimed OpenAI couldn’t make AGI safe as a non-profit (Hao, Reference Hao2025, p. 75). That same year, Altman started a for-profit division of OpenAI, and the company released a charter saying its mission was to ‘ensure that artificial general intelligence … benefits all humanity’.Footnote 24 When, in late 2020, AI researcher Dario Amodei, who had moved to OpenAI in 2016, and his sister, AI entrepreneur Daniela Amodei, who had moved there in 2018, left the company to co-found Anthropic with other former OpenAI employees, they framed their move as driven by a desire for a better approach to good AGI (Hao, Reference Hao2025, p. 167; Torres, MS). Similarly, when in 2023, Musk founded xAI as a competitor to OpenAI, he did so ‘to build a good AGI’ (McBride et al., 2023). Meta CEO Mark Zuckerberg joined the fray in 2024 with his own push to build AGI.
Preoccupation with AGI makes sense in the context of modern transhumanism. Projecting a golden future in which technological advancement enables humans to transcend their limitations and become a race of super-intelligent, super-long-lived beings who enjoy super well-being (Thomas, Reference Thomas2025), this tradition treats AGI as the path to utopia. AGI is supposed to be a key step to artificial super intelligence (ASI), that is, to computers with above-human-level intelligence capable of improving themselves and triggering a self-sustaining, exponentially increasing cycle of technological progress, and this ‘intelligence explosion’ (Good, Reference Good1966) or ‘Singularity’ (e.g., Vinge, Reference Vinge1986 & Reference Vinge1993; for the term’s history, see Kurzweil, Reference Kurzweil2005, pp. 21–24) is taken to be humanity’s occasion for seizing its techno-glory-to-come. A major consolidation of money and power is foreseen to be required to build AGI, and central strands of transhumanist thought embed political ideals that seem to justify this. Although transhumanism is sometimes represented as free of any political orientation (Bostrom et al., Reference Bostrom2005), and although there is a progressive wing of modern transhumanism that accents democratic ideals of inclusivity (see Hughes, Reference Hughes, Ranisch and Sorgner2014, pp. 143–145), Silicon Valley’s transhumanism is mostly libertarian, treating individual autonomy, pared down government, and the ‘free’ flow of capital as essential for reaching techno-utopia (Thomas, Reference Thomas2025, esp. Ch. 1).
These transhumanist themes circulated widely in the 1990s in some key AI forums, such as the Extropian email list in which Bostrom participated. One self-designated transhumanist who was active on this list is Google researcher, inventor, and author Ray Kurzweil (Hagey, Reference Hagey2025). During the 1990s, Kurzweil wrote a couple of books (Kurzweil, Reference Kurzweil1990 & Reference Kurzweil1999) envisioning humanity’s future as one of augmented intelligence, freedom from disease, great longevity, and space colonization, and his 2005 bestseller The Singularity is Near popularized the notion of the Singularity (Kurzweil, Reference Kurzweil2005, also Reference Kurzweil2024). Toward the middle of the 1990s, while still a teenager, Yudkowsky joined the Extropian email list and for a number of years described himself as a transhumanist (Yudkowsky, Reference Yudkowsky2024). Some of Yudkowsky’s Extropian contacts supported him in founding the Singularity Institute for Artificial Intelligence (SIAI), the precursor to the Machine Intelligence Research Institute, where he is today, and, in 2005, Yudkowsky met Thiel, who, for a while, was a mentor and also a key funder of MIRI. In 2006, Thiel, Kurzweil, and Yudkowsky started the Singularity Summit, an annual conference that ran through 2012, and that, among other things, in 2010, the year DeepMind was founded, hosted talks by two of its co-founders, Legg and Hassabis (see Torres, MS).
Transhumanism’s influence in Silicon Valley reaches beyond those who identify as transhumanists. Thiel, who places Christianity at the centre of his belief-system, doesn’t use the label but has supported organizations dominated by transhumanists, such as, in addition to MIRI and the Singularity Summit, the anti-aging Methuselah Foundation (Hughes, Reference Hughes, Ranisch and Sorgner2014, p. 140) and Humanity+, the successor organization to the WTA, for which Thiel raised money in 2009, with an eye to replacing its left-leaning leadership with libertarian allies (Hughes, Reference Hughes, Ranisch and Sorgner2014, pp. 142–143).Footnote 25 A staunch proponent of politically autonomous spaces for companies to build capital, Thiel was an original funder of the transhumanism-influenced Seasteading Institute and remains a leading advocate and funder of ‘network states’ or ‘freedom cities’ (e.g., Anselmi, Reference Anselmi2025). Taking space to be capitalism’s ‘limitless frontier’ (Thiel, Reference Thiel2009), he regards the pro-sustainability environmental movement associated with Greta Thunberg as an ‘existential risk’ veering toward totalitarianism, the extinction of freedom, and the loss of all hope of a radiant technological future (Thiel, Reference Thiel2025). Thiel’s most famous mentees are Zuckerberg and Altman, the latter of whom is a major investor in anti-aging technologies (see Hao, Reference Hao2025, p. 197) and a fan of freedom cities. Some of Altman’s recent utopian musings depict us as on the verge of AGI and as already starting on a ‘gentle Singularity’ that might lead, by 2035, to ‘space colonization … and high bandwidth brain-computer interfaces’ (Altman, Reference Altman2025). Like Thiel, Altman has worked with Musk, who, as Torres notes, tacitly sympathizes with transhumanism in his professional projects. Musk’s company Neuralink aims to ramp up human evolution; his company SpaceX tries to hasten space colonization; his company xAI addresses so-called safe AGI; and so on (Torres, Reference Torres2022 & MS). Throughout Big AI there are similar suggestions of the transhumanist dream of a grand future in which we build AGI, ‘go to space and live forever’ (Becker, Reference Becker2025, p. 1).
Contestants in the AI ‘arms race’ sound transhumanist themes about AGI’s importance while debating to what extent it is also a danger. The disagreements get couched in terms of Bostromian ‘existential risk.’ At issue is the size of the ‘existential risk’ that an AGI not ‘aligned’ with human values would snuff out humanity and destroy its potential. Those who take this question seriously and call for efforts to solve the ‘alignment problem’ are AI ‘doomers’. Most AI leaders are doomers in the sense of maintaining that at least some care needs to be taken to avoid lethal AGI. In 2023, the longtermist FLI released an open letter calling for ‘all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4’. Among the moderate doomers who did not sign were Altman and Gates. Among the more severe doomers who did were Hassabis, Legg, Dario Amodei, and Musk.Footnote 26 There is also a doomer-extreme represented by Yudkowsky, who, a few days after FLI’s open letter, argued it doesn’t go far enough, that, as things are, a ‘superhumanly smart AI’ would kill all life on Earth and that the US should halt AI research and destroy any ‘rogue datacenter by airstrike,’ even at the risk of triggering a ‘full nuclear exchange’ (Yudkowsky, Reference Yudkowsky2023; also Soares & Yudkowsky, Reference Soares and Yudkowsky2025). Opposing all doomers are ‘accelerationists’ who favour unalloyed positivity about AGI and other technologies. One outspoken accelerationist is software engineer, venture capitalist, and friend of Thiel’s, Marc Andreessen, who in 2023 issued ‘The Techno-Optimist Manifesto,’ disparaging appeals for safe AI on the ground that ‘any deceleration of AI will cost lives’ and amount to ‘murder’ (Andreessen, Reference Andreessen2023). Setting aside the extremes, most AI heads sing in a heterogeneous choir of doomers raising the alarm on the supposedly inevitable technology that they are rushing to build, while also raising immense sums to keep it, they claim, from ending humankind (e.g., O’Neill, Reference O’Neill2023, p. 79).
The rallying-cry of doomers is AI safety, a cause area that includes only the ‘existential risk’ of a theoretical, vaguely conceived rogue future AGI. This leaves out AI ethics, which addresses harms related to things like lack of data privacy, algorithmic biases, deepfakes, fake news, generated hate speech and also things like the exploitation of workers doing data-annotation, extractivist injuries, and environmentally devastating water and energy use. There is no reason to think that it is possible to make AI systems safe in any ordinary sense without dealing with ethical issues having to do with great and even mortal harms (see Raji et al., Reference Raji, Smart, White, Mitchell, Gebru, Hutchinson, Smith-Loud, Theron and Burns2020). But doomers’ concern with AI safety, which typically leads them to favour ‘safety’-related democratic oversight, is often accompanied by efforts to block non-‘safety’-related regulation. Altman adopted a version of this stance in May 2023, a month in which, during testimony before the U.S. Senate, and in a blogpost, he appealed for ‘safety’ measures, and in which he also threatened to take OpenAI out of the EU on the ground that the then-in-draft, justice-oriented EU AI Act was intrusive ‘over-regulation’ (Hao, Reference Hao2025, Ch. 13; and Gebru & Torres, Reference Gebru and Torres2024). This stance makes sense to other doomers who believe that racial justice and related issues aren’t, as computer scientist and ‘godfather of AI’ Geoffrey Hinton put it on CNN in 2023, ‘as existentially serious as the idea of these [machines] getting more intelligent than us and taking over’ (O’Neill, Reference O’Neill2023). Some doomers, including both Hinton and president of FLI and co-author of its Open Letter Max Tegmark, try to put a kindly spin on the posture by insisting that social justice issues are urgent while still arguing we should prioritize AGI alignment because, as Tegmark put it in 2023, ‘once we’re all extinct … all these other issues cease to even matter’ (Bengio et al., Reference Bengio, Tegmark and Petty2023, discussed in Gebru & Torres, 2024).
So, though doomers differ from accelerationists in stressing safety, they also take AGI, the speculative project of a small cadre of immensely wealthy, Global North-based men, to be of paramount importance and to outweigh justice and environmental issues that are existential for so many. That is the stance to which longtermist philosophers give their moral imprimatur.
3. Longtermist Logic and a Double Defect
Longtermism is at base a strain of the moral theory consequentialism. Longtermists tend to insist on their own uncertainty about the correct moral theory (see MacAskill, Reference MacAskill2022, pp. 186–187; Ord, Reference Ord2020, pp. 56–57; and Bostrom Reference Bostrom2024, p. 24), but this superficially modest gesture hedges bets within only a restricted range of positions. Despite their uncertainty-claims, longtermists lay out consequentialist arguments that equate value with wellbeing and so count as utilitarian doctrines (for more on longtermists’ notions of value and wellbeing, see Crary, Reference Crary2021 & Reference Crary2023). In doing so, they adopt a morally significant methodological practice, viz., presupposing that value is recognizable from a dispassionate point-of-view-of-the-universe. This presupposition is morally substantial in seeming to enable us to treat wellbeing as a metric for comparisons of value anywhere, not only across space to the global poor and across species to other animals – points that longtermists’ Effective Altruist descendants stress – but across time to those in the very distant future.
In terms of its philosophical derivation, longtermism is a position in the field of population ethics, which derives from the work of philosopher Derek Parfit.Footnote 27 Population ethics is mostly an inside game, dealing with ethical conundrums that result from applying specifically utilitarian modes of thought to future humans. The action turns on questions about whether claims to moral rightness reflect total aggregate wellbeing, average wellbeing, or wellbeing above a particular threshold, as well as on questions about whether these claims reflect equal versus unequal distributions of wellbeing. Longtermists index moral assessments to total aggregate wellbeing, even while recognizing that this stance, a form of total utilitarianism, entails what Parfit calls the ‘Repugnant Conclusion’, namely, the conclusion that, compared with a population whose members have a good quality of life, a population many times larger whose members live barely tolerable lives is morally preferable (Parfit, Reference Parfit1984, Ch. 17; see also MacAskill, Reference MacAskill2022, pp. 179–184, and Ord, Reference Ord2020, p. 260). Resisting the seemingly incontrovertible idea that their posture’s tie to this conclusion is a reductio, longtermists barrel on. In 2021, a large group, including MacAskill and Ord argued that, in MacAskill’s words, ‘the fact that a theory of population ethics entails the Repugnant Conclusion shouldn’t be a decisive reason to reject that theory’ (MacAskill, Reference MacAskill2022, p. 181; for the statement, see Zuber et al., Reference Zuber, Venkatesh, Tännsjö, Christian Tarsney, Steele, Spears, Sebo, Pivato, Ord, Kuruc, Hutchinson, Yew-Kwang, Masny, MacAskill, Lawson, Gustafsson, Greaves, Forsberg, Fleurbaey, Coffey, Cato, Castro, Campbell, Budolfson, Broome, Berger, Beckstead and Asheim2021). This leaves it unclear why, for longtermists, we shouldn’t cheer a Matrix-like scenario in which vast numbers of humans are forcibly held, deluded but not in pain, for the harvesting of their bio-power.
Taking a page from transhumanism, longtermists claim that suitably enhanced posthumans have the potential to live on for billions of years and colonize the cosmos and that today we are at a heightened risk of destroying this potential. Against this backdrop, longtermists’ commitments imply that any situation in which eight billion humans lead meaningful, Earth-bound, democratic, just, and sustainable existences, living and dying for hundreds of thousands of years before peacefully going extinct, is worse, by orders of magnitude, than one in which trillions of posthumans live on into a distant cosmic future leading barely tolerable lives. It appears it would be a supreme moral feat to lead humankind to the allegedly preferable outcome and that, if there are ‘existential risks’ that could prevent this, it would be a supreme moral accomplishment to reduce the likelihood of such threats even by a fraction of a percentage. These projects appear to be so significant that they would justify almost any steps, including steps that result in mass near-term suffering and mortality. Hauntingly, longtermist moral logic thus echoes the reasoning of fascist dictators and sci-fi villains (e.g., Thanos in the Marvel Universe) who aren’t averse to mass death if it brings on futures they hail as utopia.Footnote 28 Compared to existential catastrophes, the great cataclysms of history, such as the two World Wars, the Black Death and smallpox, appear, from a longtermist perspective, in Bostrom’s words, as ‘mere ripples on the surface of the great sea of life’ (Bostrom, Reference Bostrom2002).
This moral logic is longtermists’ gift to AI leaders. Echoing longtermist claims, AI leaders tell us that AGI is humankind’s bridge to utopian glory and that building this bridge should be a, or even the, moral priority. Doomers add that unaligned AGI is likely the biggest threat to successful bridge crossing and that minimizing this threat, even minutely, is of paramount moral importance. All the while, there is doomer-accelerationist agreement that AGI matters more than any injustices and harms for which it is responsible if neglecting these things helps us secure a route to utopia.
This is a travesty of moral reasoning. Longtermism’s failure as a moral theory is traceable to its consequentialist spine. Like other consequentialism-inflected theories, it slides into forms of welfarism that are conservative in simply presupposing existing social arrangements.Footnote 29 This is because consequentialist doctrines lack the resources to assess complex social interventions directed toward transforming practices and institutions. Admittedly, consequentialists can undertake laborious calculations of the consequences of complex collective actions and call these calculations evaluations. But this response is point-missing because it just reaffirms the idea that we’re limited to abstract calculations without addressing the objection that the limitation is a philosophically unjustified obstacle to social understanding.Footnote 30 Consequentialists’ reliance on the god’s eye method that seems to enable them to quantify and aggregate values anywhere prevents them from registering the sorts of social circumstances that repeatedly cause misery and that liberating social movements aim to change (e.g., weak political institutions that leave room for corruption, iniquitous laws that make it probable that poverty is inherited and social hierarchies maintained, etc.).
Consider overlapping anti-racist, feminist, and Indigenous rights movements engaged in struggle against social systems and structures that inflict physical, social, and emotional wounds and cause intergenerational setbacks. Many of the repeating injustices for which these movements seek redress are invisible apart from an appreciation of the history of the structures in question. The abstract, quantitative methods of consequentialism-influenced theories are incapable of illuminating the structures in ways that teach us about the kinds of steps needed to change them. Worse, such methods veer toward strengthening harmful social mechanisms, since one way these mechanisms perpetuate themselves is through obfuscation that hides their workings. Wedded to abstract methods, consequentialist approaches in ethics cannot avoid these liabilities.
This is both a fundamental moral critique of longtermism and a key to understanding why it receives backing from some of the world’s largest tech players. These individuals are among the biggest lottery winners in history of capitalist socio-economic arrangements. They accept longtermism’s version of the greatest-good-logic, with its insensitivity to structural injustices and ecological devastation, because it aligns with capitalism’s core tendency to roll on, regardless of structural injustices and ecological damage, toward the greatest amount of ‘free’ market-produced wealth.
Longtermism’s nucleus, shared with other positions in populations ethics, is the belief that an abstract account of an action’s consequences on future populations’ well-being provides the basis for the action’s moral assessment. But such wellbeing-calculations not only fail to enable us to judge the rightness of actions, they also obscure the institutional and interpersonal circumstances that are the touchstones of responsible moral judgment. Longtermist approaches in ethics are a moral train wreck, and the wreckage extends to AI companies’ reliance on them. Projections of the total amount of digital wellbeing in an AGI-enabled, utopian techno-future not only fail to show that it is right to neglect harms that developments in AI are causing right now, but obscure the circumstances that would enable us to judge what is demanded of us. The moral absolution that longtermism seems to offer AI leaders is an empty show.
This takedown of AI companies’ longtermist self-justifications does not depend on doubts about whether AI companies are really hastening toward AGI. A case for such doubts would amount to an additional critique. And such a case could be made. AGI or powerful AI is ill-defined, and even those most intent on it are, as Dario Amodei once put it, ‘in the awkward position … we don’t know what it looks like’ (cited in Hao, Reference Hao2025, p. 142). The rationale for investing hope in LLMs turns on thinking that these ‘pure language’ models are candidates for natural language understanding, and LLMs can converse in ways that give the impression of intentional speech. But they manage this through sensitivity to the relative frequency of expressions in their databases (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021). They have weaknesses in the areas of accuracy and causal reasoning that have been dramatized by studies showing they ‘collapse’ when fed the output of other LLMs (Shumailov et al., Reference Shumailov, Shumaylov, Zhao, Gal, Papernot and Anderson2024) and that figure in ongoing technical debates about whether understanding will require a new approach that supplies worldly ‘grounding’ (see Hao, Reference Hao2025, Ch. 5; also Marcus, Reference Marcus2025). Yet the industry is still organized largely around efforts to build bigger LLMs. Even if accuracy-gains are made on this path, there’s still room for a philosophical intervention related to the idea, at play in many accounts of AGI, that the successful candidate will be able to self-improve and begin a process of explosive technological growth. A machine that is literally in the business of self-improvement would need to be endowed with agency, which, on a plausible classic view, involves the ability to step back from one’s impulses to believe or do particular things, and to ask whether one should believe or do those things.Footnote 31 It’s not clear why we should expect LLMs to become agential or, to use the AI-world’s neologism, agentic (see also Thorstad, 2025). Those committed to the current race to AGI are entitled to observe that complex systems sometimes have novel emergent features. But if talk of AGI’s imminence is based on the hope of such features, it appears, as Signal Foundation President Meredith Whittaker puts it, that this talk is ‘much closer to an article of faith, a sort of religious fervor, than it is to scientific discourse’ (cited in Heaven, Reference Heaven2023).
The moral theory longtermism is bankrupt and incapable of justifying anything, so a fortiori it cannot justify the quest to build AGI. Nor is it clear that what is going on in AI circles is in fact accurately described as such a quest. The defences longtermists offer of the ‘AGI arms race’ are afflicted by a double defect. They are morally empty, and their claims about the extraordinary moral importance of building ‘safe’ AGI serve as a mere ideological cover for the efforts of a small number of individuals to enrich themselves by building larger LLMs. There is no excuse, of the sort AI leaders and longtermists suggest we have, for overlooking the harms of this damaging enterprise.
4. AI’s Harms and Injustices
In 2018, Eritrean, Stanford-educated computer scientist Timnit Gebru moved from Microsoft to Google where she would, for a few years, co-lead the company’s Ethical AI team, charged with ensuring that AI products didn’t perpetuate racism or other discriminatory practices. A few years earlier, Gebru had read a groundbreaking Propublica study showing some AI tools used to assess the risk criminal offenders would re-offend were systematically biased against Black people (Angwin et al., Reference Angwin, Larson, Mattu and Kirchner2016; also Perrigo, Reference Perrigo2021). While at Microsoft, she had worked with Canadian-American computer scientist Joy Buolamwini on a study showing that, when used on two widely used datasets for facial recognition that are overwhelmingly composed of lighter-skinned subjects, three commercial gender classification systems worked best for lighter-skinned individuals and worst for darker-skinned women (Buolamwini & Gebru, Reference Buolamwini and Gebru2018). Around the same time, Buolamwini founded the Algorithmic Justice League which, in 2020, led a successful campaign to get Amazon, Microsoft, and IBM to temporarily stop sales to police of facial recognition technology (Hao, Reference Hao2025, p. 172).
At Google, Gebru began to focus on LLMs. After a few years at the company, she agreed to work collaboratively with her Google Ethical AI co-lead, American computer scientist Margaret Mitchell, and two academics on problems related to the increasingly large and less discriminated datasets on which the models rely and co-wrote a paper describing how bias is perpetuated by standard methods for managing these datasets (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021; for an account of the industry’s turn to larger and more unwieldy datasets, see Hao, Reference Hao2025, Ch. 5). The authors underline problems with strategies designed to capture the best represented ideas, with reliance on sites like Reddit, Wikipedia, and Twitter, the content of which is mostly created by men, with a complementary neglect of small oppositional blogging sites, and with filtering out as ‘unintelligible’ fledgling or very local liberating idioms. ‘In all cases,’ they write, ‘the voices of people most likely to hew to a hegemonic viewpoint are also more likely to be retained’. ‘This means,’ they add, ‘that white supremacist and misogynist, ageist, etc. views are overrepresented in the training data, not only exceeding their prevalence in the general population but also setting up models trained on these datasets to further amplify biases and harms’ (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, p. 613). The resulting harms depend partly on the fact that ‘humans mistake LM output for meaningful text’ (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, p. 616). Some harms are simple functions of re-creating hegemonic outlooks, sometimes in subtle ways (e.g., ‘referring to women doctors as if doctor itself entails not-woman’ (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, p. 617) and sometimes in clearly derogatory ways, involving hate speech, microaggressions, and dehumanization. There are also issues with injuries that result when LLMs or word embeddings from them are used as components of systems for classification or other tasks (e.g., job selection, loan assessment, lesson plans, hospital discharges, etc.) (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, p. 617). Yet a further type of problem arises when bad actors use LLMs to generate conspiracy theories and recruit extremists. All these problems, Gebru and her co-authors stress, can be addressed with thoughtful practices of ‘curation, documentation, and accountability’ (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, p. 615), but, if not adequately addressed, LLMs will injure and further marginalize socially exposed populations.Footnote 32
This project is part of a large body of research about ways new AI tools hurt the marginalized. Some contributors to this corpus are, like Gebru and her colleagues, concerned with the perpetuation of bias. This includes Safiya Umoja Noble whose 2018 book sets out to show that Google searches replicate racist worldviews (Noble, Reference Noble2018), and it includes Bangladeshi-American data scientist Rumman Chowdhury, who, while directing an ethics team at Twitter in 2020, made the case that Twitter’s algorithms tended to amplify the political right (O’Neill, Reference O’Neill2023, p. 78). Other scholars stress that the implementation of even unbiased AI tools, can harm socially precarious people. British economist Seeta Peña Gangadharan studies how automated systems for things like welfare benefits, housing, loan-applications, and jobs leave those whose needs aren’t met by them locked out (O’Neill, Reference O’Neill2023, p. 78). American tech journalist Karen Hao, in turn, spent years tracking how big AI companies practice ‘disaster capitalism’, identifying people in economic crises in, e.g., Kenya, Colombia, and Venezuela to work, for little, doing forms of data annotation such as content-moderation, which can involve pouring through reams of text or images to flag the most horrific material for removal and thus ruining one’s mental health for a meagre and unreliable wage (Hao, Reference Hao2025, Ch. 9).
After Gebru’s collaborative study sailed through ordinary approval channels at Google, a company vice president reached out to say there were problems with it. Gebru was told either to retract the article or take her name and those of her Google colleagues off it. She countered that she would do so only if the objections to the study and the names of those making them were revealed (Perrigo, Reference Perrigo2021; see also Hao, Reference Hao2025, Ch. 7). Mitchell was fired three months later. Today their collaborative study has a canonical status in the field of responsible AI.
The construction and running of LLMs comes at a substantial environmental cost. To support progressively larger LLMs, AI companies have built sprawling data processing centres, some bigger than the campuses of the largest US Ivy League universities, and, as Hao notes, some companies like Google, Microsoft and Amazon build their data centres, which run 24/7, ‘in threes to have a backup for the backup in case any facility goes down’ (Hao, Reference Hao2025, p. 288). These are mineral-intense endeavours, using significant amounts of copper and lithium. They require enormous amounts of water that is potable and so ‘clean enough to avoid clogging pipes and bacterial growth.’ Yet around twenty percent of the campuses are built in places with already stressed watersheds (Hao, Reference Hao2025, p. 288). By 2027, their combined water-use is projected to surpass that of all of Denmark.Footnote 33
These data centres consume colossal amounts of energy. A landmark 2019 study by a team led by American computer scientist Anna Strubell notes that most of the energy for natural language processing isn’t derived from carbon-neutral sources (Strubell et al., Reference Strubell, Ganesh and McCallum2019, p. 3645) and estimates that training a then-standard LLM took fifty-five times the average person’s annual carbon budget. Subsequent researchers such as Gebru and her collaborators were able to update Strubell’s research, documenting the increase in energy-use as models grew (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, §3).Footnote 34 In the years after Gebru was pushed out of or left Google, tech giants began to hide ‘even more of their models’ technical details, making it … hard to estimate and track their carbon footprints’ (Hao, Reference Hao2025, p. 297). But it is clear they are growing. On 21st January 2025, the second day of his second term in office and a day after withdrawing the U.S. from the Paris Climate Agreement, President Trump announced Stargate, a joint private venture to devote up to $500 billion to new data centres by 2029 (Smith Goodson, Reference Smith-Goodson2025). In April 2025, the International Energy Agency estimated that by 2030 data processing centres would need as much energy as Japan today,Footnote 35 thereby, as AI researcher Kate Crawford put it, ‘adding a whole new industrial nation onto the grid’ (Crawford, 2025).
The environmental impact of LLMs, like the technology’s other harms, falls most heavily on already vulnerable populations. For example, xAI has built a massive computing facility, ‘Colossus,’ in South Memphis, a predominantly Black community. The facility requires more energy than local utilities can supply, so it runs partly on polluting gas turbines. This will not shock anyone familiar with the scholarship on environmental racism. ‘It is’ Gebru and her co-authors observe, ‘well documented in the literature on environmental racism that the negative effects of climate change are reaching and impacting the world’s most marginalized communities first’ (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021, p. 612).
So, the practices of AI companies that longtermism wrongly invites us to regard as morally justified are practices that inflict grievous harms, including devastating environmental injuries, and they disproportionately inflict these harms socially marginalized and racialized people the world over.
5. Philosophical Morals
The story of longtermism is partly a tale about how money – in the form of fancy research institutes, jobs, titles, book endorsements, publicity budgets, and ties to governments and elite universities – has given shoddy and immensely damaging ideas a foothold among philosophers and the wider public. This tale’s basic yet perennially pertinent moral is that worldly appeal doesn’t reliably indicate good ideas and can mask bad ones.
Longtermism’s story is also partly a related tale about the many lives of utilitarianism. In 1961, Iris Murdoch declared ‘there should have been a revolt against utilitarianism; for many reasons it has not taken place’ (Murdoch, Reference Murdoch and Bradbury1975 [1961], p. 18). There are well-known normative critiques of utilitarianism such as the one John Rawls advanced in 1971 when he wrote that, in aggregating value, utilitarians fail to ‘take seriously the distinction between persons’ (Rawls, Reference Rawls1971, p. 24). When Rawls launched this attack on utilitarianism, he was inheriting a Kantian idea of the unquantifiably precious ‘dignity’ of every human being (e.g., Rawls, Reference Rawls1971, p. 289). Human dignity is a politically radical idea in a world in which nations’ wealth tends to get measured, in a quasi-utilitarian manner, in terms of the total value of goods produced, and in which national wealth is accordingly consistent with the misery of many. But Rawls situates his radical idea within an entrenched metaphysic, of the sort also taken for granted by utilitarians, on which the world is drained of value and on which its every feature thus reveals itself to abstract survey. Given the fit of utilitarianism with dominant politico-economic modes of thought, it should come as no surprise that, in the absence of a revolutionary rethinking of the metaphysic that informs our social imaginary, utilitarian thinking returns again and again. The anti-utilitarian point Murdoch made in 1961, while consistent with Rawls’s, cuts deeper because it calls for a metaphysical revolution. Murdoch presents herself as struck by the fact that there had been, in Anglo-American philosophical circles, no sustained critical reflection on the metaphysical and epistemological assumptions that modern utilitarian theories take for granted. Her observation remains true today, and it bears directly on the longtermism-AI-saga.
Murdoch’s critical ire is directed toward the metaphysical assumption that reality is hard in the sense of being bereft of moral value (Murdoch, Reference Murdoch1958). This assumption, which is a signature of European modernity and a cornerstone of contemporary analytic philosophy’s mainstream, appears to license us to regard the world as indefinitely extractable, instrumentalizable, and commodifiable. One of the great themes of Murdoch’s work is that we need to question such a metaphysic if we’re to grasp challenges of getting our worldly circumstances into view in a manner suitable for ethics. We fail to get these things into view if we succumb to the pressure, exerted by the metaphysic and cheered on by longtermists, to survey the world from a supposed ‘standpoint of the universe’. What is required instead is an openness to the prospect of a value-laden world that is accessible only to thought that explores historical, cultural, ecological, and other evaluative perspectives and assesses their cognitive power. That is the kind of demanding thought required to register the gravity of the structural racist, ableist, and sexist injuries, and of the environmental harms, that AI companies, abetted by longtermism, are inflicting, and it is the kind of demanding thought we need if we are to formulate an appropriate political response to Big AI’s calamitous enthusiasm, now on lurid display in the US, for dismantling democratic institutions, accommodating fascist politics, and building network states in which Capital is free to exploit workers and ravage the Earth.Footnote 36
