1.1.1 Unmet Demand

Deloitte reports that the COVID-19 pandemic exacerbated mental health concerns and triggered declines in well-being, with a dramatic rise in the prevalence of depression, anxiety, post-traumatic stress symptoms and stress (Aiyer et al., Reference Aiyer, Bucaille and Westcott2021; Clark & Boulos, Reference Clark and Boulos2022: 8). The World Economic Forum estimates that over 970 million people worldwide live with a mental illness (Klopper, 2025), while the World Health Organisation estimates more than one billion people worldwide live with a mental health condition (World Health Organisation, 2025: vii). Accessing professional mental health support is often difficult, slow and costly (Heaney, Reference Heaney2025; Hickie, Reference Hickie2023; Stringer, Reference Stringer2023), for example, only one in ten people with depression worldwide receive adequate treatment (World Health Organisation, 2025: viii). The economic costs of poor mental health are predicted to exceed $16 trillion by 2030 (Klopper, 2025). The issue is particularly acute for young people: the prevalence and impact of poor mental health are increasing at an alarming rate, yet only a small proportion of young people receive appropriate care (McGorry et al., Reference McGorry, Mei and Dalal2024).

In this context, the appeal of developing chatbots and related digital tools for mental health support is clear (World Economic Forum and Deloitte, 2021). Since the advent of the mobile phone (Flore, Reference Flore2023: 33), digital mental health interventions have expanded in both voluntary and coercive settings (Bossewitch et al., Reference Bossewitch, Brown and Gooding2022; Gooding, Reference Gooding2019). In voluntary settings, direct-to-consumer digital mental health support tools operate in different ways. Some link consumers with a mental health professional (e.g., 7 Cups; BetterHelp), while others are designed for use within a professional mental health counselling service (e.g., SilverCloud; Thrive). Others support diagnosis and treatment advice by mental health professionals, such as through experiments in digital phenotyping (Tekin, Reference Tekin2021: 450). Finally, and the focus of this Element, specifically designed therapy bots provide mental health support directly to consumers, while the more general offerings, AI companions and generative AI chatbots, may also be used by consumers for mental health support.

1.1.2 Growing Investment and Use

The numbers involved in the direct-to-consumer digital mental health support market are not trivial. In 2021, the American Psychological Association calculated that app stores offered 10,000 to 20,000 products expressly purporting to provide mental health support (The Economist, 2021; World Economic Forum and Deloitte, 2021: 5). In 2025, Deloitte observed that $10 billion had been invested in innovative mental health and well-being companies since 2020 (Gates, Reference Gates2025).

The figures for AI companions are also impressive. In 2025, there were 337 active for-profit AI companion apps worldwide, which had been predicted to generate over $120 million by the end of that year (Perez, Reference Perez2025). Use of these products is rising. The Economist reports a 2025 survey finding that 42% of high school students had interacted with, or had a friend who had interacted with, an AI as a friend, and 18% as a romantic companion (Economist, 2025a). Character.AI alone is reported to have 20 million active users in 2025 (Economist, 2025a).

Consumers are also seeking mental health support from general-purpose generative AI chatbots, such as ChatGPT, Gemini or Claude. A 2025 study reported that 13.1% of a representative sample of young adults in the United States used general-purpose AI chatbots for mental health advice (McBain et al., Reference McBain, Bozick and Diliberti2025). Increased reliance on chatbots for mental health support among young people has also been reported in the United Kingdom (Hall, Reference Hall2025a) and Australia (Cross et al., Reference Cross, Bell and Nicholas2024).

Whether due to their greater conversational engagement or their less demanding approach to mental health, general-purpose generative AI chatbots and companion AI may be outpacing purpose-built therapy bots and other direct-to-consumer digital mental health support tools in delivering mental health support. The Economist reports that

Although these various products differ in the support they offer to provide for well-being or good mental health, as the next Section discusses, the underlying technologies are likely similar, albeit with varying guardrails.

1.2.1 Digital Tools Specifically Designed for Mental Health Support

Digital mental health support tools use web and app-based interfaces – including recommendations, dashboards and, increasingly, chatbots – to provide services relevant to mental health support. Some of the popular or commonly discussed direct-to-consumer mental health support tools and their self-described service are listed in Table 1.

As the descriptions in Table 1 show, it is not always clearly apparent what kind of service a direct-to-consumer mental health support tool provides. A review of popular tools available on the market suggests that they offer three main categories of service: mindfulness recommendations, mood tracking and, increasingly, therapy bots.

Mindfulness and meditation recommender tools provide curated recordings, videos and written content to support consumers in these practices. The tools may seem only loosely connected to mental health support. However, developers promote the close relationship between mindfulness and mental health, primarily in their capacity for reducing symptoms of poor mental health, such as anxiety or sleeplessness, with meditations, mindfulness techniques and general lifestyle advice.

Mood tracking tools allow consumers to record their experiences of mental health and then provide dashboards and other visualisations to assist them in tracking those symptoms. Mood tracking may be combined with monitoring physical health, notably, a service offered through the Apple Health app. Mood tracking and recommender tools, although tailored to the customer, are essentially static tools, providing resources with little interaction with the user, leading to interest in chatbots for mental health support.

Therapy bots use established clinical practices to support consumers in managing symptoms of poor mental health. A common support provided through therapy bots is cognitive behavioural therapy (‘CBT’). Cognitive behavioural therapy is a form of structured psychotherapy that emphasises identifying and replacing unhelpful thought patterns and behaviours with more adaptive ones. It is used to treat a number of mental health conditions, such as depression, anxiety and eating disorders (InformedHealth.org). The attraction of combining established therapeutic techniques with chatbot functionality is in providing an interactive conversational exchange that engages the user, as opposed to merely directing information at them.

Therapy bots purport to replicate clinical practice, and this means that their interactions with consumers are restricted to conversations that mimic established clinical practice. They do not ‘just chat’; they recommend. It appears that consumer users may prefer greater companionship and fewer dictates from their therapy bots. The retention rates of digital mental health tools are relatively low: studies have found that user engagement typically drops off after just fifteen days (Baumel et al., Reference Baumel, Muench, Edan and Kane2019; Adams et al., Reference Adams, Davies, Wattanatakulchat, Galante, Miller, D’Alfonso and Van Dam2026).

1.2.2 AI Companions

The phenomenon of AI companions refers to app or web-based chatbots that provide ‘friendship’ through text-based conversation and varying degrees of emotional, romantic and sexual intimacy (Purtill, Reference Purtill2023; Muldoon, Reference Muldoon2024). Consumers may create and interact with AI companions in a game-like virtual world experience using avatars they have chosen or designed. Some of the more well-known AI companions and their self-described service are set out in Table 2.

As can be seen from Table 2, AI companions promote their capacity for lifelike interactions. While these interactions may have a purpose of entertainment, they mimic human companionship and may be used in much the same way that humans relate to friends or partners (Bernardi, Reference Bernardi2025). Studies have shown that AI companions are being used by consumers for online virtual or imaginary companionship and as a way of exploring social or sexual identity. AI companions are also being used as a resource for responding to relationship problems, loneliness or poor mental health (Cann, Reference Cann2025). This is a form of support that humans often look to from their friends. It is, therefore, perhaps unsurprising that consumers might also seek this kind of support from an AI companion. Indeed, from the perspective of consumers, the boundary between AI tools providing mental health support based on established psychological practices and offerings providing companionship may be highly porous.

Both specifically designed therapy bots and AI companions make claims to the kind of insights that might be seen as supporting mental health, at least in the vernacular sense, along with friendship, understanding or care. Notably, some AI companions promote their capacity to address symptoms of poor mental health. For example:

Conversely, therapy bots also promote humanlike interactions that bring them close to the space occupied by AI companions. For example:

The mixed messaging in these claims may be confusing for consumers: are they interacting with a lifestyle product with health benefits or a chatbot trained in established psychological techniques? Of course, AI companions only promote companionship. It is possible that consumers may expect a higher level of expertise from general-purpose generative AI chatbots.

1.2.3 Generative AI Chatbots

Outside the relatively specialised services provided by therapy bots and the imaginary friend-like services offered by AI companions, humans are turning to general-purpose generative AI chatbots, such as ChatGPT (OpenAI), Claude (Anthropic) or Gemini (Google), for mental health advice and support (Haensch, Reference Haensch2025; Hall, Reference Hall2025a; Wallace, Reference Wallace2025a). General-purpose generative AI chatbots are conversational programs powered by generative AI, which is based on large language models. The same models may inform other kinds of chatbots, such as those used specifically for therapy (see 2.2.1). The contrast is that general-purpose chatbots are not restricted to any one particular use.

For consumers, the attractions of using general-purpose generative AI chatbots for mental health may lie in a combination of factors. Generative AI chatbots are easy to access (Cross et al., Reference Cross, Bell and Nicholas2024: 6). They interact in a fluent and conversational style. General-purpose generative AI chatbots do not try to assist users to change their behaviours or suggest therapeutic practices (Wallace, Reference Wallace2025a). It is also possible that consumers hold unrealistic expectations about the level of expertise, responsiveness and care that can be expected from generative AI chatbots given their underlying technology.

2.2.1 Recommender Systems

Some direct-to-consumer digital mental health support tools rely on recommender systems to provide broadly relevant information to consumers about meditation, mindfulness or other healthy practices (see, e.g., Calm, sign up). The tools collect information about the user through an initial questionnaire and, typically, interactions between the tool and the consumer. This data may then be processed by a machine learning model to provide targeted recommendations based on inferences about the individual user’s likely preferences or interests (Chen & Nakamori, Reference 70Chen and Nakamori2021). For example, the Headspace mindfulness app is promoted as ‘the mental health app for every moment’, while the underlying technology is described as follows:

2.2.2 Dashboards

Mood tracking tools rely largely on dashboards to represent information entered by consumers, often combined with recommendations relating to the patterns of behaviour or mood revealed by that information. Even more personalised insights into users’ preferences and emotional state might be gleaned from collecting biometric data. In particular, Apple offers an emotion tracking tool that combines insights from biometric data collected from the Apple Watch (Apple). This kind of data is no doubt highly valuable to the developers of the tool, in refining their services, developing new products, for research or commercial endeavour (see further Section 3.2).

2.2.3 Rule-Based, NLP and Generative AI Therapy Chatbots

Recommender systems simply provide content. Chatbots are software programs designed to decode written language and respond appropriately to questions, requests or statements. Chatbot technology allows real-time, responsive interactions with users. Modern chatbots vary in their language capacity (Onlim, 2024; Pierce, Reference 82Pierce2024).

Chatbots can provide a more interactive experience for consumers, including advice for better mental health and structured therapeutic interventions, such as cognitive behavioural therapy. Chatbots are becoming increasingly prominent forms of digital mental health support, including for example through aHead, Elomia, Youper, Woebot and Wysa. In 2025, the provider of recommendations for mindfulness practices, Headspace, introduced a chatbot, ‘Ebb’, for well-being and mindfulness support (Headspace, ‘Meet Ebb’). AI companions also make use of chatbot technology. For example, ‘Replika uses a sophisticated system that combines our own Large Language Model and scripted dialogue content to create more natural and engaging conversations’ (Replika, How Does Replika Work).

Chatbots themselves may operate in different ways. Some use rule-based programs to determine their response to statements made by the user, while others combine natural language processing, a form of machine learning, with rule-based systems (Adamopoulou and Moussiades, Reference Adamopoulou and Moussiades2020; Caldarini et al., Reference Caldarini, Jaf and McGarry2022) and, increasingly, deploy generative AI to inform text-based interactions.

2.3.1 Accessibility and Ease of Use

Direct-to-consumer therapy bots and other direct-to-consumer digital mental health support tools are commonly promoted as offering an easy to use and accessible response to the otherwise unmanageable demand for mental health services. Improving access to psychotherapy was among the attractions attributed to Eliza: ‘If the method proves beneficial, then it would provide a therapeutic tool which can be made widely available to member hospitals and psychiatric centres suffering a shortage of therapists’ (Weizenbaum, citing Colby, Reference Weizenbaum1976: 5). Similarly, the developers of many current direct-to-consumer mental health support chatbots state a purpose of making mental health care ‘accessible for everyone’ (Youper). For example:

At their best, direct-to-consumer therapy bots serve people who are not in crisis. This may in turn reduce demands on human mental health professionals, allowing them to respond to more complex or critical cases. These services can be delivered at relatively low cost, at any time and from any location. For some, the attractions of digital mental health support tools go beyond efficiency or accessibility. They may also appeal to consumers with low levels of trust in formal medical systems (Lucas et al., Reference Lucas, Gratch, King and Morency2014). Young people, in particular, may feel more comfortable receiving mental health support from a chatbot or app (Keirerleber, Reference Keirerleber2022). Tools that utilise generative AI offer the promise of personalised, expert and convenient support (Ghose, Reference 75Ghose2021). They appear to allow anonymity (consider whether the extensive data collection practices accompanying these tools genuinely allow this), seem empathetic, and are never judgmental.

These promoted attractions of direct-to-consumer therapy are in some ways amplified for AI companions and general-purpose generative AI. Not only are the latter options easily available; companion AI and general-purpose generative AI chatbots carry no agenda of encouraging the consumer to ‘work’ towards better mental health, they appear purely to entertain or serve. And they are sometimes said to ‘know’ the needs of the consumer user.

2.3.2 Personalisation

Therapy bots and other direct-to-consumer digital mental health support tools are often promoted for their capacity to provide a personalised response to consumers as opposed to the generic (and often static) information provided through online portals or webpages (Chen & Nakamori, Reference 70Chen and Nakamori2021; Robb, Reference Robb2024). Therapy bots, in particular, are promoted not merely as offering ‘off the shelf’ content but also as using technology to tailor material directly to ‘you’, the consumer.

For example:

This promoted capacity for personalisation in therapy bots may be more limited than the headline statements of the developers suggest, and may not correspond with consumers’ expectations.

Notably, the data profiles on which direct-to-consumer therapy bots and other direct-to-consumer digital mental health support tools rely are not drawn from the personal knowledge that might arise from the interactions that occur between health professionals and consumers. Instead, responses are based on statistical correlations between the individual user and the profiles or segments drawn from the training data to develop the digital mental health support tool. Thus, recommendations or predictions based on data profiles are only ever an approximation of what might be suitable for individual consumers because the data is merely a proxy for the attribute the tools are trying to predict (Hildebrandt, Reference Hildebrandt2022). Generative AI works in a similar way, because any response to a human prompt is based on probabilities learnt from the training data set not a deep understanding of language, or emotion. Nonetheless, the combination of an interactive conversation and personalised content may give rise to an impression that therapy bots ‘care’ about their consumer users.

2.3.3 Empathy, Expertise and Insight

Along with personalisation, the promotion for direct-to-consumer digital mental health tools often stresses the capacity of their chatbot offerings for empathy, expertise and insight. For example:

The marketing for AI companions similarly often emphasises their capacity for humanlike emotion. For example:

Consumers may attribute similar characteristics even to general generative AI chatbots. Research by Myra Cheng et al. has found that US consumers ‘perceive AI as warm and competent’ (Cheng, Reference Cheng2026b).

Opinions may differ on whether humans should be turning to chatbots for mental health advice. Specialist therapy bots may provide useful mental health advice, as indeed may companion AI and general-purpose generative AI chatbots (Gilbert, Reference Gilbert2025). It is less clear they provide empathy or care. For a consumer, talking to a chatbot might feel like talking to a friend or therapist. But AI chatbots are neither friends nor therapists in the way those terms are normally understood. AI chatbots only ever mimic human manifestations of care, empathy or other interpersonal interactions; they do not feel them (Paterson, Reference Paterson2025). For consumers, mimicked emotion may not detract from the sense of connection they experience in interacting with a chatbot. Humans interacting with Eliza found the experience meaningful. Nonetheless, there may be normative objections to this kind of ‘fabricated’ interaction (Ciriello et al., Reference Ciriello, Hannon, Chen and Vaast2024; Sharkey & Sharkey, Reference Sharkey and Sharkey2021).

AI chatbots cannot exhibit the sort of genuine empathy that comes with friendship or the insight that comes with a genuine therapeutic relationship between a human health professional. It seems problematic and even misleading to suggest the contrary. Additionally, AI companions and generative AI chatbots are commercial products sometimes made available for a fee or at the cost of personal data collection for training, which must affect any loyalty they might show to the human consumer.

Relatedly, although they may purport to ‘care’, AI companions and general generative AI chatbots have no purported expertise in the field of mental health. Indeed, companion AI and generative AI chatbots have been criticised for failing to recognise and appropriately respond to suicidal ideation (Turc, Reference Turc2022; Daws, Reference Daws2020; Atillah, Reference Atillah2023; Xiang, Reference Xiang2023a; Duffy, Reference 72Duffy2024) or even encouraging such behaviours (Duffy, Reference Duffy2025; Chatterjee, Reference Chatterjee2025). Generative AI chatbots have a tendency to sycophancy, in the sense of reflecting back the apparent preferences of the user, which Myra Cheng et al. observe is harmful to ‘social judgment’ (Cheng et al., Reference Cheng, Lee and Rapuano2026a) and has led to reports of AI psychosis (Hudon & Stip, Reference Hudon and Stip2025; Moore et al., Reference Meese and Tan2026).

At this point in time when AI is relatively new, there is therefore considerable scope for humans to misunderstand the nature of their relationship with a chatbot therapist, who has neither insight nor empathy for the human, even if they are able to offer suggestions for better mental health that have some validity (Khawaja & Bélisle-Pipon, Reference Khawaja and Bélisle-Pipon2023). Weizenbaum himself worried that ‘if, as appears to be the case, the public’s attributions are wildly misconceived, then public decisions are bound to be misguided and often wrong’ (Reference Weizenbaum1976: 8). Perhaps the somewhat confused nature of the relationship between humans and AI therapy bots might be forgiven if these tools are providing significant therapeutic benefits. However, therapy bots carry a number of as yet unresolved risks and limitations.

3.1.1 Studies and Surveys

There is relatively little empirical or clinical evidence about the long-term performance of therapy bots and other direct-to-consumer digital mental health support tools. This may not be surprising given the products are relatively new to the market. But it does mean they should not be over-promoted by developers.

3.1.2 Regulating Therapeutic Products

Therapeutic devices, such as products used to diagnose and treat medical conditions, are typically subject to safety protocols imposed by drug or therapeutic goods laws. These laws are in turn overseen by designated regulators such as the Therapeutic Goods Administration (‘TGA’) in Australia, the Medicines and Healthcare Products Regulatory Agency (‘MHRA’) in the UK or the Food and Drug Administration (‘FDA’) in the US.Footnote ¹ The goal of regulators is to ensure that medical devices are effective and safe before entering the market. Product safety requirements for medical devices are scaled according to the assessed risk of the product, with higher risk products requiring clinical evaluation (e.g. randomised controlled trials), documentation and monitoring.Footnote ²

The therapeutic device regulators in the US, UK and Australia have tended to exercise only a light-touch regulatory approach to digital mental health support tools. In this regard, the approach taken by the FDA in the US has been influential, given that many therapy bots have been developed in that jurisdiction (Duggal et al., Reference Duggal, Brindle and Bagenal2018; Wallace & Pestaina, Reference Wallace and Pestaina2023). In 2016, following changes to its constituent legislation, the FDA stated it would not regulate devices intended only ‘for maintaining or encouraging a healthy lifestyle’ (Food and Drug Administration, 2016, 2026).

So-called ‘wellness’ tools have typically been subject to minimal regulatory oversight on the ground that they do not fall within the FDA category of medical device (Shuren et al., Reference Shuren, Patel and Gottlieb2018). This approach has been justified by the FDA on the grounds that ‘active FDA oversight would provide little to no public health value while unnecessarily delaying patient access to potentially beneficial technologies’ (Shuren et al., Reference Shuren, Patel and Gottlieb2018: 338). This approach has been criticised. Scatterday argues that it represents a policy decision to ‘decrease the “regulatory burdens” associated with app development to drive marketplace innovation and technological advancement’ (Reference Scatterday2022: 653; see also, Taylor, Reference Taylor2021: 2281). Mattioli observes that since the FDA ‘put its waiver into effect, makers of unapproved apps branded as “wellness” tools rebranded their products as medical interventions’ (Mattioli, Reference Mattioli2021: 10). Torous et al. observe that ‘well over 99% of publicly available mental health related apps [are] marketing themselves as well-being and not health devices’ (Torous, Reference Torous, Linardon and Goldberg2025: 160).

A similar approach has been taken in Australia and the UK. In both jurisdictions, tools that are considered ‘health products’ (Department of Health, Disability and Aging, 2025), and do not involve ‘assessing risk, diagnosing, predicting, monitoring, treating or preventing mental health conditions and/or symptoms’, are outside the ‘software as a medical device designation’ (MHRA, 2025: 10). Examples of such tools include sports performance (MHRA, 2025: 8, 17) and healthy sleep monitoring (MHRA, 2025: 16). In Australia, there is an exception for tools that are ‘based on established clinical guidelines that are referenced and displayed in the software in a manner that is reviewable by the user’ (Department of Health, Disability and Ageing, 2024).

More recently, the FDA has addressed the interest in using generative AI to inform mental health support tools, and the rise of AI therapists. In 2025, the FDA committed to ongoing oversight recognising that:

The UK regulator has released guidance that includes considering the level of functionality of a mental health tool in classification for the purposes of the regulatory regime (MHRA, 2025: 13). This means a tool using a chatbot for interactions will have higher functionality, and hence more stringent oversight, than a tool that provides information as fixed text (MHRA, 2025: 22–23). In Australia, the TGA recommended a review into digital mental health tools using AI, considering that the current exclusions from regulation may no longer be appropriate (Department of Health, Disability and Ageing, 2025). In the UK, regulatory guidance considers promotional material in characterising the device as falling inside or outside the ‘software as a medical device regime’ (MHRA, 2025: 8, 17), and this has also been suggested in Australia (Department of Health, Disability and Ageing, 2025: 13). It is clear that more research and clinical evidence on the efficacy and safety of digital mental health tools in responding to anxiety and depression is required (Torous et al., Reference Torous, Linardon and Goldberg2025). Such research takes time and, in the interim, there is still a need for the products to be acceptably safe.

In this light, statements that the products produce genuine therapeutic or clinical benefits should be viewed with caution. Therapy bots and other direct-to-consumer digital mental health tools are not mediated through trained health professionals who could explain the limitations of the underlying studies or assess whether the product is suitable for a given user. This context may justify demands for care in promotional statements, or even support concerns that some promotional statements pointing to qualified health benefits are misleading. This is a topic that is considered in more detail in Section 3.4.2. The absence of a medical practitioner in providing mental health support also raises issues of privacy and confidentiality, along with data protection.

3.2.1 Data Practices

Any product claiming to provide personalised recommendations will rely on collected and processed personal data to some degree, and in the case of therapy bots and other digital mental health support tools, potentially sensitive personal data about health-related matters. Indeed, this lies at the heart of these kinds of self-monitoring or self-help tools. Ghose (Reference 75Ghose2021) describes health support tools generally as leading to ‘the “quantified self”, a phenomenon where individuals start tracking their behavioural, physiological, biological, and other kinds of health markers’. Flore (Reference Flore2023: 39) describes mental health and well-being tools as enacting ‘self-care in a moment of habitual data tracking and monitoring’. Consumers engaging with such tools will understand they are sharing sensitive personal data and have an expectation that data will be kept securely and stored discreetly. They may not understand the breadth of data being collected nor the uses made of that data.

3.2.2 Data Privacy Laws

Consumers of therapy bots and related mental health support tools may have some protections against harms arising from data breaches, unauthorised sharing or other misuse of personal information under data protection legislation. As in other areas, one functional limitation of these measures lies in consent, which may legitimate data collection despite being obtained through arguably manipulative practices.

3.3.1 AI-Specific Harms

In the main, principles of ethical and safe AI seem to have had little impact on the design or deployment of digital mental health tools. Through the lens of ethical, safe and responsible AI principles, the key risks associated with therapy bots and other digital mental health support tools are bias and errors, inaccuracies and inappropriate content and opacity or lack of transparency. Failure to engage with these principles may result in products that are, at best, ineffective for some groups of users and, at worst, unsafe.

3.3.2 Developer Responses to Misaligned or Inappropriate Outputs from Therapy Chatbots

The developers of therapy bots have, to some extent, aimed to embrace the adaptivity and responsiveness of AI, and impose some guardrails around the use of their tools. These safeguards primarily manifest as training for chatbots for expertise in mental health and warnings to consumers about the circumstances in which the tools should not be used.

3.3.3 Legal Responses to Concerns about AI Safety

The most direct response to concerns about harms arising from the use of AI in any kind of human-centred service is through laws specifically targeted at AI or designed to address AI digital safety. The criticism of these kinds of laws is that they may be insufficiently nuanced to the use of AI in particular domains. Thus, there have been some attempts in different jurisdictions to ban or impose restrictions on therapy bots and related direct-to-consumer digital mental health support tools, as well as on AI companions, also considered later. There is litigation against the developers of general-purpose generative AI chatbots, alleging negligence in the chatbots’ responses to manifestations of poor mental health. For these kinds of legal interventions to be effective, they must be guided by clear objectives informed by an understanding of the relevant risks and the ways in which the tools are used, to which the discussion now turns.

3.4.1 The Limits of Consumer Choice

It might be argued that provided there is accurate information about the various digital mental health tools and alternative offerings (as discussed in Section 3.3.3), then consumers should be left to choose for themselves whether to use the tools and what tools they will use. This approach might be suggested to have the merit of giving consumers agency in accessing the mental health support they prefer and that consumers’ assessment of what is useful to them should not be second guessed by regulators or lawmakers. Certainly, in principle, respect for individual autonomy means allowing parties to select the services they desire and empowering them to enter into the contracts they choose on the terms they assess as suiting them best. This approach allows market mechanisms to reward the developers of useful products and permits parties to allocate risk in the manner they judge most efficient, with corresponding adjustments in price.

In reality, consumers of the services of therapy bots and other direct-to-consumer digital mental health support tools will usually be at a relative disadvantage in selecting a service suitable for their needs and preferences. Consumers are likely to be unfamiliar with the workings of the AI technologies being deployed and the contractual terms may differ drastically from the arrangements in place when dealing with a human mental health professional. The services of such tools are provided without the intervention or involvement of a professional who might otherwise guide consumers on how and when they should be used. Moreover, chatbot services are presumably being used by cohorts of consumers who are highly likely to be experiencing vulnerability, people concerned about their mental health, who may, during their use of the tool, face a mental health crisis.

Here, it might be suggested the answer is for consumers to seek reliable information about the tools suitable for their needs. However, finding this information is likely to prove difficult even for the narrower category of specifically designed therapy bots, as opposed to AI companions or general-purpose chatbots co-opted for this use. This difficulty arises because different and often inconsistent measures are used to promote and review them.

In their promotional materials, developers of therapy bots and related digital mental health support tools typically seek to establish the efficacy of their products in a number of ways. Once interrogated, these measures are less robust than they might at first appear. Importantly, these measures also focus on discrete features of the tools, making an overall assessment of the benefits and risks difficult.

3.4.2 The Role of Consumer Protection Law

Consumer protection law commonly operates through a combination of what is sometimes referred to as ‘ex-ante’ and ‘ex-post’ requirements. Ex-ante consumer protection laws impose safety standards and expectations that may apply before products enter the market and operate while they are in the market, with the aim of reducing the risk of harm. Ex-ante consumer protection in this context would lie in therapeutic goods regulation and online safety laws, discussed in Sections 3.1.2 and 3.3.3.

Ex-post regulation imposes liability for penalties and compensation where harm does occur. Ex-post regulation responds to instances of harm but should also provide incentives to businesses to exercise greater care in providing their services. Ex-post protections of this kind are found in common law actions for negligence and breach of contract. They are also found in legislation governing the supply of goods and services to consumers and enforced by regulatory agencies, such as Ofcom in the UK, the Australian Competition and Consumer Commission in Australia or the Federal Trade Commission (‘FTC’) in the US. Regulators are responsible for enforcing prescriptive conduct obligations, such as prohibitions on misleading, unfair or unconscionable conduct found in the US Federal Trade Commission Act (15 U.S.C. §§ 45(a)(1), 45(n)), the EU Unfair Commercial Practices Directive (Directive 2005/29/EC, art. 6, 2005 O. J. (L 149) 22, 28), UK Consumer Protection from Unfair Trading Regulations 2008 regs 5–6 or the Australian Consumer Law ss 18 and 21. These core ‘safety-net’ protections are typically expressed as open-textured standards, which makes them capable of adapting to new manifestations of the prohibited conduct, including those raised by emerging technologies (Paterson & Maker, Reference Paterson, Maker, Lim and Morgan2024). They certainly have an application in the domain of therapy bots and other direct-to-consumer digital mental health support tools, as well as to AI companions and general-purpose generative AI chatbots.

4.1.1 Regulatory Arbitrage

Regulatory arbitrage refers to the efforts of regulated entities to structure their products so that they fall, in form if not in substance, outside a regulated regime. As discussed in Section 3.1.2, to the extent that therapy bots and related direct-to-consumer digital mental health support tools are classed as lifestyle products or as merely applying established psychological therapies, they fall outside the scope of therapeutic goods regulation. Developers of these products appear to be using contractual terms and disclaimers to ensure their products avoid classification as therapeutic devices, which would subject them to additional scrutiny. As illustrated in Table 3, some developers of therapy bots are promoting their products using language that comes close to offering treatment for the symptoms of poor mental health, suggesting a therapeutic application. In most of these cases, the fine print terms of service of these chatbots expressly disclaim any therapeutic effect. The approach might be seen as mere caution or even respect for the complexity of mental health issues. However, the effect of these disclaimers is to take the chatbots outside the therapeutic goods regime and the robust oversight of outcomes that comes with the characterisation of a tool providing diagnosis or treatment.

There are several viable and available legal responses to the uncomfortable juxtaposition of these kinds of headline representations and contractual disclaimers. One response would focus on the headline claims used to promote a therapy bot and related tools in determining whether it falls within the scope of therapeutic goods regimes. On this approach, if the tool is promoted as having therapeutic effects, then it warrants regulation under that regime regardless of any disclaimer in the contractual terms. If the tool is genuinely not supportive of mental health in a therapeutic sense, it should not be described in terms that would be understood as suggesting otherwise.

Another response would be to focus on the inconsistency between the headline claims, to support mental health, and the fine print disclaimers, which deny this effect. A headline claim that is not supported by the substance of the product provided (as would be suggested by the extensive disclaimers) is misleading and should be sanctioned as such (via a regulatory strategy that has been used to respond to misleading privacy claims, discussed in Section 3.4.2).

Still another approach would focus on the care and skill required under tort and equivalent statutory regimes from the developers of direct-to-consumer mental health support tools and, similarly, companion AI or generative AI chatbots. If there are significant limitations to the efficacy of the product or it is critical that products not be used for crisis management or for diagnostic purposes, then this information is too important to be left to obscure fine print warnings, limitations or exclusion clauses. The information should be drawn to the attention of consumers in ways that are likely to be salient for their decision-making. Failing to do so should be treated as a failure of reasonable care, as it would likely influence consumers to make decisions they would not otherwise have made.

The broader point is that the effective regulation of therapy bots and other direct-to-consumer digital mental health tools, should not depend solely on whether the product is classed as inside or outside the therapeutic goods regime. Rather, the relevant regulatory regime should be understood as encompassing the entire body of laws that combine to promote good outcomes for consumers. This approach demands regulatory co-ordination and reduces the risk of ineffective or ‘performative’ regulation.

4.1.2 Regulatory Inattention

The strength of therapeutic goods regimes lies in providing ‘entry-level’ scrutiny of medical devices (Carpenter, Reference Carpenter, Balleisen and Moss2010: 164–165), which requires the operation of the product to have been validated and subject to ongoing monitoring. This kind of robust entry-level regulation aims to weed out harmful or poorly performing products and support health consumer and practitioner choice (Carpenter, Reference Carpenter, Balleisen and Moss2010: 189; Shuren et al., Reference Shuren, Patel and Gottlieb2018: 337). However, consumers using therapy bots outside the therapeutic goods regime still have protections under other laws, such as AI-targeted laws, data protection laws, consumer protection laws and the general laws of tort and contract.

These other kinds of laws have been used to support the objectives of the therapeutic goods regimes, for example, by ensuring that developers of products have a proper basis for any claims made about the products’ benefits. For example, the FTC has relied on its jurisdiction over misleading conduct to regulate marketing claims about health-related products:

A similar approach could be taken to therapy bots that claim to be beneficial for mental health without verifying those claims by reference to research that withstands scientific scrutiny or disclaims effectiveness in the contractual terms of service. Currently, the consumer protection regulators in the UK (Ofcom), US (FTC) and Australia (ACCC) have yet to consider the consumer protection aspects of therapy bots and other direct-to-consumer digital mental health tools. This is the case even though the marketing for such products, and the contractual terms, uses the kinds of practices that have been subject to criticism and sanction in more traditional consumer markets.

It may be that the lack of regulatory attention given to therapy bots and other direct-to-consumer digital mental health tools arises from the combination of the optimistic promotion of the services offered by the chatbots and belief in the capacity of AI to supplement human services. In a climate where there are concerns about mental health and not enough support for people experiencing poor mental health or mental illness, optimism about the potential of therapy bots and other direct-to-consumer AI mental health tools has proved persuasive. In this light, regulators may simply have not considered that the products present sufficient risk to warrant closer examination.

Moreover, as noted in Section 3.1.2, the headline marketing for the services provided by therapy bots, although invoking mental health, avoids a direct reference to mental illness, which would squarely fall within therapeutic regimes. The products therefore sit at the boundaries of different regulatory schemes and this may account for them not squarely having been the focus of any one regulator. Lisa Parker et al. (Reference Parker, Bero and Gillies2019b: 171) argue that ‘[d]ifferent aspects of a single [mental health support tool] may come under the jurisdiction of multiple upstream regulators, with none of them appearing to take overall responsibility for the product’ (Parker et al., Reference Parker, Bero and Gillies2019b). The lack of any sustained regulatory scrutiny of direct-to-consumer therapy bots may also reflect the impact of discipline, not just regulatory, silos.

Studies on therapy bots and other direct-to-consumer digital mental health support tools have tended to focus on discrete issues, such as the effectiveness of the therapy offered or the level of privacy protection provided, as opposed to a more holistic assessment. This absence of formal regulatory attention means other forces, such as those arising from funding and market opportunities, will influence the way in which the products are developed and deployed. As Christine Parker et al. point out:

To date, regulators, and health industry bodies, have missed opportunities to design effective regulatory responses to the risks of direct-to-consumer digital mental health chatbots that would operate as market gatekeepers, reward best practices and sanction harmful ones. In the resulting vacuum, developers have sought to control the narrative, positioning their products as innocuous and essential, in the sense of filling an unmet demand for individualised mental health support.

The combined effect of how the chatbots are promoted and their aspiration to ‘helpfulness’ is regulatory inattention, a pattern observable in other fields as well. Chatbots are being deployed in government, law, finance and retail services (Paterson, Reference Paterson2025). In these areas, too, the dominant narrative is on assisting users and promoting efficiency. The many concerns arising from these uses, such as error, bias and data protection, are addressed individually if at all, rather than by reference to their cumulative impact. There are failings here of regulatory co-ordination, which also lead to reactive responses when things do go wrong.

4.1.3 Reactive Regulation

Growing concerns about the risks to consumers in relying on AI companions or general-purpose generative AI for mental health support have prompted some regulatory intervention. As discussed in Section 3.3.3, there are moves in some jurisdictions to provide greater protection for children through age-based limits on access and reminders to take a break. Additionally, California and New York have prohibited companion chatbots from operating without a protocol for identifying indications of self-harm or suicidal ideation (California, 2025; New York, 2025).

These might prove useful interventions. However, the initiatives have not precipitated a broader discussion about the need for more readily available mental health support nor how it should be provided. It is possible, therefore, that these kinds of response to the mental health harms of companion AI and general-purpose generative AI chatbots may be more reactive than effective. Put starkly, requiring chatbot developers to identify signs of ill health and refer those consumers to mental health support services may be technically unrealistic and practically unhelpful. Referrals to support lines provide little relief to those with poor mental health if the support services are underfunded and oversubscribed (Stokel-Walker, Reference Stokel-Walker2025). And there is no evidence that a referral or reminder to take a break provides an effective prompt to a consumer to change their habits when they are enamoured with their AI companion or feel they have no other forms of social support.

4.2.1 Innovation and the ‘Cost’ of Regulation

A common objection to new laws applying to AI, or new ways of applying existing laws, is that such initiatives will stifle innovation (Productivity Commission, 2025). The concern is that more focused and expansive regulation will impede the development of useful applications of the technology, such as genuinely beneficial therapy bots and other digital mental health support tools. The characterisation of innovation and regulation as being mutually exclusive has been criticised (Paterson, Reference Paterson2026) and has certainly not been established. It is entirely plausible that using the suite of effectively targeted laws to promote best practice will lead to more innovative outcomes benefiting consumers. Equally, uneven enforcement of existing law may favour non-conforming or poorer quality services: the proverbial race to the bottom.

An illustration of this paradigm is found in the withdrawal of Woebot from the market in July 2025. Woebot was a mental health support chatbot using cognitive behavioural therapy (see Table 1). The developers of Woebot stated that the reasons for its withdrawal from the market lay in the challenge of meeting the FDA’s requirements for using large language models to provide mental health support and a lack of clarity about those requirements (Stat News, 2025). It may be that the service provided by Woebot required too much oversight or was too rigid to be viable as a stand-alone commercial product. However, if we take the developers’ statement at face value, the outcome may be seen as a perverse consequence of an overly reactive approach to the use of chatbots for mental health support. Woebot was regarded by some commentators as one of the better products available to consumers (Wallace, Reference Wallace2025b). Woebot used a rules-based approach in its chatbot to constrain the risk of inappropriate responses (Darcy, Reference Darcy2023). Woebot was rated as being privacy-preserving by Mozilla (2023) and providing a good user experience by One Mind PsyberGuide (n.d.). Woebot was built and operated by psychologists undertaking clinical research on the performance of the product (Darcy et al., Reference 71Darcy, Daniels and Salinger2021).

The current approach to regulating direct-to-consumer digital mental health tools arguably favours those products that are designed to navigate around the boundaries of therapeutic goods regulation. It may also leave the ground open for generative AI offerings that do not purport to support mental health but may nonetheless be used in this way despite having no expertise and a propensity to inept responses (Wallace, Reference Wallace2025a). By contrast, a more holistic understanding of the way in which existing laws, and other market interventions, respond to the harmful manifestations of AI therapy bots – distributed, networked regulation – might support a level playing field for the development of more robustly validated, tested and refined approaches to digital mental health support.

4.2.2 Regulatory Networks

While the nuances of relevant laws may vary between jurisdictions, the underlying demand for the regulation of products being offered to consumers is relatively straightforward. State-of-the-art technology should not divert attention from the misuse of practices that are known to erode consumer welfare in other contexts. Lifestyle products should deliver safe and welfare-enhancing outcomes consistent with their headline representations. Health-related products should be safe for their promoted and foreseeable uses. Data should not be extracted and used against consumers or in ways that are contrary to their reasonable expectations. Contractual rights and limitations should be balanced, consistent with headline marketing and salient to consumer users.

The challenge lies in co-ordinating the various regulators that may be involved in ensuring these outcomes, given that the products that may make claims to mental health support are situated across several fields of regulation and invoke the expertise of mental health professionals and the lived experience of those who use the technology. In this context, there is no one law or lead regulator who can resolve concerns about the risks associated with the support services being offered. Instead, there is a network of market participants, who have the capacity to influence the ways in which the services are delivered and the quality of what is provided.

This picture of multiple influences and modes of intervention might be seen as haphazard or dysfunctional. However, it reflects the reality of regulation in the kind of dynamic market presented by direct-to-consumer digital mental health support tools. Part of the complexity arises from the way in which the products themselves are presented and marketed. Additionally, regulating complex products in heterogeneous markets is inevitably multifaceted; this is all the more so when the purpose of the product is human well-being.

Christine Parker et al. argue that analyses and suggestions for regulatory interventions often assume that regulation is a unitary tool affecting relatively stable markets. Instead, they point to the insights available from a regulatory network analysis (Parker et al., Reference Parker, Carey, De Costa and Scrinis2017). Regulatory network analysis suggests that ‘we should consider markets … as a set of changing relations in which public and private standards are continually being created, adjusted, solidified, or destroyed’ (Parker et al., Reference Parker, Carey, De Costa and Scrinis2017: 370; also Braithwaite, Reference Braithwaite2006: 885). This insight allows recognition that changes to industry behaviour may most effectively be produced by acknowledging and co-ordinating the network of laws, regulators and other participants, that have the capacity to influence behaviour in the market in question. It also provides an opportunity for regulators, professional organisations and civil society groups with shared objectives to collaborate in shaping and promoting the kinds of market outcomes that genuinely benefit society. Applied to therapy bots and other direct-to-consumer digital mental health support tools, a regulatory network analysis means acknowledging the role of developers, tech companies, medical associations and consumer groups in shaping the affordances of the products being offered.

Network regulatory analysis means recognising that the law is not the only or even the most significant influence on outcomes in the regulated space. Rather ‘the state becomes part of a network of regulation in which the tasks of regulation are redistributed in various ways among actors within the network’ (Drahos & Krygier, Reference Drahos, Krygier and Drahos2016: 3). Within the apparatus of the state, however, there is a network of regulation in which tasks are distributed between laws and regulators. In other words, there is a sense in which the legal regime itself operates as a regulatory network and the success of regulatory objectives is reliant on a network of cognate interrelated laws (Figure 3).

Figure 3

Legal networks

A regulatory network analysis recognises that a product’s features may enliven numerous laws and that more than one regime may offer a valid response to concerns about the efficacy or safety of that product. The requirements for testing and oversight provided by therapeutic goods regulations set a basis for safer, more effective products but need to be supplemented by consumer protection oversight, ensuring that contracting practices are not misleading or otherwise overreaching. To the extent that some products fall outside therapeutic goods regulation, consumer protection laws need to be rigorously enforced. Regulators should be particularly vigilant in the case of products using new forms of AI, especially generative AI, given the associated risks of bias and inaccuracy. The approach requires co-ordination and also a degree of dynamism as the risks and challenges evolve with different iterations of the available products as frequently as every six months (Torous & Rodriguez-Villa, Reference Torous and Rodriguez-Villa2021).

Stakeholders other than regulators have a role in promoting a greater level of safety and reliability in therapy bots and other direct-to-consumer digital mental health support tools. The work of regulators might be complemented by cross-industry standards or guidelines which reduce the work that needs to be done by individuals in assessing standardised features of mental health chatbots, such as privacy and security (Australian Commission on Safety and Quality in Health Care, 2020). Work has already been done on developing guidelines and standards for digital mental health support tools (World Economic Forum and Deloitte White Paper, 2022: 12–15; Neary and Schueller, Reference Neary and Schueller2018: 533–534). Australia has implemented standards and a voluntary accreditation scheme for digital mental health products to address key safety and quality concerns (Australian Commission on Safety and Quality in Health Care, 2020; see also, Steindl, Reference Steindl2023).Footnote ⁶ The Mental Health Commission of Canada (2023) has developed a National Assessment Framework for e-Mental Health support tools based on standards covering data and privacy; clinical evidence; clinical safety; usability and accessibility; security and technical stability; cultural safety, social responsibility and equity; and enhanced data sovereignty. The World Economic Forum with Deloitte (World Economic Forum, 2021) has also developed a framework for governance of digital mental health support tools.

Critically, there needs to be consensus on the content, design and implementation of such guidelines and standards, as well as an authoritative means of verifying compliance (Fraser et al., Reference Fraser, Parker and Haines2024). As Henry Fraser et al. explain, ‘standards tend to work best, and carry the greatest imprimatur of legitimacy, when there is consensus about goals and at least some degree of agreement and clarity about practical implementation’ (Fraser et al., Reference Fraser, Parker and Haines2024: 112). A stronger mechanism for achieving consensus could involve external expert certification (cf. Fraser et al., Reference Fraser, Parker and Haines2024: 116) and a system for signalling rankings to mental health professionals and consumers, possibly through a star system or trust mark (Neary & Schueller, Reference Neary and Schueller2018: 534–535). Neary and Schueller (Reference Neary and Schueller2018: 534) also argue that ‘[m]ulti-dimensional ratings are useful for consumers and clinicians because some characteristics might be more important for different people or purposes’. Additionally, there should be stakeholder education about the role and scope of the guidelines or standards.

Another site of regulation lies with mental health professionals and their professional associations. Professional associations have a role in guiding developers and educating members about their operations. For example, the American Psychiatric Association has developed a five-stage app rating model for clinicians based on a series of questions to investigate across five fields: access and background; privacy and safety; clinical foundation; usability; and therapeutic goal (American Psychiatric Association, n.d.). This kind of approach is useful in its focus on the needs of patients and treating psychiatrists. Mental health services in Australia have provided guidance to consumers on choosing a digital mental health product, and the limitations of more general generative AI chatbots (Beyond Blue; Black Dog Institute).

The capacity of intermediaries in the market for therapy bots and other direct-to-consumer digital mental health support tools to hold developers to appropriate standards of performance should be recognised and utilised. App stores should be responsible for removing products that misrepresent their scope, or which make claims that are entirely unverified (Mattioli, Reference Mattioli2021: 34). For developers of therapy bots that want to comply with best practice, there are guidelines, standards and assurance frameworks developed in conjunction with industry, professionals and users that can assist. Verified compliance with these standards is a way of signalling trustworthiness by developers of digital mental health support tools and a way for health professionals to be secure in recommending the products to clients. Compliance with standards can also be a necessary (but not conclusive) indication that a service provider has exercised reasonable care in improving the service as required under tort and some statute law.

4.2.3 Distributed Regulation

While existing systems of regulation can be understood as working most effectively as networks of influence, the debate around regulating products or services involving AI often focuses on the need for ‘a’ law responding to the harms of the technology. Given the rapidly changing capacity of the technology and the multiple touchpoints for harm, laws might better be conceived as distributed across multiple nodes. This means recognising more than one site of response for different kinds of harm or risk. Without centralising responsibility in any one agency, regulatory responsibility is distributed across participants in the network, promoting those with the best capacity to promote any particular regulatory objective in their relevant sphere of operation that is relevant in the context at any point in time.

The need for different legal nodes responding to AI harms need not represent a failure in the system. Rather, it is a sign of adaptivity: it means the legal regime as a whole contains multiple sites for refining and reframing the characterisation of rights and responsibilities for addressing new technologies or other developments in the market (Figure 4).

Figure 4

Distributed (legal) regulation

For example, as discussed earlier, the risks to consumers from data processing may be addressed by a combination of data protection law requirements for notice, consent and conduct-style duties applying to data processing, consumer law prohibitions on misleading conduct, privacy torts, breach of confidence or even the development of fiduciary-like duties for those who hold sensitive information (on the latter, see Dove et al., Reference Dove, Taylor and Richardson2026). The FTC has signalled a preparedness to use its regulatory powers to address concerns about bias and unfairness in consumer AI applications (Jillson, Reference Jillson2021; Australian Competition and Consumer Commission, 2019; Innovation, Science and Economic Development Canada, n.d.; Competition and Markets Authority, 2020). Contractual limitations may be tackled as unfair or misleading terms, or circumvented as irrelevant for the purposes of therapeutic goods characterisation of products for which the headline advertising promotes a mental health response (Medicines & Healthcare Products Regulatory Agency, 2025).

More effective regulatory responses to the harms that may be occasioned by therapy bots and other direct-to-consumer digital mental health tools do not necessarily ensure the ongoing viability or efficacy of these products. Consumers may continue to prefer AI companions or general-purpose generative AI chatbots over specialised therapy bots. Rule-based therapy bots may be more secure but linguistically limited and more conversationally adept generative AI therapy bots may still not be engaging as an AI companion. Health professionals may develop new kinds of products or use instead AI to improve the efficiency of administrative processes, giving them more time for clients. However, greater recognition of the role of distributed regulatory networks may address the more overtly harmful aspects of direct-to-consumer digital mental health tools and also improve their overall performance in the eyes of consumers.

Regulatory initiatives focused on specifically designed digital mental health tools still leave the risks of harms from companion AI or general-purpose generative AI chatbots to be resolved. But similar lessons can apply to these offerings as well. Moreover, a more effective regulatory stance on therapy bots may create space for a more deliberate and informed discussion about the place for these other kinds of AI chatbots, and their influence on the mental health of the humans who interact with them.