Generative artificial intelligence (GenAI) shows promise for mental healthcare by increasing access to treatment. In this article, we analyse recent evidence on the use of GenAI chatbots as a treatment for common mental disorders. We examine key ethical and methodological considerations and discuss the specific risks for delusions. Adopting a precision psychiatry perspective, we propose that the therapeutic alliance can be improved by tailoring GenAI to mimic a user’s psychological traits, a version of socioaffective alignment.