Industry leading development

In the UK, commercial organisations have primarily taken a lead on the development of digital mental health innovations and have produced most of the e-therapies and apps that are available. Examples that are available in the UK include the Big White Wall, IESO Digital Health, ^{Reference Kessler, Lewis, Kaur, Wiles, King and Weich14} Kooth and Mood District. Commercial organisations, with expertise in online or app-based technology, may have an advantage over healthcare and academic organisations in terms of their skills and experience in both producing and commercialising usable and engaging products. However, they may lack knowledge on the care pathway or the clinical, patient and healthcare system benefits that are required to develop cost- and clinically-effective products. Furthermore, industry may not be best placed to involve service users and healthcare professionals in the co-design of digital products. These considerations highlight the need for effective joint working between healthcare providers and industrial partners.

Some local NHS organisations have commissioned commercially developed e-therapies (such as Big White Wall and Kooth); however, these technologies have not yet been widely adopted across the NHS. A possible reason for this is that these products are reaching the market prior to any formal evaluation of their efficacy and without high-quality peer-reviewed research evidence. An exception to this is ‘Beating the Blues’ which is NICE approved, ¹⁵ although this has recently been found to offer no additional benefit beyond usual general practitioner care. ^{Reference Gilbody, Littlewood, Hewitt, Brierley, Tharmanathan and Araya16} As many e-therapies are based on CBT, there can be a tendency for developers to claim that they are evidence based as CBT itself may have an evidence base with a particular patient population. Yet clearly the evidence-informed novel e-therapy is not the same as the non-e-therapy version, and may differ in relation to service user engagement, acceptability, therapeutic alliance, processes of change, and ultimately, treatment outcomes. ^{Reference Jones17} Although some industrial providers have established academic partnerships to evaluate their products once they have been disseminated, evaluating online treatments retrospectively sits at odds with the principle of evidence-based practice and raises difficulties if the research does not produce favourable results. We recommend involving academic partners from the start to ensure that adequate and appropriate evaluation is built into the development plan.

In-house development within the NHS

Those working within the NHS may have the option of using in-house expertise from their trust's internal website development team. In-house options may seem appealing, as they are likely to appear more affordable than commissioning external organisations. However, this should be balanced with the extensive time required from internal personnel, the need for specific skills and resources and the fit with the organisation's broader strategic vision. Examples of NHS-built innovations that have now been rolled out into clinical services include a moderated social network for service users, first developed for adults with eating disorders (the Support Hope and Recovery Online Network, SHaRON) created in Berkshire Healthcare Foundation Trust, and an SMS texting helpline for school nurses to support young people (ChatHealth) that was set by up Leicestershire Partnership NHS Trust.

As for industry-led developments, rigorous evaluation has not always been central to the development of e-therapies within the NHS, presumably reflecting a lack of time or available resources to undertake research beyond routine service audit. Notably in the most recent reviews of digital interventions for anxiety and depression ^{Reference Olthuis, Watt, Bailey, Hayden and Stewart13,Reference Pennant, Loucas, Whittington, Creswell, Fonagy and Fuggle18} there were no examples of clinical or health-economic outcomes associated with e-therapies that have been developed within the NHS. Some NHS-led digital innovations for bipolar disorder and psychosis have promising research behind them (for example Simon et al, ^{Reference Simon, Budge, Foster, Bell, Christensen and Price19} Palmier-Claus et al ^{Reference Palmier-Claus, Ainsworth, Machin, Barrowclough, Dunn and Barkus20} ) but have yet to be adopted at scale across the NHS. ^{Reference Davies21}

Individual NHS trusts that develop e-therapies will also need to consider whether and how to disseminate innovations beyond their trust. Disseminating the innovation to other trusts (and healthcare systems) maximises its reach and may provide an income stream for the trust that led the development. Although there may be initial resistance to commission an innovation from other NHS trusts because of a desire to create their own innovations, the growing recognition of the specialist skills and time required to develop and evaluate e-therapies is likely to promote between-trust commissioning going forwards. Effective communication regarding what is already available will be critical to prevent NHS trusts reinventing the ‘digital wheel’, and to enhance learning from both successful and unsuccessful case studies in the NHS.

In-house development within universities

The work of Gerhard Andersson and colleagues in Sweden is arguably the most prolific example of e-therapy development within a university setting. This group have established their own in-house web developer to create psychological interventions ^{Reference Månsson, Ruiz, Gervind, Dahlin and Andersson22} that has enabled the rapid and cost-effective development and evaluation of a range of online interventions with demonstrated efficacy compared with a waitlist control (see for example Meyer et al ^{Reference Meyer, Berger, Cespar, Beevers, Andersson and Weiss23} ) and equivalence to face-to-face treatment (see for example Nordgren et al, ^{Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki and Eriksson24} Andersson et al ^{Reference Andersson, Hesser, Veilord, Svedling, Andersson and Sleman25} ).

An alternative approach that university-based researchers have commonly taken is to use external funding to commission a private company to build, host and maintain a website and/or app. Most of the online treatment programmes for child and adolescent anxiety have taken this approach (for example Vigerland et al ^{Reference Vigerland, Ljótsson, Thulin, Öst, Andersson and Serlachius26} Spence et al ^{Reference Spence, Holmes, March and Lipp27} Morgan et al ^{Reference Morgan, Rapee and Bayer28} ). A particular consideration here is that research funding is often fairly slow moving, with lags between bids and awards often being over 12 months. Furthermore, if a tendering procedure is required, this again can be slow, with a UK tender typically taking 6 months and a European Union one up to 9 months. In contrast, digital innovation moves at a fast pace, and what may have been ‘innovative’ at the time the idea was conceived, may become technologically redundant by the time the project starts and technical specifications may need to change. Academics will also need to keep in mind that research funding may cover development, maintenance and hosting of the website/app for a specified period (typically the grant duration) but it is unlikely to cover the costs of maintaining and hosting the online tool beyond this period. Equally, initial funds are unlikely to cover the cost of changes that may be needed to the content or technical specifications, in response to research and development and changes in devices and operating systems. All of these funding limitations create the risk that the e-therapy may become out of date without additional sources of funding being secured. As such, it is important that developers consider scalability and sustainability from the outset of any digital mental health project, including consideration of how the e-therapy will be disseminated within NHS trusts. One approach is to work in partnership with an external company who supply digital mental healthcare solutions and who could take the product to market. This could ultimately result in a revenue stream to maintain and upgrade the system.

Working collaboratively with industry

An alternative approach to digital mental health innovation that has the potential to overcome many of the challenges highlighted above is for clinicians and researchers to work in a collaborative partnership with a website/app development company. Here, clinical and academic partners can contribute clinical insight and research skills while the industrial partner brings technical knowledge and experience of working in the digital health space, and of getting products adopted by the NHS. An example is an online CBT platform that is used routinely across several Improving Access to Psychological Therapies services that was developed collaboratively by Berkshire NHS Foundation Trust, who provided the treatment content, and Silvercloud Health, who developed the technology and market the product to other healthcare providers. Academic partnerships have facilitated research into the efficacy and acceptability of the platform. ^{Reference Richards, Timulak, Doherty, Sharry, Colla and Joyce29,Reference Richards, Timulak, Doherty, Sharry, McLoughlin and Rashleigh30}

Collaborations between clinical, academic and industrial partners facilitate the development of innovations that are based on sound psychological theory and clinical practice that has a demonstrable evidence base and that has research and evaluation firmly built into the development phase (i.e. prior to commercialisation). Through a collaborative arrangement all parties are likely to be invested in making a product that is sustainable and scalable. Another advantage of the collaborative approach for industrial partners is that it facilitates access to the views of potential beneficiaries of the product (such as service users) to inform development and to contribute to design and usability testing. Incorporating the views of potential beneficiaries throughout the development process is likely ultimately to lead to the development of a product that is engaging, credible, relevant and valued within services.

Pitfalls of the collaborative approach include potential changes in the direction of the company's business plan that may then deviate from shared goals, or that the company's financial situation may not allow them to continue to support the collaboration, for example if they fold or are taken over. Another potential issue is that the aims of the groups may differ, with the commercial partner likely motivated to take the product in a direction that results in greater financial rewards, whereas the clinical/academic partners may be more concerned with channelling resources into adapting the innovation to maximise effectiveness. Appropriate steps need to be taken when contractual terms are agreed to mitigate these risks.

Development considerations

Across all development options, co-design between clinicians, researchers, industry and service users is crucial. Applying principles from the human factors approach to co-design would help ensure the tool is relevant, usable and engages users. ^{Reference Kushniruk31} Careful consideration should be given to specific populations such as older adults whose ability to work with digital technology is variable, or forensic patients in secure settings who may have restricted access to information technology. There is also an issue about how to best manage and appropriately analyse the enormity of data (‘big data’) generated using digital tools, alongside connectivity complexities, such as ensuring digital tools can integrate into healthcare systems and retain internet connection in different health settings.

Commercialisation

Whichever route is followed to develop and evaluate a novel e-therapy, specific attention should be given to how the product can be disseminated in a way that will generate funds to cover ongoing maintenance, research and development. In the UK, adoption by the NHS is typically the Holy Grail for digital mental health innovation. A clear business model is crucial to achieve this and will need to include consideration of the potential market, what the problem is that needs to be solved in any given care pathway, how the technology or product provides a solution to this problem, the quantifiable health benefits for patients, clinicians, commissioners and the health system itself, how the implementation of this product will take place and the anticipated revenue generation.

E-therapy developers will need to carefully consider the stage in development at which dissemination will occur. The time required to achieve the ‘gold standard’ of evidence from a randomised controlled trial (RCT) contrasts with the fast pace of technological advancements and demand for online access to psychological treatments. Furthermore, for industry, there is limited commercial appeal in waiting years before seeing a return of an investment (particularly if it risks the product going out of date or being pre-empted by a competitor). However, this should not be seen as justification for widespread NHS uptake of products that do not have an evidence base. Instead, developers and funders need to carefully consider ways to evaluate new products efficiently. For example, it may be more efficient to use online technologies to assess and recruit potential participants to an RCT and to evaluate outcomes than to use a face-to-face approach (although this will not be appropriate in all circumstances). There are also a range of study designs, other than the RCT, ^{Reference West, Duan, Pequegnat, Gaist, Des Jarlais and Holtgrave32} that may prove useful in evaluating novel e-therapies. Healthcare providers can usefully inform decision-making about what constitutes appropriate evaluation, by clearly specifying the level of evidence that should be required to justify commissioning and insisting that this is met.