Service user engagement and empowerment

A critical consideration for integrating self-report data into routine healthcare practices pertains to the willingness and capability of service users to engage in self-reporting, along with achieving acceptable compliance levels. A series of studies, including pooled data sets and meta-analytic approaches, have demonstrated that service users can reliably provide self-reports in the flow of daily life using ESM, irrespective of their diagnosis (Myin-Germeys et al., Reference Myin-Germeys, Oorschot, Collip, Lataster, Delespaul and van Os2009), reaching an average response rate in research studies of 78% (Vachon, Viechtbauer, Rintala, & Myin-Germeys, Reference Vachon, Viechtbauer, Rintala and Myin-Germeys2019) and of 70% in people with a diagnosis of psychosis (Rintala, Wampers, Myin-Germeys, & Viechtbauer, Reference Rintala, Wampers, Myin-Germeys and Viechtbauer2019). Although compliance may decrease when applied clinically, a pilot study still achieved an acceptable 50% compliance rate in routine mental health care (Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023). Moreover, findings from an app-based study focusing on monitoring early signs to enhance well-being, engagement, and recovery in psychosis, revealed that 91% of service users randomized to the app-based monitoring actively engaged with the app, with observed improvements in mental health outcomes and treatment adherence (Gumley et al., Reference Gumley, Bradstreet, Ainsworth, Allan, Alvarez-Jimenez, Beattie and Gleeson2020). These data substantiate the potential of ESM for actively engaging individuals in their care.

Furthermore, ESM has been proposed to enhance individuals' insight, understanding, and empowerment. As our ability to report on our emotional and mental states is dependent on our capacity to understand and recognize our internal experiences, self-monitoring enhances our sensitivity to recognize different mood and cognitive states and what may influence them. Qualitative studies with clinicians (Bos, Snippe, Bruggeman, Wichers, & van der Krieke, Reference Bos, Snippe, Bruggeman, Wichers and van der Krieke2019; Weermeijer, Kiekens, Wampers, Kuppens, & Myin-Germeys, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024), and service users (de Thurah et al., Reference de Thurah, Kiekens, Sips, Teixeira, Kasanova and Myin-Germeys2023) have supported this notion. One quantitative study tested this in a clinical trial in individuals with a diagnosis of depression and demonstrated that engaging in daily self-monitoring increased feelings of empowerment (Simons et al., Reference Simons, Hartmann, Kramer, Menne-Lothmann, Höhn, van Bemmel and Wichers2015).

Self-management and recovery

Numerous studies have demonstrated that longitudinal self-report data collected through ESM are exceptionally adept at elucidating meaningful patterns of associations, helping service users to better self-manage their mental health problems. For example, these momentary data provide more accurate information compared to retrospective reports and clinical interviews, concerning mood instability (Solhan, Trull, Jahng, & Wood, Reference Solhan, Trull, Jahng and Wood2009), relapse (Gumley et al., Reference Gumley, Bradstreet, Ainsworth, Allan, Alvarez-Jimenez, Beattie and Gleeson2020), functioning (Schneider, Reininghaus, van Nierop, Janssens, & Myin-Germeys, Reference Schneider, Reininghaus, van Nierop, Janssens and Myin-Germeys2017), and quality of life (Leendertse et al., Reference Leendertse, Myin-Germeys, Lataster, Simons, Oorschot, Lardinois and Reininghaus2018). Furthermore, the iterative nature of assessment enables the identification of dynamic patterns in the variation and co-variation of mood, cognition, behavior, and context over time (Myin-Germeys et al., Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018). For example, in the field of psychosis, real-life increases in anxiety, and decreases in self-esteem have been linked to increases in the intensity of paranoia (Thewissen et al., Reference Thewissen, Bentall, Oorschot, à Campo, van Lierop, van Os and Myin-Germeys2011). However, a closer examination at the individual level revealed that these group findings may not uniformly apply to each individual (Oorschot, Lataster, Thewissen, Wichers, & Myin-Germeys, Reference Oorschot, Lataster, Thewissen, Wichers and Myin-Germeys2012), underscoring the importance of adopting a person-centered approach that facilitates the individual-level understanding of these associations in each person's context. Similarly, ESM research identified altered stress reactivity (Reininghaus et al., Reference Reininghaus, Kempton, Valmaggia, Craig, Garety, Onyejiaka and Morgan2016), delayed stress recovery (Vaessen et al., Reference Vaessen, Viechtbauer, van der Steen, Gayer-Anderson, Kempton, Valmaggia and Myin-Germeys2019), and poor sleep quality as important targets for treatment (Kasanova, Hajdúk, Thewissen, & Myin-Germeys, Reference Kasanova, Hajdúk, Thewissen and Myin-Germeys2020). These examples underscore how offering personalized feedback on individual patterns of risk and behavior represents a unique tool for personalized psycho-education, insight, and self-management (van Os et al., Reference van Os, Verhagen, Marsman, Peeters, Bak, Marcelis and Delespaul2017). Findings from a randomized clinical trial involving patients with chronic depression have demonstrated that providing feedback on specific patterns of behavior can effectively reduce depressive symptoms (Kramer et al., Reference Kramer, Simons, Hartmann, Menne-Lothmann, Viechtbauer, Peeters and Wichers2014) and induce lasting behavioral changes (Snippe et al., Reference Snippe, Simons, Hartmann, Menne-Lothmann, Kramer, Booij and Wichers2016). In another cluster randomized controlled trial involving service users with a psychotic disorder, app-based self-monitoring significantly reduced relapse rates and facilitated symptomatic recovery when compared to routine monitoring (Gumley et al., Reference Gumley, Bradstreet, Ainsworth, Allan, Alvarez-Jimenez, Beattie and Gleeson2020).

Goal direction in clinical assessment and management of care

The detailed information elucidating the crucial associations among symptoms, key problem areas, and their context, as described above, can also help clinicians tailor treatments towards specific and personalized goals. Nevertheless, there is limited research on its actual application in clinical practice.

In addition, this detailed and fine-grained information may contribute to better management of care, by providing more comprehensive assessments of treatment outcomes, encompassing not only symptom management but also overall well-being and functioning, while facilitating a more nuanced evaluation of possible side effects. Two studies directly comparing ESM measures to retrospective questionnaires in assessing treatment outcomes, found ESM measures to outperform conventional clinical measurements, both among individuals with depression (Moore, Depp, Wetherell, & Lenze, Reference Moore, Depp, Wetherell and Lenze2016), and psychosis (So, Peters, Swendsen, Garety, & Kapur, Reference So, Peters, Swendsen, Garety and Kapur2013). Moreover, ESM data have been shown instrumental in identifying early processes of change that may bear relevance for the further course of treatment. For instance, early improvements in positive affect as measured with ESM during the first week of antidepressant treatment, were found to predict the depression severity and remission after 6 weeks (Geschwind et al., Reference Geschwind, Nicolson, Peeters, van Os, Barge-Schaapveld and Wichers2011). Additionally, ESM enables a more accurate assessment of treatment side effects, for example by demonstrating a dampening effect of antipsychotic medication on psychotic symptoms and mood in individuals with psychosis (Lataster et al., Reference Lataster, Myin-Germeys, Wichers, Delespaul, van Os and Bak2011).

Shared decision-making

A cornerstone of the shared decision-making process is situational diagnosis (Wieringa et al., Reference Wieringa, Rodriguez-Gutierrez, Spencer-Bonilla, de Wit, Ponce, Sanchez-Herrera and Snoek2019), wherein the service user's situation is elucidated, and both the service user and clinician reach a mutual understanding of the treatment focus. In mental health care practice, this requires a comprehensive insight into moments of mental risk and resilience by both parties, aiding in the joint identification of indicators of distress and (mal)adaptive patterns of behavior (Slade, Reference Slade2017). Since ESM can provide relevant and qualitative day-to-day information on key problem areas and pertinent contextual factors essential for treatment decisions and evaluating treatment progress, using ESM would foster agency and true collaborative care driven by the needs of patients (Simblett et al., Reference Simblett, Matcham, Siddi, Bulgari, Barattieri di San Pietro and Hortas López2019). Despite some evidence in chronic somatic patients, showing that self-monitoring increased patients' understanding of the illness (Mamykina et al., Reference Mamykina, Heitkemper, Smaldone, Kukafka, Cole-Lewis, Davidson and Hripcsak2017), scientific evidence for the role of ESM in improving shared decision-making processes is lacking.

Overall, a growing body of evidence demonstrates the value of ESM-based approaches in enhancing person-centered mental health care. However, ESM is hardly integrated into clinical practice. It is crucial to examine the reasons behind this discrepancy. What are the primary barriers and facilitators for the integration of ESM into routine mental health care and how can we harness the full potential of ESM to promote truly person-centered mental health care?

Understanding and improving the motivation and ability of service users to adhere to the ESM intervention

The successful implementation of novel digital innovations hinges on the willingness and capability of service users to adopt the methodology. Previous research on the broader use of health-related mobile applications indicated that service users are open to adopting a new digital tool if they perceive it as personally relevant (Greer et al., Reference Greer, Robotham, Simblett, Curtis, Griffiths and Wykes2019), and seamlessly integrated into their daily routine without imposing stigma (Simblett et al., Reference Simblett, Matcham, Siddi, Bulgari, Barattieri di San Pietro and Hortas López2019; Torous, Nicholas, Larsen, Firth, & Christensen, Reference Torous, Nicholas, Larsen, Firth and Christensen2018). Crucially, service users must feel that the app empowers them to comprehend and manage their illness (Mamykina, Smaldone, & Bakken, Reference Mamykina, Smaldone and Bakken2015), which can be achieved by providing them with meaningful access to their own data (Ancker et al., Reference Ancker, Witteman, Hafeez, Provencher, de Graaf and Wei2015). Two qualitative studies with service users on the ‘hypothetical’ use of ESM in clinical practice, indicated empowerment, shared decision-making, and self-awareness as main advantages, whereas burden, negative impact on symptoms and well-being, and validity were seen as main limitations (Bos et al., Reference Bos, Snippe, Bruggeman, Wichers and van der Krieke2019; de Thurah et al., Reference de Thurah, Kiekens, Sips, Teixeira, Kasanova and Myin-Germeys2023). This was corroborated by one implementation pilot study, showing that improved self-insight and self-awareness are clear strengths, whereas burden and lack of personalization are the key caveats (Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024) Engaging service users more in the development and implementation of ESM-based digital tools is needed to develop a balanced clinical ESM tool that combines the strengths of active engagement and collecting high-frequency data while reducing participant burden and achieving a high level of personalization.

Understanding and improving the motivation and competence of clinicians in routine healthcare settings to integrate ESM into the workflow

The effectiveness of technological advancements in healthcare is intricately linked to the readiness of healthcare professionals to adopt them. Clinicians are more inclined to embrace digital innovations when they are designed as technology-enabled services rather than stand-alone products (Mohr et al., Reference Mohr, Weingardt, Reddy and Schueller2017). This approach shifts the focus from the technology to how it integrates within the broader context of the healthcare ecosystem. Clinicians exhibit favorable attitudes towards using digital technology when they contribute to an improvement in the quality of care, without a significant increase in workload (Kerst, Zielasek, & Gaebel, Reference Kerst, Zielasek and Gaebel2020). Notably, one survey study (Piot et al., Reference Piot, Mestdagh, Riese, Weermeijer, Brouwer, Kuppens and Bos2022), and two qualitative studies involving clinicians (Bos et al., Reference Bos, Snippe, Bruggeman, Wichers and van der Krieke2019; Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024) identified gaining relevant insights and monitoring the clinical process and treatment as assets of clinically implementing ESM. The interactive use together with the service users was recognized as a notable strength, while challenges such as inadequate training, lack of guidelines, and time constraints were identified as significant obstacles. Interestingly, although personalization was identified as an important requirement (Piot et al., Reference Piot, Mestdagh, Riese, Weermeijer, Brouwer, Kuppens and Bos2022), an implementation pilot study revealed that only a minority of clinicians personalized the ESM approach (Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024). This underscores the imperative for employing mixed-method approaches to develop comprehensive strategies together with clinicians for integrating ESM and other digital technologies into the clinical workflow.

Identifying the technical requirements

To make ESM clinically actionable, it is essential to derive clinically meaningful insights from the wealth of real-life data. This is not an easy task as the numerous ESM variables can be correlated with themselves and across time, depending on the individual's characteristics and context. Therefore, the development of personalized user-friendly and practical methods for clinical data interpretation is crucial. This entails the creation of a platform capable of running algorithms to extract relevant information for each individual, along with a dashboard presenting easy-to-understand visualizations. Basic, robust, evidence-based statistics can illuminate fundamental properties of both mental health states and the environment, such as the intensity and variability of symptoms, frequency of social isolation, or the allocation of time towards productive goal-directed activities. While more complex statistical methods like network modeling (Epskamp et al., Reference Epskamp, van Borkulo, van der Veen, Servaas, Isvoranu, Riese and Cramer2018) have been proposed to elucidate intricate patterns, concerns about their validity have been raised by clinicians (Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024). Indeed, a recent report highlighted significant variability in outcomes when the same data were analyzed using network models by different teams (Bastiaansen et al., Reference Bastiaansen, Kunkels, Blaauw, Boker, Ceulemans, Chen and Bringmann2020), emphasizing the risks of prematurely applying new statistical models in clinical practice. On the other hand, ML solutions that make optimal use of data, e.g. by integrating across data modalities or group-level information can reduce such overfitting phenomena (Koppe, Meyer-Lindenberg, & Durstewitz, Reference Koppe, Meyer-Lindenberg and Durstewitz2021). For instance, multimodal ML models for dynamical systems reconstruction that integrate across time series modalities, have already proven to benefit the discovery of mechanisms and improve time series forecasting (Kramer, Bommer, Tombolini, Koppe, & Durstewitz, Reference Kramer, Bommer, Tombolini, Koppe and Durstewitz2022). Since passive sensor data has repeatedly been demonstrated to be predictive of mental health (Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021), the integration of active and passive data with advanced ML models has the potential to enhance the precision and personalization of ESM in clinical practice. However, the integration of passive and active data into personalized ML models for single patient predictions remains to be demonstrated.

To effectively convey ESM data in a comprehensible manner, easy-to-understand visualizations are crucial. While clinicians have identified the use of visualizations as a strength (Bos et al., Reference Bos, Snippe, Bruggeman, Wichers and van der Krieke2019), there is limited research on optimizing visualizations of real-life data. A qualitative study with clinicians identified time series, bar charts, and pie charts as usable visualizations, showcasing fluctuations in relevant psychological outcomes (e.g. positive or negative affect), emotions in different contexts, and the frequency of engaging in certain (social) activities (Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024). However, these are static and aggregate representations. Bringmann, van der Veen, Wichers, Riese, & Stulp (Reference Bringmann, van der Veen, Wichers, Riese and Stulp2021) developed a moving presentation of raw data to capture the dynamics in ESM data, but this approach has not been extensively investigated yet. More research is needed to develop intuitive and optimal ways for visualizing dynamic, real-life ESM data.

Another critical aspect of clinical implementation is interoperability, which involves integrating ESM data with existing electronic patient files or hospital systems and making them accessible for obtaining broad consent by service users and, thereby, harnessing this data at scale. Clinician dashboards should seamlessly integrate into existing IT tools and workflows, be easily interpretable, and align with clinicians' preferred treatment approaches (Burton et al., Reference Burton, Szentagotai Tatar, McKinstry, Matheson, Matu, Moldovan and Wolters2016). Achieving this requires the cooperation of healthcare management and their IT services, identifying them as significant stakeholders. Currently, most mobile apps are custom-made for specific studies or clinical settings (Miralles et al., Reference Miralles, Granell, Díaz-Sanahuja, Van Woensel, Bretón-López, Mira and Casteleyn2020), and not apt for wider scale-up. Unlike in somatic medicine, where numerous countries have developed large-scale initiatives addressing interoperability (Cuggia & Combes, Reference Cuggia and Combes2019), this is mostly lacking in mental health research, and even more so in mHealth approaches. To transition ESM from a research tool to a clinical device ready for scale-up in practice, specifications for interoperability with the international research community, such as Fast Healthcare Interoperability Resources (FHIR), and integration capabilities with hospital systems and electronic patient repositories need to be developed. This will facilitate the seamless transfer of ESM data within clinical and research settings, enhancing its utility in real-world mental health care scenarios and supporting their sustainable reuse under the FAIR guiding principles for scientific stewardship (van Damme, Löbe, Benis, de Keizer, & Cornet, Reference van Damme, Löbe, Benis, de Keizer and Cornet2024).

Identifying the anticipatory governance requirements

Several reports have raised significant concerns regarding data privacy, security, and confidentiality in numerous existing mental health apps (Miralles et al., Reference Miralles, Granell, Díaz-Sanahuja, Van Woensel, Bretón-López, Mira and Casteleyn2020; Zhou, Bao, Watzlaf, & Parmanto, Reference Zhou, Bao, Watzlaf and Parmanto2019). Recent European regulations, including the General Data Protection Regulation (Regulation (EU) 2016/679) and the Medical Device Regulation (Regulation (EU) 2017/745), have imposed stricter requirements on the technology transfer process to address these concerns (Marelli, Lievevrouw, & Van Hoyweghen, Reference Marelli, Lievevrouw and Van Hoyweghen2020). However, the stricter data quality and safety requirements have complicated the translation of ESM-based apps from research to routine mental health care, due to increased regulatory complexity, but also to a misalignment between the existing regulations and the core values embedded in digital health technologies (Lievevrouw, Marelli, & Van Hoyweghen, Reference Lievevrouw, Marelli and Van Hoyweghen2024). The recent implementation of these EU regulatory changes has therefore resulted in a regulatory grey zone, where existing (mental) health apps have to navigate between being considered ‘lifestyle and wellbeing’ apps and heavily regulated ‘medical’ digital applications (Lucivero & Prainsack, Reference Lucivero and Prainsack2015). Developing increased reflexivity and awareness concerning the complex and scattered regulatory and policy landscape within which mobile technology operates is essential for realizing true clinical implementation. However, this task is challenging given that the European governance and regulatory framework remains a fragmented ethical, regulatory, and policy landscape, despite ongoing efforts for standardization (Marelli et al., Reference Marelli, Lievevrouw and Van Hoyweghen2020). Thus, understanding and navigating the regulatory and policy space is crucial for ensuring ESM's ethical and responsible use in mental health care.

Determining the economic value of implementing ESM in routine mental health care

An essential prerequisite for clinical implication is understanding the economic value and the financial requirements for the scale-up of the intervention. Clinicians have identified financial constraints as significant barriers to implementation with reimbursement from healthcare insurers potentially aiding the adoption of novel digital tools (Weermeijer et al., Reference Weermeijer, Wampers, de Thurah, Bonnier, Piot, Kuppens and Kiekens2023, Reference Weermeijer, Kiekens, Wampers, Kuppens and Myin-Germeys2024). However, very few studies have evaluated the cost-effectiveness and the actual financial implications of implementing and scaling up ESM in routine mental care. One study suggested that utilizing ESM combined with weekly feedback on behavioral patterns is cost-effective for service users with major depression, with a willingness-to-pay threshold of €50 000 for one additional Quality Adjusted Life Year (QALY) gained (Simons et al., Reference Simons, Drukker, Evers, van Mastrigt, Höhn, Kramer and Wichers2017), which is at the lower limit of what is currently viewed as acceptable for gaining a QALY in people with severe mental disorder in Europe and the USA (Eichler, Kong, Gerth, Mavros, & Jönsson, Reference Eichler, Kong, Gerth, Mavros and Jönsson2004). However, more research is needed to assess the marginal cost of implementing ESM in clinical practice relative to its marginal benefit, expanding current evidence to include also cost-utility analyses (CUA) (Dernovsek, Prevolnik-Rupel, & Tavcar, Reference Dernovsek, Prevolnik-Rupel, Tavcar, Ritsner and Awad2007). This entails calculating the economic cost of delivering the intervention – representing the value of all resources consumed by the intervention, both within and beyond the health system – in relation to the benefits it produces, measured as QALYs. The incremental cost incurred by the intervention is related to QALYs gained attributable to the intervention to compute the incremental cost-effective ratio per QALY. This value can inform the decisions on reimbursements by healthcare insurers and public funders. Furthermore, a budget impact assessment should be conducted to provide policymakers with more meaningful information to assess the affordability and feasibility of scaling up the intervention.

Conducting implementation studies and developing an evidence-base

Despite the lack of evidence being a crucial barrier for clinicians to implement digital innovations in their clinical practice, there is a lack of rigorous testing of the usability and effectiveness of mobile mental health interventions (Miralles et al., Reference Miralles, Granell, Díaz-Sanahuja, Van Woensel, Bretón-López, Mira and Casteleyn2020). Only 16% of all papers in the review by Miralles et al. (Reference Miralles, Granell, Díaz-Sanahuja, Van Woensel, Bretón-López, Mira and Casteleyn2020) reported findings from an RCT and only 36% of these RCTs (5.7% of total papers) were specifically focused on effect assessment. Thus, more rigorous and extensive testing of these approaches is needed, to provide service users and clinicians with a trustworthy product that they can safely use in clinical practice. There is also very little cross-validation, with only 14% of the apps being used in multiple studies, again pointing to the limited scale-up of existing mobile mental health apps, including the clinical use of ESM. More rigorous effect assessment is therefore needed if we want to transfer ESM from research into practice.

Furthermore, limited research has focused on implementing mobile health solutions in mental health (Proctor et al., Reference Proctor, Landsverk, Aarons, Chambers, Glisson and Mittman2009). While some studies have identified e-literacy, perspectives of service users and clinicians on app usage, user acceptance and usability, and cost-utility as potential barriers to further implementation and scale-up (Simblett et al., Reference Simblett, Matcham, Siddi, Bulgari, Barattieri di San Pietro and Hortas López2019), few studies have delved into strategies for overcoming these barriers.

Utilizing established implementation science frameworks will enable the optimization of implementation strategies and the investigation of implementation processes and outcomes within implementation studies that move beyond previous frameworks of developing and evaluating complex interventions (Skivington et al., Reference Skivington, Matthews, Simpson, Craig, Baird, Blazeby and Moore2021), and take the specific requirements of implementing novel technologies into account. In tailoring the ESM application for clinical practice, an initial assessment of anticipated barriers and facilitators influencing implementation can be conducted using the ‘non-adoption, abandonment, scaleup, spread, and sustainability’(NASSS) implementation science framework (Greenhalgh et al., Reference Greenhalgh, Wherton, Papoutsi, Lynch, Hughes, A'Court and Shaw2017). Specifically designed for the implementation of novel technologies, the NASSS framework addresses seven domains: the condition or illness, the technology, the value proposition, the adopter system, the organizations, the wider (institutional and societal) context, and the interaction and mutual adaptation between all these domains over time. By leveraging the NASS framework, implementation strategies for novel technologies like ESM can be effectively optimized and tailored.

The RE-AIM framework encompasses key aspects relevant to evaluating the implementation of novel interventions such as ESM across individual and various ecological levels (e.g. staff, setting, system) (Holtrop et al., Reference Holtrop, Estabrooks, Gaglio, Harden, Kessler, King and Glasgow2021). Specifically, this framework assesses Reach (extent of service user participation), Effectiveness (interaction between efficacy and implementation in real-world settings), (c) Adoption in routine clinical care settings, (d) Implementation (delivery of the intervention as intended by clinicians), and (e) Maintenance (integration of ESM into routine care). This, in turn, allows for establishing the public health impact as a central aspect for decision-makers and potential adopters of ESM and, hence, may serve as a direct bridge across the research-to-practice gap (Holtrop et al., Reference Holtrop, Estabrooks, Gaglio, Harden, Kessler, King and Glasgow2021).

Alongside the RE-Aim framework, a detailed process evaluation could provide in-depth insight into the implementation and sustainability of ESM in routine clinical care pathways and establish what works, for whom, in what circumstances, in what respects, to what extent, and why, using a realist evaluation framework. The realist evaluation framework allows us to identify configurations of contexts, mechanisms of change, and how these are associated with outcomes of implementation and intervention. It combines the strengths of quantitative and user-based experiences to produce a coherent and plausible explanation. This process evaluation is optimally conducted within existing pathways to care and treatment frameworks, attending to person-, system-, and context-based factors that influence or determine the effective use, implementation, and maintenance of ESM within existing pathways to care and treatment frameworks across different mental health settings.