Introduction
Self-harm behaviours are defined as any act of self-injury or self-poisoning, which causes trauma to body tissue, with or without conscious suicidal intent (Hawton et al., Reference Hawton, Saunders and O’Connor2012). The prevalence of self-harm rates has increased among adolescents in the UK (McManus et al., Reference McManus, Gunnell, Cooper, Bebbington, Howard, Brugha, Jenkins, Hassiotis, Weich and Appleby2019) and is also an important risk factor for suicide (Hawton et al., Reference Hawton, Saunders and O’Connor2012). The onset of self-harm is typically around 13–15 years of age (Ammerman et al., Reference Ammerman, Jacobucci, Kleiman, Uyeji and McCloskey2018; Groschwitz et al., Reference Groschwitz, Plener, Kaess, Schumacher, Stoehr and Boege2015; Plener et al., Reference Plener, Schumacher, Munz and Groschwitz2015). Research suggests that adolescents struggling with self-harm are less likely to seek professional support, with only 18.8% accessing mental health services (Ystgaard et al., Reference Ystgaard, Arensman, Hawton, Madge, van Heeringen, Hewitt, Jan de Wilde, De Leo and Fekete2008). Self-harm is common in many psychiatric disorders, including mood disorders (Clements et al., Reference Clements, Morriss, Peters, Cooper, While and Kapur2014; Hawton et al., Reference Hawton, Saunders and O’Connor2012), post-traumatic stress disorder (Panagioti et al., Reference Panagioti, Gooding, Triantafyllou and Tarrier2015), and borderline personality disorder (BPD; Bohus et al., Reference Bohus, Stoffers-Winterling, Sharp, Krause-Utz, Schmahl and Lieb2021). Dialectical behaviour therapy (DBT) is a commonly used treatment for individuals with BPD (Linehan, Reference Linehan1993; McMain et al., Reference McMain, Links, Gnam, Guimond, Cardish, Korman and Streiner2009; Miller et al., Reference Miller, Rathus and Linehan2007; Rathus and Miller, Reference Rathus and Miller2002, Reference Rathus and Miller2015; Rizvi et al., Reference Rizvi, Hughes and Thomas2016). DBT has multiple modes of intervention delivery, including: weekly group skills training to increase capabilities, weekly individual therapy to increase motivation to implement new skills, between-session telephone coaching to support with skill generalisation, and therapist consultation team meetings to support therapists and adherence to the model (Linehan, Reference Linehan1993). DBT for adolescents utilises these modes of treatment with the additional inclusion of parents/carers in various aspects of the intervention (Miller et al., Reference Miller, Rathus and Linehan2007; Rathus and Miller, 2015). Research shows that DBT for adolescents (DBT-A) effectively reduces self-harm, suicidal ideation, and accident and emergency department (A&E) visits compared with control interventions in adolescent populations (Bahji et al., Reference Bahji, Pierce, Wong, Roberge, Ortega and Patten2021; Kothgassner et al., Reference Kothgassner, Goreis, Robinson, Huscsava, Schmahl and Plener2021). Homework completion and practising skills outside of sessions is thought to be pertinent for the generalisation of DBT skills into their daily lives, and difficulties with this may impede treatment outcomes (Linehan, Reference Linehan1993; Linehan, Reference Linehan2015). Thus, attempts to optimise skills practice and homework tasks outside of session, including the use of technology, may be useful for augmenting outcomes.
Smartphone applications (apps) have been blended into standard psychological interventions (app plus psychological therapy) and as a standalone intervention for addressing self-harm and suicidal behaviours in adults and adolescents (Arshad et al., Reference Arshad, Gauntlett, Husain, Chaudhry and Taylor2020; Melia et al., Reference Melia, Francis, Hickey, Bogue, Duggan, O’Sullivan and Young2020; Witt et al., Reference Witt, Spittal, Carter, Pirkis, Hetrick, Currier and Milner2017). This is particularly important for adolescents who self-harm because the majority of adolescents are active users of technology, thus apps are likely useful means for facilitating skills practice outside of sessions to augment outcomes (Anderson and Rainie, Reference Anderson and Rainie2020; Melvin et al., Reference Melvin, Gresham, Beaton, Coles, Tonge, Gordon and Stanley2019; Rizvi et al., Reference Rizvi, Hughes and Thomas2016; Stallard et al., Reference Stallard, Porter and Grist2018). However, given the preliminary nature of existing studies with small sample sizes and no control groups, this suggested benefit requires further exploration. Self-harm and suicidal behaviours in adolescents are also significant public health concerns with huge cost implications for health services (Hawton et al., Reference Hawton, Saunders and O’Connor2012; Tsiachristas et al., Reference Tsiachristas, Casey, Brand, Leal, Park, Geulayov and Hawton2017; World Health Organization, 2021). Previous research suggests that using smartphone apps as adjuncts to traditional psychological interventions can help increase clients’ engagement and improve access to mental health skills (Austin et al., Reference Austin, Jansen, Petersen, Jensen and Simonsen2020; Donker et al., Reference Donker, Petrie, Proudfoot, Clarke, Birch and Christensen2013; Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011). In child and adolescent samples, a few studies have examined smartphone apps as an adjunct to, primarily, cognitive behaviour therapy (CBT; Silk et al., Reference Silk, Pramana, Sequeira, Lindhiem, Kendall, Rosen and Parmanto2020; Stallard et al., Reference Stallard, Porter and Grist2018; Wilansky et al., Reference Wilansky, Eklund, Milner, Kreindler, Cheung, Kovacs and Mills2016). Few studies have investigated the use of blended interventions for self-harm outcomes. However, Stallard et al. (Reference Stallard, Porter and Grist2018) reported on the use of the BlueIce app plus a 12-week psychological intervention, finding that 15% of adolescents stopped self-harm, 58% reported less frequent acts of self-harm, and 27% reported no reduction in self-harming behaviours by the end of the intervention.
A few studies have examined the use of a blended app and DBT programme for adults (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011; Rizvi et al., Reference Rizvi, Hughes and Thomas2016; Schroeder et al., Reference Schroeder, Wilkes, Rowan, Toledo, Paradiso, Czerwinski and Linehan2018). Rizvi and colleagues (Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011) examined the feasibility of using a DBT app (i.e. DBT Coach app) as an adjunct to a DBT programme for adults with BPD and substance use disorders. This study focused on enhancing the use of a specific DBT skill called ‘opposite action’ throughout a 14-day trial (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011). Participants reportedly found the app effective, engaging, and relevant for practising ‘opposite action’ (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011). A significant decrease in emotion dysregulation, depressive symptoms, and urges to use substances were found by the end of the trial (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011). A follow-up study (Rizvi et al., Reference Rizvi, Hughes and Thomas2016) further tested the same app as an adjunct to a 6-month DBT programme for adults with BPD symptoms. The authors reported that users who were more active on the app had a significant decrease in urges to self-harm following the app use (Rizvi et al., Reference Rizvi, Hughes and Thomas2016). The frequency of self-harm and severity of distress at the end of treatment also negatively correlated with the frequency of app use; however, no correlations were found between app usage and general psychopathology and emotion dysregulation (Rizvi et al., Reference Rizvi, Hughes and Thomas2016). Finally, a later iteration of the same app (renamed Pocket Skills), was trialled for 4 weeks with individuals in psychotherapy (Schroeder et al., Reference Schroeder, Wilkes, Rowan, Toledo, Paradiso, Czerwinski and Linehan2018). Their findings suggested that the app supported engagement with DBT skills practice and implementation, and thus in their perceived ability to cope.
Despite growing evidence suggesting that the use of smartphone apps as an adjunct to psychological therapy is acceptable and feasible for adults with BPD (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011; Rizvi et al., Reference Rizvi, Hughes and Thomas2016; Schroeder et al., Reference Schroeder, Wilkes, Rowan, Toledo, Paradiso, Czerwinski and Linehan2018), no known research has investigated the use of an app blended into a DBT intervention for adolescents engaging in self-harm and suicidal behaviours. Thus, the current study evaluated the acceptability and feasibility of integrating the DBT Coach app and DBT Coach clinician’s portal (Resiliens, 2021) into a DBT programme for adolescents. The DBT Coach app utilised in this study, while it had the same name as apps examined in previous studies (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011; Rizvi et al., Reference Rizvi, Hughes and Thomas2016), was an unrelated commercially available app. Thus, this is the first known study to include this app and the corresponding clinician portal. It was also of interest whether using the DBT Coach app showed preliminary effectiveness in increasing DBT skills use or impacting BPD symptoms or other clinical outcomes. Therefore, the exploratory hypotheses for quantitative outcomes were as follows:
-
(1) Adolescents will find the app acceptable and feasible as an adjunct to DBT.
-
(2) Clinicians will find the clinician’s portal acceptable and feasible as an adjunct to DBT.
The qualitative analysis explored the experiences of adolescents and clinicians using the DBT Coach app and the clinician portal as adjunctive to the DBT intervention.
Method
Participants and recruitment
Participants were recruited from a national UK DBT programme for adolescents (for a comprehensive programme description, see Camp et al., Reference Camp, Hunt and Smith2023a). Fifteen adolescents and seven clinicians opted in after providing consent for participants over the age of 16 years, and assent plus parental/legal guardian consent for participants under the age of 16. Inclusion criteria included being suitable for the DBT service and due to start the DBT skills group. Participants met inclusion criteria for the service if they were between the age of 13 and 17 years at referral, had at least one episode of self-harm in the past 6 months and presented with symptoms in at least a further five of the diagnostic domains of BPD, as assessed by the BPD subscale of the Structured Clinical Interview for DSM-IV (SCID-BPD; First et al., Reference First, Gibbon, Spitzer, Williams and Benjamin1997). Participants were unsuitable for the service and thus the study if they had another psychiatric disorder(s) that required more urgent assessment or treatment, or had opted out of the DBT service within the past 3 months.
Clinicians were recruited into the study if they were allocated as DBT individual and/or group therapists for participating adolescents and consented to take part. Clinicians were trained and experienced in delivering DBT to adolescent populations and their parents/carers (for further description, see Camp et al., Reference Camp, Hunt and Smith2023a).
Study design and procedure
A mixed-method approach was used to evaluate the acceptability and feasibility of implementing the app and portal as an adjunct to a comprehensive DBT programme. At conceptualisation of the study, clinicians within the service were consulted to ascertain need for the study, increase buy-in, and decide which available app would be most suitable. The DBT Coach app (Resiliens, 2021) was selected by clinicians as it most closely aligned with the content and aims of the skills group. The app was primarily introduced in the DBT skills group mode of treatment, which constituted the first 6 months of the 8- to 12-month DBT programme. Therefore, data collection was timed with the duration of the group component of the intervention. However, participants were also invited to use the diary card and remaining app functions in the other modes of their treatment (e.g. individual therapy), instead of traditional paper copies and similar. This was not mandatory to take part in the study, as the a priori focus was to integrate the app into the skills group component specifically the programme which hosted this study is an adaptation of the DBT intervention model for implementation within routine practice in the UK National Health Service (for a discussion of adaptations to the DBT programme, see Camp et al., Reference Camp, Hunt and Smith2023a).
Adolescent and clinician participants who consented to take part were offered an initial orientation to the app and access to the first author throughout for technical support. The DBT Coach app and clinical portal underwent a data protection impact assessment to assess data safety. To ensure data protection was in line with procedure within the host service, no identifiable information was loaded onto the app. Individual accounts for participants were created with anonymised email addresses created for this study and their participant number was used as their log-in credentials and name on the app. The clinical portal was monitored during and between sessions by DBT therapists and skills trainers allocated to the participants, and by the first author, to monitor risk.
Adolescent participants’ comfort with using technology and expectations of using the app was measured at baseline. Acceptability and feasibility outcomes (for adolescent and clinician participants) were collected 2 weeks after activating the app or portal (baseline) and at the end of the group intervention for Hypotheses 1 and 2.
Early indicators of effectiveness (dependent variables: self-harm, emerging BPD symptoms, emotion dysregulation, depression and anxiety symptoms, and skills use) were measured at the end of the intervention. App usage (independent variable) was measured throughout and regressed onto end of group scores of the dependent variables. Control variables included baseline (assessment) symptom scores of corresponding outcome measures and attendance to skills group as a proxy for treatment dosage. This was with the aim of presenting statistical trends to inform future trials.
All participants were invited to complete a qualitative interview after completing the DBT skills group. The interviews were recorded via Microsoft Teams. If participants withdrew from the study, they were still invited to attend the interview.
The DBT Coach app and clinician portal
The DBT Coach app and clinician portal is a commercially available mobile phone app developed by Resiliens Digital Health Platform (Resiliens, 2021). The app was developed in consultation with clinicians and includes content from all four modules of the adult DBT model (Linehan, Reference Linehan2015). Some Walking the Middle Path skills were included in the Interpersonal Effectiveness module, as per the adult DBT skills manual (Linehan, Reference Linehan2015). The app included a function for completing DBT diary cards, a tool used in DBT to track therapy targets, such as self-harm and suicidal behaviours, emotions, and other personalised targets (see Fig. 1). The app also included skill-based teaching inclusive of text and videos, and practical exercises to support skill practice, such as mindfulness practices, sleep diaries, journalling features, and the recording of pleasurable activities. The app was linked to a clinician portal (see Fig. 2), where clinicians were able to access all content from the app. The clinicians were able to view and download their client’s diary card responses and view which exercises they had participated in. This allowed therapists to see the number of times they had made a diary card entry or participated in an activity. Individual therapists and skills trainers were also able to set homework for individual sessions and skills group via the clinician portal, which participants could view on the app.
Examples of the DBT Coach app features (Resiliens, 2021).
The clinician’s portal home screen (Resiliens, 2021).
Measures
Assessment measures
A sociodemographic questionnaire included questions about adolescents’ age, ethnicity, sex, gender identity, sexuality orientation, and work/education status. Clinicians were also asked about their age, sex, and ethnicity.
Acceptability and feasibility measures
The Expectations and Comfort Questionnaire for adolescent participants (adapted from Rizvi et al., Reference Rizvi, Hughes and Thomas2016) is a 4-item self-report scale assessing adolescents’ comfort with using technology and how much they anticipated using the app during the study (sample item: ‘how comfortable are you using a smartphone?’). The responses were on a 5-point Likert scale option from ‘extremely comfortable’ to ‘extremely uncomfortable’.
The DBT Coach App Feasibility and Acceptability for adolescent participants (adapted from Rizvi et al., Reference Rizvi, Hughes and Thomas2016) was assessed via a 7-item self-report scale. A sample item included: ‘how helpful was the DBT Coach app in your DBT skills group?’ and the responses on the 5-point Likert scale ranged from ‘extremely helpful’ to ‘unhelpful’.
The Clinician’s Portal Feasibility and Acceptability for clinicians (adapted from Rizvi et al., Reference Rizvi, Hughes and Thomas2016) was assessed via a 5-item self-report scale. A sample item included: ‘how helpful was the DBT Coach clinician’s portal?’ and the responses on the 5-point Likert scale ranged from ‘extremely helpful’ to ‘unhelpful’.
App usage
App usage was measured by the average number of diary card entries and skills practice exercises completed on the app each week, as this was the only proxy for app usage that was available in the software. This was extracted via the clinician portal by the first author.
Early indicators of effectiveness measures
Self-harm was defined as an act of self-injury of one’s body tissue or self-poisoning, regardless of the motivation or suicidal intention, in line with the UK clinical practice guidelines (National Institute for Health and Care Excellence, 2022). Non-suicidal and suicidal self-harm were collapsed into one group due to difficulties making clear distinctions between complex motivations and functions of the behaviour, similar to previous research (e.g. Camp et al., Reference Camp, Hunt and Smith2023a; Hawton et al., Reference Hawton, Saunders and O’Connor2012; Ougrin et al., Reference Ougrin, Tranah, Stahl, Moran and Asarnow2015). The total count frequencies for self-harm were collected for the first 6 weeks of DBT (including pre-treatment) and the final 6 weeks of the DBT skills group.
A&E attendances and occupied in-patient bed days due to mental health crises and to manage risk behaviours were counted for the 6 months before DBT started and the full 6-month duration of the DBT skills group.
The Zanarini Rating Scale for Borderline Personality Disorder (ZRS; Zanarini et al., Reference Zanarini, Vujanovic, Parachini, Boulanger, Frankenburg and Hennen2003) is a self-report measure of the severity of BPD symptoms. The response scale for each item differs. Ratings are summed to create a total score. Greater scores indicate greater BPD symptom severity and the ZRS has established psychometric properties (Zanarini et al., Reference Zanarini, Vujanovic, Parachini, Boulanger, Frankenburg and Hennen2003).
The Difficulties in Emotion Regulation Scale (DERS; Gratz and Roemer, Reference Gratz and Roemer2004) is a 36-item self-report measure of emotion regulation difficulties. The DERS consists of a total score and six subscales; however, only the total emotion regulation strategies subscale scores were used for this study. Higher scores indicate increased difficulties with emotion regulation and total scores of 128 or above are suggested as a clinical cut-off (Camp et al., Reference Camp, Hunt and Smith2023a). The DERS has established psychometric properties (Gratz and Roemer, Reference Gratz and Roemer2004).
The DBT Ways of Coping Checklist, Skills Subscale (DBT-WCCL-SS; Neacsiu et al., Reference Neacsiu, Rizvi, Vitaliano, Lynch and Linehan2010) is a self-report measure of DBT skill use. The measure consists of 38 items. Higher scores indicate increased skills use. The scale shows high internal consistency and test–retest reliability for each subscale (Neacsiu et al., Reference Neacsiu, Rizvi, Vitaliano, Lynch and Linehan2010).
The Moods and Feelings Questionnaire (MFQ; Angold and Costello, Reference Angold and Costello1987) is a 33-item self-report measure of depressive symptoms in adolescents. Scores of 29 or above are predictive of a diagnosis of major depression and higher scores indicate higher severity of depression symptoms (Daviss et al., Reference Daviss, Birmaher, Melhem, Axelson, Michaels and Brent2006). The MFQ has good internal consistency, test–retest reliability, and construct validity (Daviss et al., Reference Daviss, Birmaher, Melhem, Axelson, Michaels and Brent2006).
The Screen for Child Anxiety-Related Emotional Disorders (SCARED; Birmaher et al., Reference Birmaher, Brent, Chiappetta, Bridge, Monga and Baugher1999) is a 41-item self-report screen for anxiety disorders in adolescents. A total score of 25 or above may indicate the presence of an anxiety disorder and higher scores indicate increased severity of anxiety symptoms. The SCARED has shown acceptable test–retest reliability, internal consistency, construct validity, and sensitivity to change (Birmaher et al., Reference Birmaher, Khetarpal, Brent, Cully, Balach, Kaufman and Neer1997; Birmaher et al., Reference Birmaher, Brent, Chiappetta, Bridge, Monga and Baugher1999).
Data analysis
All quantitative data were analysed using SPSS (IBM, version 28). Intention to treat methods were used for all analyses where the data were available. Descriptive data regarding demographics, the acceptability and feasibility of the DBT Coach app are presented.
For early indicators of effectiveness, a three-stage multiple hierarchical regression was conducted with end of group scores on the ZRS, DERS, DBT-WCCL-SS, MFQ, SCARED, and self-harm, and during-treatment counts of A&E attendances and occupied in-patient bed days (dependent variables). Baseline scores of corresponding dependent variables were entered to control for symptom severity scores at the start of treatment. At stage 2, DBT group attendance was entered to control for the intervention dose proxy. At stage 3, the average app use was entered to determine the relationship between the app usage (independent variable) and the outcomes at the end of group, while controlling for the treatment dose and the baseline symptoms. Relevant assumptions of this statistical analysis were tested, including independence of observations (Durbin Watson test), linearity for each variable, homoscedasticity of residuals, multi-collinearity, normality of residuals, and no significant outliers. Standardised and unstandardised beta coefficients were included as preliminary indicators of the potential relationship. The overall final model fit and statistical trends were also reported. In order to obtain 80% power of detecting a statistically significant effect, a sample size of n=43 would be needed for a medium effect (Cohens f2=0.15) or up to n=311 for a small effect (Cohen’s f2=0.02). These models were included to present statistic trends to inform future study design.
Qualitative data were analysed using a critical-realist thematic analysis approach (Braun and Clarke, Reference Braun and Clarke2022). Thematic analysis was used due to its ability to summarise meaning from qualitative data and as it has been previously used to assess the acceptability and feasibility of smartphone apps in adolescents with self-harm (Čuš et al., Reference Čuš, Edbrooke-Childs, Ohmann, Plener and Akkaya-Kalayci2021; Schiffler et al., Reference Schiffler, Seiler-Ramadas, Štefanac, Haider, Mües and Grabovac2022;). The six stages of thematic analysis were followed (Braun and Clarke, Reference Braun and Clarke2022). The authors identified and reviewed the codes, themes, and quotations representative of adolescents’ and clinicians’ experiences.
Results
Recruitment and socio-demographics
Sociodemographic variables of adolescents and clinicians can be found in Table 1. Most of the clinicians were clinical or counselling psychologists by professional training, with a few exceptions.
Sample sociodemographic characteristics
Quantitative results
Acceptability and feasibility outcomes
Feasibility outcomes, such as sample attrition, are detailed within Fig. 3.
CONSORT flow diagram of participant attrition.
Eighty-six per cent of the participants anticipated using the app more than five times per week. By the end of the group, 84.6% reported that after completing the group that they continued using the app for the remainder of their DBT treatment.
Acceptability outcomes, such as the helpfulness and usefulness of the app are summarised in Table 2. All adolescents expected the app to be ‘moderately helpful’ to ‘extremely helpful’ in practising their DBT skills at baseline. Over 70% of the adolescents anticipated that they may use the app rather than calling their therapist for telephone coaching.
Adolescents’ comfort with technology and expectations for the DBT Coach app
Responses for questionnaire items: Q1: 0=extremely uncomfortable, 1=somewhat uncomfortable, 2=neither, 3=somewhat comfortable, 4=extremely comfortable; Q2: 0=not helpful at all, 1=slightly helpful, 2=moderately helpful, 3=very helpful, 4=extremely helpful; Q3: 0=never, 1=once a week, 2=2–3 times a week, 3=4–5 times a week, 4=more than 5 times a week; Q4: 0=extremely unlikely, 1=moderately unlikely, 2=neither, 3=moderately likely, 4=extremely likely.
Overall, more than half of the adolescent sample found the app ‘a lot’ to ‘a great deal’ useful as an adjunct to the DBT skills group at the start and at the end of the group (see Table 3). At the start of the DBT skills group, 80% of the adolescent sample reported that they were somewhat ‘likely’ to ‘extremely likely’ to use the app for the remaining part of their DBT treatment after group completion. More than half of the adolescents at the start of the group and at the end of the group reported that the DBT coach app might be ‘very’ to ‘extremely helpful’ for other young people with difficulties around self-harm and/or suicidal behaviours.
Acceptability and Feasibility Questionnaire for the DBT Coach app (adolescent participants)
1 Baseline=2 weeks after the app activation and orientation. Responses for questionnaire items 1–7; Q1: 0=not at all, 1=a little, 2=moderate amount, 3=a lot, 4=a great deal; Q2: 0=unhelpful, 1=somewhat helpful, 2=neutral, 3=very helpful, 4=extremely helpful; Q3: 0=unhelpful, 1=somewhat helpful, 2=neutral, 3=very helpful, 4=extremely helpful; Q4: 0=extremely difficult, 1=somewhat difficult, 2=neither difficult nor easy, 3=somewhat easy, 4=extremely easy; Q5: 0=never, 1=once a week, 2=2–3 times per week, 3=4–5 times per week, 4=more than 5 times per week; Q6: 0=extremely difficult, 1=somewhat difficult, 2=neither easy nor difficult, 3=somewhat easy, 4=extremely easy; Q7: 0=extremely unlikely, 1=somewhat unlikely, 2=neither likely nor unlikely, 3=somewhat likely, 4=extremely likely.
Clinician ratings of the app acceptability are summarised in Table 4. Overall, 85.9% of the clinicians found it ‘slightly’ to ‘moderately helpful’ to assign and review homework via the portal at baseline, and one clinician found it ‘very helpful’ (14.3%). By the end of the group, 83.3% found it ‘moderately helpful’ to assign and review homework via the portal, and one clinician found it ‘slightly helpful’ (16.7%).
Acceptability and Feasibility Questionnaire for clinician’s DBT Coach portal (clinician participants)
1 Baseline=2 weeks after the portal activation and orientation. Responses for questionnaire items 1–5; Q1: 0=not at all, 1=slightly helpful, 2=moderately helpful, 3=very helpful, 4=extremely helpful; Q2: 0=not at all, 1=slightly helpful, 2=moderately helpful, 3=very helpful, 4=extremely helpful; Q3: 0=never, 1=once a week, 2=2–3 times per week, 3=4–5 times per week, 4=more than 5 times per week; Q4: 0=extremely difficult, 1=somewhat difficult, 2=neither easy nor difficult, 3=somewhat easy, 4=extremely easy; Q5: 0=extremely unlikely, 1=somewhat unlikely, 2=neither likely nor unlikely, 3=somewhat likely, 4=extremely likely.
Early indicators of effectiveness
The multiple regression analysis (see Tables 5 and 6) showed a positive and significant relationship between the weekly average app use and skills practice scores (DBT-WCCL-SS) at the end of the group intervention, after controlling for baseline scores and the treatment dosage. The beta values suggest that for each weekly app access, the DBT-WCCL-SS score increased by 0.14 points at the end of the intervention. The overall model accounted for 67% of variance in DBT-WCCL-SS end scores. However, no statistically significant relationships were found between average app use and any other clinical outcomes (see Tables 5 and 6). Given the limited correlations observed and small sample size, the impact of the app on clinical outcomes should be interpreted cautiously. Descriptive statistics and statistical trends for all relevant variables are also presented in Table 7.
Step 1 and step 2 of the multiple regression model
Models include clinical outcomes at the end of treatment as the dependent variable and the following independent variables: baseline scores on the matched measure (step 1), weeks in the intervention (step 2). *p<0.05. n, sample size for each model; R 2 change, how much extra variance was explained by adding the final IV (average app use) to the second model; F, fisher’s F statistic; d.f., degrees of freedom; Self-harm, frequency of self-harm incidents in final six weeks of group; Bed days, in-patient bed days; A&E, frequency of accident and emergency department visits for the duration of the intervention. The remaining clinical outcomes were collected at the end of the intervention: ZRS, Zanarini Rating Scale for BPD; DERS, Difficulties in Emotion Regulation Scale total score, clinical cut-off total score = >128; DERS-SS, Difficulties in Emotion Regulation Scale, Skills Subscale; DBT-WCCL-SS, DBT Ways of Coping Checklist, Skills Subscale; MFQ, Moods and Feelings Questionnaire total score, clinical cut-off = >29; SCARED, Screen for Child Anxiety-Related Emotional Disorder total score, clinical cut-off = >25.
Effect of average app use on clinical outcomes using multiple regressions (step 3)
Direct relationship outputs represent relationship between average weekly app use and clinical outcomes at end-treatment. Final model includes clinical outcomes at the end of treatment as the dependent variable and the following independent variables: baseline scores on the matched measure (step 1), weeks in the intervention (step 2), and average weekly app use (step 3). *p<0.05. **p<0.01; n, sample size for each model; b, unstandardized beta; SE, standard error; 95% CI UB, confidence interval upper bound and LB, lower bound; R 2 change, how much extra variance was explained by adding the final IV (average app use) to the second model; F, Fisher’s F statistic; d.f.,= degrees of freedom; Self-harm, frequency of self-harm incidents in final six weeks of group; Bed days, in-patient bed days; A&E, frequency of accident and emergency department visits for the duration of the intervention. The remaining clinical outcomes were collected at the end of the intervention: ZRS, Zanarini Rating Scale for BPD; DERS, Difficulties in Emotion Regulation Scale total score, clinical cut-off total score = >128; DERS-SS, Difficulties in Emotion Regulation Scale, Skills Subscale; DBT-WCCL-SS, DBT Ways of Coping Checklist, Skills Subscale; MFQ, Moods and Feelings Questionnaire total score, clinical cut-off = >29; SCARED, Screen for Child Anxiety-Related Emotional Disorder total score, clinical cut-off = >25.
Descriptive statistics for outcome variables included in the multiple regression model
n, sample size for each model; Self-harm, frequency of self-harm incidents in final 6 weeks of group; Bed days, in-patient bed days; A&E, frequency of accident and emergency department visits for the duration of the intervention. The remaining clinical outcomes were collected at the end of the intervention: ZRS, Zanarini Rating Scale for BPD; DERS, Difficulties in Emotion Regulation Scale total score, clinical cut-off total score = >128; DERS-SS, Difficulties in Emotion Regulation Scale, Skills Subscale; DBT WCCL-SS, DBT Ways of Coping Checklist, Skills Subscale; MFQ, Moods and Feelings Questionnaire total score, clinical cut-off=>29; SCARED, Screen for Child Anxiety-Related Emotional Disorder total score, clinical cut-off = >25.
Qualitative results
Five subthemes from the qualitative interviews were established (within two over-arching themes) for the adolescent and clinician sample (see Table 8).
Over-arching themes and subthemes from the qualitative interviews
Qualitative results for adolescent participants
Over-arching Theme 1 – DBT Coach App Benefits
This over-arching theme categorises adolescent participants’ positive experiences of using the DBT Coach app.
Subtheme 1 – Diary Cards Accessibility
Adolescents described the DBT Coach app as an accessible tool for increasing the ease of completing their weekly diary cards. For example, participant (P)10 said:
[DBT Coach app] definitely made it easier to complete the diary card … because when the diary card was on paper, it was not good, like I barely ever did it. I think it’s just a lot more accessible, and you can do it from anywhere, and you don’t have to go get the piece of paper or pen.
Subtheme 2 – Skills Practice
Adolescents described that the app supported with the practising of DBT skills between sessions. For example, P7 reported: ‘I found it helpful that [the DBT Coach app] had skills that we were learning in [skills] group. it was helpful how it had step-by-step guide, like they taught us in the group’. P2 also reported: ‘I definitely used [the app] for my homework, read about the skills, and even when I did not understand, I used the lessons, videos [on the app]’.
Subtheme 3 – Memory Aid
Adolescents described the app as a useful tool in remembering to complete their homework and discuss events that occurred over the week during their individual sessions. P13 shared:
yes, it’s good to, like, track … my mood … daily because it can change quite a lot. It was good just to see what’s going wrong in different places or … how I could have acted. I like to analyse the day, I guess. The app was super helpful ’cause … I’m really forgetful. So like if … I’m … on the app anyways, and I see that I have homework, and that’s like helpful. Just to remind me, ’cause, I usually forget.
Over-arching Theme 2 – DBT Coach App Challenges
Subtheme 4 – Technical Difficulties
Adolescents described some technical challenges of using the DBT Coach app, for example being logged off or the app crashing. This sometimes had a particular impact when participants were attempting to use the app while distressed or for skills coaching. For example, P7 said:
I found it challenging, sometimes it was hard to navigate and [I] got logged out of the app, which was quite difficult especially if I was trying to use the app while I was distressed as I was already emotionally volatile. And then going to the app and being logged out was another trigger that I found hard to deal with, and I did not have the skills to deal with it because I could not access the app.
Subtheme 5 – User Unfriendliness
Some adolescents reported that they struggled with finding different handouts or exercises on the app, and sometimes they also found it difficult to read the text on the app; P4 said: ‘the layout could have been a bit easier; like I am not great with reading things, and it could have been structured simpler’.
Qualitative results for clinician participants
Over-arching Theme 1 – Clinician’s Portal Benefits
Subtheme 1 – The Usefulness of Journalling
Most clinicians found the portal helpful for reviewing diary cards. They specifically liked the journalling feature of the diary card. This function was reported to be helpful when completing chain analyses during individual sessions. For example, Clinician (C) 1 said: ‘[the adolescent participant] would often journal a bit, and that would almost begin the chain analysis… and so I think it was quite good for promoting a bit of an awareness and noticing emotions and thoughts and triggers’. C2 shared: ‘[the journal] contextualized where the emotions were coming from and what was happening on that day’.
Subtheme 2 – Homework Assignment Efficiency
Some clinicians found the portal helpful and efficient for assigning homework. For example, C5 said: ‘I think because [the DBT Coach portal] makes it just a lot quicker and efficient to assign the homework … you just go into the app and set [homework] for everyone’. C6 also reported: ‘just being able to quickly set homework whilst you’re like in the session. I’d be able to go OK, I’m gonna just send your homework on to the app’.
Over-arching Theme 2 –Clinician’s Portal Challenges
Subtheme 3 – Getting to Grips with New Technology
Some clinicians shared their difficulties with learning or navigating the DBT Coach portal in addition to other responsibilities within a busy clinical context. For example, C6 shared:
I think when you’re really busy, it just felt like an extra thing, you know. Like, oh God, I gotta print [the diary card] off or take it down or get my phone or log in and or what’s the login and that kind of thing.
C7 also reported:
The whole process of checking if the piece of equipment [iPad] is plugged in and if I’m picking up the internet that I’ve got the information [from the electronic diary card]. And … you know you could export the information, but all of that just doesn’t come to me easily or quickly, or particularly when I’m rushing between sessions.
Subtheme 4 – Unclear Diary Card Layout
Most clinicians found the layout of the diary card, as presented on the clinician’s portal, confusing and challenging. One reasons for this was that the ratings for the presence of action/behaviours, such as self-harm, were rated on a scale of 1–5 severity rather than a ‘yes’ or ‘no’ or frequency count. For example, C6 expressed: ‘I don’t like that [the diary card] doesn’t have a ‘yes’ or a ‘no’ for self-harm [behaviours]. There’s space for misunderstanding and miscommunication about what has happened … There is scope for things getting missed, which is a problem’. C1 also stated:
You know, the old [paper] diary card, we are so used to it, and it’s so like one of the advantages of the traditional diary card on paper, or even receiving it in a PDF … I think as an experienced DBT therapist you get used to, just looking at the … [paper diary card], and you can … look at that diary card and … you can … absorb it fairly efficiently. One of the things I’ve found hard adjusting to the portal was that I couldn’t … see [the contents of the diary card] all instantly like that. That would be better.
Subtheme 5 – Technical Difficulties
Clinicians also experienced technical difficulties with the portal, such as being logged off, the web link not working, or problems with the two-step authentication code for logging in. For instance, C6 shared: ‘it’s a bit of a pain logging into the portal … You… put [in] your details, and you have a code being text to you, and sometimes it doesn’t come for like five minutes, and that’s really annoying…’.
Discussion
The current study evaluated the feasibility and acceptability of using the novel DBT Coach app and clinician’s portal (Resiliens, 2021) within a comprehensive DBT programme for adolescents. This study used a combination of quantitative and qualitative methods. To inform future studies, preliminary findings on statistic trends regarding potential relationship between app usage and outcomes of clinical interest were included.
In terms of the acceptability and feasibility findings, the current study suggests that the majority of adolescents who completed the follow-up reported finding the app helpful. Most adolescents recommended the DBT Coach app for other young people with difficulties associated with self-harm and/or suicidal behaviours. The quantitative findings suggest that adolescents largely felt comfortable with using smartphone apps in anticipation of the study. However, there was a slight decrease in the proportion of adolescent participants rating the app as helpful generally and for self-harm than they anticipated at the start of the study. Based on the qualitative results, adolescents also found the app helpful for remembering to complete their therapy homework, including completing diary cards and practising skills. These findings are consistent with the previous studies demonstrating that DBT apps were experienced as a helpful addition to an adult DBT programme for learning and practising DBT skills (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011; Rizvi et al., Reference Rizvi, Hughes and Thomas2016). Similarly, another study found that adolescents liked mental health apps with reminder notifications, alongside their mental health intervention, to help them remember to complete exercises on the app (Kenny et al., Reference Kenny, Dooley and Fitzgerald2016). Therefore, using apps in conjunction with interventions, such as with DBT, may be valuable in promoting skills use and completing therapy tasks, such as symptom monitoring forms (i.e. diary cards). This is an important finding as it is thought that homework completion and practising skills can increase the generalisation of DBT skills outside of the therapy context (Linehan, Reference Linehan1993; Linehan, Reference Linehan2015), and DBT skill use is found to promote improved outcomes (Linehan et al., Reference Linehan, Korslund, Harned, Gallop, Lungu, Neacsiu and Murray-Gregory2015). However, more robust methodology is needed to test out these suggestions.
Largely, the findings suggest that adolescents found the app easy to navigate. However, other aspects of the qualitative results suggest that some experienced the app as challenging to use at times due to technical difficulties. They also shared that the layout/design of the app was not always easy to navigate, such as finding it difficult to find the skills when in different subsections or navigate the amount of text in the skills information. This was particularly problematic when young people were seeking to use the app while distressed for skills coaching, as it was harder to access the required information quickly and thus participants reported often relying on their own knowledge of skills or phone coaching with their therapist. This may be a barrier for relying on this technology in its current form for skills coaching and crisis support. It was not the intention of this study, however, for the app to replace the telephone coaching mode of DBT, but to support skill enhancement. Therefore, it may be that barriers to app use when distressed means that the app may better support the implementation of other treatment modes or considered as a way to continue skills practice after the end of DBT. Despite these challenges, most adolescents continued using the app for the remaining part of their DBT programme (i.e. once the group aspect finished). Previous research highlights the importance of designing mental health apps as accessible, visually interactive, and easy to navigate (Kenny et al., Reference Kenny, Dooley and Fitzgerald2016; Schiffler et al., Reference Schiffler, Seiler-Ramadas, Štefanac, Haider, Mües and Grabovac2022). Therefore, the current findings suggest that the design and technical problems could causes barriers to using the app for their intended function, thus reducing aspects of acceptability and feasibility.
In terms of the acceptability and feasibility findings for clinicians, the results from the clinician participants suggested more mixed findings of implementing the use of DBT Coach clinician’s portal. Clinicians found the portal particularly helpful for reviewing the journal portion of the diary card, as it provided additional information compared with the traditional paper diary card. According to clinicians, the data from the diary card was discussed with the adolescent during individual therapy sessions and helped them recall specific information on events from the week. Another benefit of using the portal was the ease of assigning homework. However, clinicians also rated the app as less helpful and less easy to navigate than they anticipated it would be at the beginning. There is limited prior evidence for a clinician’s portal function in other DBT-related apps reported on in previous research (Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011; Rizvi et al., Reference Rizvi, Hughes and Thomas2016). One study using a CBT intervention plus app found that clinicians experienced that corresponding portal as easy to use and efficient for assigning and reviewing homework (Silk et al., Reference Silk, Pramana, Sequeira, Lindhiem, Kendall, Rosen and Parmanto2020), similar to some of the findings of this study. The small sample and preliminary nature of the current study, however, should be interpreted with caution.
In contrast, some clinicians experienced challenges with the portal and a few indicated that they were somewhat unlikely to use the portal after the study. This was largely due to technical difficulties and the layout of the diary card output. A particular issue raised by clinicians on the diary card output was the ratings for behaviour targets (e.g. self-harm, suicidal behaviours), which were on a rating scale of severity rather than a ‘yes/no’ or frequency format. Clinicians raised concerns about possible misunderstandings about how the app diary card monitored treatment targets and risk behaviours, and the possibility of missing risk events. Despite this, clinicians confirmed that they had encountered no safety issues regarding risk when they asked their young person about the frequency and severity of self-harm incidents. However, the app and portal design may make the app less acceptable and feasible to be integrated with a DBT intervention from clinicians’ perspectives and it is important when managing high risk behaviours that information is clear and accessible to prevent risk incidents. Also, if clinicians were unable to check the risk information via the app regularly between sessions, this would not be a suitable replacement for other mechanisms to support risk management between session, such as via telephone coaching (Linehan, Reference Linehan1993). For this reason, some clinicians were hesitant to implement new technology in a busy clinic for high-risk young people. This suggests that clinicians with high caseloads, time constraints, and levels of stress may experience it as challenging to find the time and capacity to master new technology and incorporate it into everyday practice, similar to previous research into implementing digital interventions in child and adolescent mental health services (Owens and Charles, Reference Owens and Charles2016). For those attempting to integrate apps with clinician-led components into existing intervention, they may benefit from providing sufficient time and support for clinicians to develop appropriate skills to use the technology and integrate it into their practice.
This study presents statistical trends from the use of a smartphone application integrated into a DBT program for adolescents, aiming to explore its potential usefulness in future research and possible implementation in clinical practice. Therefore, these quantitative findings from the regression analysis should be treated tentatively; they should not be used to infer effectiveness or causation. In the future, fully powered trials are needed to investigate the nature and magnitude of these relationships. Previous studies have found a relationship between the use of apps within blended interventions and decreases in self-harm, BPD symptoms, and depression symptoms; however, these findings are not replicated in all studies (Kennard et al., Reference Kennard, Goldstein, Foxwell, McMakin, Wolfe, Biernesser and Brent2018; Rizvi et al., Reference Rizvi, Dimeff, Skutch, Carroll and Linehan2011; Rizvi et al., Reference Rizvi, Hughes and Thomas2016; Schroeder et al., Reference Schroeder, Wilkes, Rowan, Toledo, Paradiso, Czerwinski and Linehan2018). No known previous study has looked at the relationship between app use, within a blended intervention, and the remaining statistical trends included in this study.
Strengths and limitations
In terms of the limitations, this study reported findings from a small sample size recruited from one service context using convenience sampling. Therefore, the findings may not generalise to broader populations and treatment settings or be representative of the wider target population. These statistical trends are useful for further exploration in the future studies. In addition, no inclusion of a control group means that inferences regarding effectiveness cannot be made and are beyond the aims of this study. Therefore, no inferences about effectiveness or causation can be made from these data. Additionally, the lack of control or measurement of potentially extraneous variables, such as therapeutic alliance, are areas for possible consideration in future studies.
The use of diary card completion and skills practice on the app as a proxy for app use may also be a blunt measure, given that this would not represent attempts to access the app for other reasons and time spent on the app. It was also not possible to determine the type of activity on the app, only that it was accessed for either of the two aforementioned reasons. It was not possible to collect data on app usage by time or any other proxy. Future studies would benefit from a more detailed and accurate measure of app usage. The app was also primarily introduced and implemented within the skills group component of the DBT programme, thus data collection focused on this treatment component. However, many participants utilised the app in other modes of the intervention (i.e. primarily individual sessions), as was evident from the focus on diary card usage and feedback about this function in the qualitative analysis. As this was not the a priori intention of the study, data collection did not intentionally focus on this outcome or on outcomes for the remaining treatment modes of the programme beyond the skills group. Future studies are needed to assess outcomes for the entire DBT programme and consider the experience of implementing the app in other modes of treatment as well. This may also include utilising attendance to individual sessions, as well as skills group sessions, as a controlling variable in any future quantitative analyses. Another limitation of this study is the exclusion of some of the ‘Walking the Middle Path’ skills from the adolescent model (Rathus and Miller, 2015), as the app was largely informed by the adult skills model (Linehan, Reference Linehan2015).
The low representation of diversity with regard to ethno-racial minoritised groups and male assigned sex at birth groups may mean results do not generalise to these or any other unrepresented population in this study. Given that people from ethnicity-related minoritised backgrounds (Al-Sharifi et al., Reference Al-Sharifi, Krynicki and Upthegrove2015; Lei et al., Reference Lei, Watkins-Muleba, Sobogun, Dixey, Bagnall and Camp2024) and male populations (Botti et al., Reference Botti, Cantão, Silva, Gonçalves Dias and Menezes2018; Canetto and Cleary, Reference Canetto and Cleary2012) have a high prevalence of and unique risk factors contributing to self-harm and suicidal behaviours, similar future studies should make efforts to recruit more diverse samples. The qualitative analysis may also be impacted by similar sampling limitations listed above.
However, the current study is the first to explore the acceptability and feasibility of the DBT Coach app as an adjunct to a DBT skills group intervention, within a comprehensive DBT programme for adolescents. The current study is also the first to explore clinicians’ experiences of using the DBT Coach portal as part of a DBT intervention. Additionally, the majority of the adolescents identified as bisexual and there was representation of gender minorities (e.g. transgender groups); a minority group typically under-represented in the literature (Harned et al., Reference Harned, Coyle and Garcia2022; Pachankis, Reference Pachankis2018). Research shows that sexual and gender minority groups are at very high risk of suicide and self-harm compared with cisgender and heterosexual groups (Clark et al., Reference Clark, Salway, McConocha and Pachankis2022; Hunt et al., Reference Hunt, Morrow and McGuire2020) and have unique needs (Camp et al., Reference Camp, Morris, Wilde, Smith and Rimes2023b). They are also over-represented in DBT populations (Camp et al., Reference Camp, Durante, Cooper, Smith and Rimes2024; Harned et al., Reference Harned, Coyle and Garcia2022).
Conclusions
The current study found that integrating the novel DBT Coach app and portal into a DBT programme for adolescents was largely feasible and acceptable to adolescent participants. While many clinicians also found this feasible and acceptable, there were important barriers highlighted. Statistical trends regarding the use of a smartphone application and clinical outcomes are presented to inform future studies.
Key practice points
-
(1) The DBT Coach app may be an acceptable and feasible addition to a DBT programme for adolescents and may support for skills practice and diary card completion.
-
(2) While there are feasible and acceptable aspects, it is of note that technical issues, complexity of layout, and differences of reporting methods on the diary card may be barriers to implementation and thus warrant consideration.
-
(3) When attempting to implement apps into existing service and intervention contexts, considerable time is needed to support clinicians to develop appropriate skills to implement the app and feel comfortable with its use.
Data availability statement
The quantitative data that support the findings of this study are available from the corresponding author, NR, upon reasonable request.
Acknowledgements
Our deepest appreciation to all the service users, parents/carers; and clinicians for investing their time to take part in this research. A particular acknowledgment to Dr Katrina Hunt for her support with facilitating this project within the DBT service.
Author contributions
Natasha Ramzan: Conceptualization (equal), Data curation (lead), Formal analysis (lead), Investigation (equal), Methodology (equal), Project administration (lead), Resources (equal), Validation (equal), Writing - original draft (lead), Writing - review & editing (lead); Jake Camp: Conceptualization (equal), Data curation (equal), Formal analysis (equal), Investigation (equal), Methodology (equal), Project administration (supporting), Resources (equal), Supervision (equal), Validation (equal), Writing - original draft (supporting), Writing - review & editing (equal); Troy Tranah: Conceptualization (equal), Data curation (equal), Formal analysis (supporting), Investigation (equal), Methodology (equal), Project administration (supporting), Resources (equal), Supervision (equal), Validation (equal), Writing - original draft (supporting), Writing - review & editing (equal).
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
All authors were or had previously worked in the host service context.
Ethical standards
This study abides by the Ethical Principles of Psychology and Code of Conduct set out by the BABCP and BPD. Ethical approval was provided by the NHS Research Ethics Committee (ID: 21/WA/0070). Informed consent was sought from participants and participants have been sent a draft of the results before submission.
Open access
Comments
No Comments have been published for this article.