Introduction
‘All young people deserve the best start in life’ was a statement published by the Departments of Health and Department of Education (2017; p. 2). However, survey data in 2022 reported that one in six children and young people (CYP) aged between 7 and 16 years were living with a probable mental health disorder (Newlove-Delgado et al., Reference Newlove-Delgado, Marcheselli, Williams, Mandalia, Davis, McManus and Ford2022). There has been a reported rise in the prevalence of mental health difficulties in CYP aged 6–16 years from 11.6% in 2017 to 17.4% in 2021, with anxiety and depression being the most common difficulty reported in school-age children (Newlove-Delgado et al., Reference Newlove-Delgado, Williams, Robertson, McManus, Sadler, Vizard and Ford2021; Sadler et al., Reference Sadler, Vizard, Ford, Goodman, Goodman and McManus2018). A dataset of 1920 children and young people referred to CYPs mental health services between 2012 and 2017 showed that more girls presented with anxiety (424 girls vs 266 boys) and depression (110 girls vs 39 boys) than boys. In contrast, almost double the number of boys presented with behavioural difficulties than girls (97 boys vs 54 girls) which suggests a gender difference in girls being referred for internalised symptoms compared with boys’ externalising symptoms (Baron et al., Reference Baron, Hurn, Ainger, Adlington and Shapiro2021). Addressing the mental health crisis facing CYP is important due to the impact these difficulties can have on a child’s quality of life, academic attainment, and social functioning (Mendlowicz and Stein, Reference Mendlowicz and Stein2000; Bhatia and Bhatia, Reference Bhatia and Bhatia2007). Mental health difficulties in childhood are associated with adverse outcomes in the transition to adulthood, such as difficulties in future employment and increased risk of receiving a mental health diagnosis later in life (Goodman et al., Reference Goodman, Joyce and Smith2011; Jones, Reference Jones2013). Therefore, effective treatment is crucial.
In a review of meta-analyses, cognitive behavioural interventions appear to be effective in treating a variety of mental health disorders, both clinical and sub-clinical in CYP across a variety of service delivery setting encompassing clinical environments, schools and community-based interventions (Hofmann et al., Reference Hofmann, Asnaani, Vonk, Sawyer and Fang2012). Specifically, cognitive behavioural therapy (CBT) was found to be preferential in treating anxiety disorders in CYP with treatment effects within the large range. For obsessive compulsive disorder, CBT showed significantly better outcomes compared with no treatment, other psychosocial treatments and medication. Evidence of the efficacy of CBT in treating depression is less strong but showed a medium effect size across reviews, which was maintained at 6-month follow-up. The review reported mixed findings regarding externalising disorders, with CBT showing better efficacy than no treatment or treatment as usual, equal efficacy to other psychosocial treatments and less efficacy than pharmacological approaches. Cognitive behavioural approaches therefore largely appear to be an effective and economical approach in treating mental health disorders in CYP. Unfortunately, what we are not able to conclude from this review is whether the effectiveness of CB approaches differed based on symptom severity or service delivery context, as no comparisons were made between these groups. Given the impact that mental health difficulties have on the lives of CYP, early intervention is paramount and there is a need to investigate the effectiveness of CB interventions as a preventative treatment to reduce the longer-term impact of mental health difficulties on both CYP and the economy (Jones, Reference Jones2013; The Children’s Society, 2017).
Current practice guidelines for this age group recommend CB-informed interventions, delivered by trained staff within an education setting (e.g. school), as first-line intervention for mild to moderate difficulties (National Institute for Health & Care Excellence, 2023). Research evaluating the efficacy of CB interventions specifically as a preventive approach is limited but emerging. Caldwell and colleagues (Reference Caldwell, Davies, Hetrick, Palmer, Caro, Lo ṕez-Lo ṕez, Gunnell, Kidger, Thomas, French, Stockings, Campbell and Welton2019) conducted a systematic review and network meta-analysis assessing the effectiveness of preventative school-based interventions that specifically aim to reduce symptoms of anxiety and depression in primary and secondary age children. Within their review they included any study which implemented universal or targeted psychological, educational or physical intervention, delivered within an education setting. This included both individual and group formats. In primary schools they found weak evidence for the effectiveness of universal CBT interventions in reducing symptoms of anxiety but not depression. For secondary schools, universal mindfulness and relaxation interventions showed a reduction in anxiety symptoms and universal CBT showed weak evidence for reducing anxiety symptoms. Targeted exercise interventions were effective for reducing anxiety symptoms relative to no intervention comparator, but no interventions were found effective in reducing symptoms of depression. The review authors concluded weak evidence for the efficacy of specific school-based interventions in reducing symptoms of anxiety and depression in CYP and suggested that interventions address the familial and structural context in which they are implemented. It should be noted that the studies included in this review were of high risk of bias and therefore must be interpreted with caution.
The Young Minds report, ‘Talking about Talking Therapies’, gathered CYP’s experiences of mental health support and advocated for the importance of accessible and timely mental health services with an emphasis on improving access and choice for CYP struggling with their mental health (Badham and Young Minds, Reference Badham2011). The Children and Young People’s Mental Health and Wellbeing Taskforce, led by NHS England and the Department of Education, provided a new vision for the delivery of CYP’s mental health services with the aim of developing and expanding the workforce, promoting early intervention, and improving access to services (Independent Mental Health Taskforce, 2016; Department of Health CYP Mental Health & Wellbeing Taskforce, 2015). This transformation is being implemented through a collaboration between health, social care and education services, aiming to create a whole-school and college approach to service delivery (Department of Health and Department for Education, 2017). A key element was establishing new services to be known as Mental Health Support Teams (MHSTs) with three main functions: promoting collaborative working between health and education services; providing early interventions to CYP within education settings (e.g. schools, colleges, and special educational needs settings); and undertaking preventative work through the whole school systemic approach to support schools in ensuring CYP receive the right support in a timely way.
To deliver on the promise of a new workforce, two graduate practitioner roles were developed, both of which have 12-month training curricula, providing low-intensity CB-informed interventions to CYP with mild to moderate mental health conditions. Children’s Wellbeing Practitioners (CWPs) are based in NHS and voluntary services, whilst the other, Education Mental Health Practitioners (EMHPs), are based in MHSTs. These practitioners have additional training related to the education setting and whole-school approach (Department of Health and Department for Education, 2017).
Given that both the CWP and EMHP roles are in their infancy, it is pertinent to evaluate their ongoing impact in practice. Currently, published research attempting to evaluate the effectiveness of these roles is limited. In a review of the new workforce, it was reported that low-intensity CB-interventions show promising outcomes, but this conclusion was made based on the review of guided self-help intervention without reference to any direct evaluation of CWP or EMHP interventions (Ludlow et al., Reference Ludlow, Hurn and Lansdell2020). A pilot practice-based evaluation of the effectiveness of CB-informed interventions delivered by both trainee CWPs in Child and Adolescent Mental Health Services (CAMHS) and trainee EMHPs in schools found moderate to large improvements across all child- and parent-reported outcome measures post-intervention in 109 CYP with common mental health difficulties (Lockhart et al., Reference Lockhart, Jones and Sopp2021). In a second paper evaluating the effectiveness of CB-informed interventions delivered by CWPs within one CAMHS in the North East of England, interventions were found to have moderate to large effects on symptom reduction across all Revised Children’s Anxiety and Depression Scale (RCADS) subscales (separation anxiety, social anxiety, generalised anxiety, obsessive-compulsive disorder, and panic disorder) in addition to supporting significant goal progress (Turnbull et al., Reference Turnbull, Kirk, Lincoln, Peacock and Howey2023).
The two practice-based evaluation studies described have strength in their high ecological validity, given the interventions were delivered within the context for which they are designed to be delivered. Together, they provide an early indicator for the effectiveness of CB-informed interventions in treatment of mild to moderate difficulties in CYP within health (CAMHS) and education settings (EMHPs). However, given the use of only pre–post data without a control group, the interpretation of these findings is inherently limited in that they may inflate the perceived effectiveness of CB intervention due to factors such as organic improvement over time. Previous research has demonstrated that CYP seen within a CAMHS setting show significantly higher baseline symptom severity than those seen within an education setting (Lockhart et al., Reference Lockhart, Jones and Sopp2021) but similar effect sizes were found across both studies, one including EMPHs and CWPs and the other only CWPs. This suggests that service context to which low-intensity CB-informed interventions are delivered may not impact outcomes, but future research is needed to evaluate these two roles separately (CWP and EMHP). Given the lack of published literature to date, it is difficult to conclude on the value of these roles in practice.
Service context
The first wave of 58 MHSTs were piloted across England in 2018 and received positive feedback from school staff (Ellins et al., Reference Ellins, Hocking, Al-Haboubi, Newbould, Fenton and Daniel2023). Additional funding went towards developing 87 more MHST sites working in collaboration with over 4700 schools and colleges across England (NHS England, Reference NHS2023). This service evaluation is of one MHST service covering four sites in the South East of England, which delivers CB-informed interventions in over 62 education settings including primary schools, secondary schools, special educational needs (SEN) schools and colleges. EMHPs work within these settings to deliver short-term (up to 10 sessions) evidence-based CB-informed interventions for mild to moderate difficulties.
This service evaluation aims to investigate the effectiveness of individual interventions delivered across four MHST sites using child- and parent-rated outcome measures collected routinely in CYP services (Wolpert et al., Reference Wolpert, Jacob, Napoleone, Whale, Calderon and Edbrooke-Childs2016).
The research questions for this paper are:
-
(1) Are interventions delivered by MHST practitioners effective in decreasing self- and parent-reported (a) symptoms of anxiety and depression and (b) emotional and behavioural difficulties, in young people with mild to moderate mental health difficulties?
-
(2) Are interventions delivered by MHST practitioners effective in reducing symptomology across the five anxiety profiles assessed on the RCADS?
-
(3) Are MHST interventions equally as effective in both boys and girls?
Method
Participants
Records were gathered for all CYP referred to the four MHST sites between January 2021 and December 2022. In total, there were 725 referrals into the service during this period. Referrals came via one of the following channels: a member of staff across the 62 partner education settings (e.g. schools and colleges), a self or parent referral, or via another CYP mental health service. Referrals were screened against service eligibility criteria including the CYP being aged between 5 and 18 years and presenting with symptoms suitable for a short-term (up to 10 sessions) CB-informed intervention. Symptom suitability was defined as CYP who presented with symptoms of a mild to moderate mental health concern, such that the symptoms were having a subjective impact on one or a few areas of functioning. Referrals were rejected if the young person did not identify any concern, if their symptoms were impacting most or all areas of functioning or if the child presented with a level risk that needed multi-disciplinary management. The screening process was completed within a multi-disciplinary team made up of at least one representative from health, social care and the MHST programme support assistant. Out of 725 total referrals, 266 were excluded from analysis either because the referral was signposted to an alternative service, or they were offered a group level intervention. There were 459 CYP remaining who had been accepted into the service and offered an individual intervention.
Demographics and intervention characteristics
Age at referral ranged from 5 to 18 years with a mean age of 11.7 years (SD=3.14; Table 1). Gender was coded based on school records and includes a mixture of both sex at birth and gender as identified by the young person. There were almost double the number of girls (65.6%) than boys (33.3%), and 83% identified as White British.
CYP demographics
At pre-intervention, according to the RCADS standard clinical thresholds (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000), the majority of CYP were in the non-clinical threshold (<65) for overall anxiety (55.6%) and depression (50.6%), 12.9% were in the borderline clinical range (65–69) for both anxiety and depression, and 31.6% and 36.5% in the clinical range (>70) for anxiety and depression, respectively. These proportions appear to reflect the remit of the MHST service.
On average each intervention consisted of nine sessions (SD=2.1) with the mean duration being 3.97 months (SD=1.57). Each intervention included assessment, formulation, and setback management in addition to the primary intervention. Intervention type, used either in isolation or in combination, was collected based on what practitioners wrote in their closing summary, or if this was not available, by reading the session notes to identify what interventions were completed. There were several types of intervention recorded, as logged by MHST practitioners on case notes, with cognitive restructuring alone (33.8%) being the most frequent, followed by worry tree alone (16.3%), graded exposure alone (12.6%), problem-solving alone (11.1%) and behavioural activation alone (9.2%). There were also a number of cognitive, behavioural and psychoeducational techniques being combined within one intervention (see full list in Table 2).
Overview of all techniques used in isolation and combined as logged by MHST practitioners
Design
In line with previous studies reviewing the effectiveness of CB-informed interventions in CYP, a repeated measures pre–post outcome evaluation design was conducted (Fuggle and Hepburn, Reference Fuggle and Hepburn2019; Lockhart et al., Reference Lockhart, Jones and Sopp2021). The severity of difficulties was quantified using child- and parent-reported versions of two measures: the Revised Children’s Anxiety and Depression Scale (RCADS; Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000), and the Strengths and Difficulties Questionnaire (SDQ; Goodman, Reference Goodman2001). These measures are included in practice guidelines set out by the Children and Young People’s Improving Access to Psychological Therapies initiative and have been validated for measuring symptom severity and treatment effects in children and young people (Law and Wolpert, Reference Law and Wolpert2014).
Measures
Revised Children’s Anxiety and Depression Scale (Chorpita et al., 2000)
The RCADS is a 47-item questionnaire with both a self- and parent-rated version, measuring the frequency of anxiety and depression symptoms on a 4-point scale from ‘never’ to ‘always’ in children aged 8–17 years, with higher scores indicating greater severity. Scores are added to provide a major depressive disorder (MDD) score and five anxiety subscale scores (social phobia (SP), separation anxiety disorder (SAD), generalised anxiety disorder (GAD), panic disorder (PD) and obsessive-compulsive disorder (OCD)). The sum of the anxiety subscales also provides a total anxiety score (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000). Raw scores are converted into T-scores using conversion tables, provided by the developers, with a T-score over 70 indicating clinical threshold (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000). We computed the reliability measure of Cronbach’s alpha for the total anxiety and depression scales in addition to the five anxiety subscales and found acceptable internal consistency for the SAD and OCD scale (α=0.71 and 0.74, respectively), good internal consistency for the SP, PD, GAD and MDD scale (α=0.82–0.88) and excellent internal consistency for the overall anxiety scale (α=0.93). The parent-rated overall anxiety scale showed excellent internal consistency (α=0.91), the MDD, SP, PD and GAD scales showed good internal consistency (α=0.81–0.88) and the SAD and OCD scales showed acceptable internal consistency (α=0.76–0.78) (George and Mallery, Reference George and Mallery2003). This is consistent with previous research reporting questionable to excellent internal consistency (α=0.60–0.96) in a non-clinical sample (Donnelly et al., Reference Donnelly, Fitzgerald, Shevlin and Dooley2019).
Strengths and Difficulties Questionnaire (Goodman, 2001)
The SDQ measure is a 25-item emotional and behavioural screening questionnaire asking participants to rate various positive and negative attributes on a 3-point scale from ‘not true’ to ‘certainly true’ based on ‘how things have been for them’ over the past 6 months. This has been produced as both a self-report measure for children aged 11–17 years and a parent-rated measure for CYP between 4 and 17 years. The questionnaire derives five scale scores for: emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems, and pro-social behaviour. The first four scales are added to provide a total difficulties score. There is a supplementary impact measure consisting of five questions assessing overall distress and impairment, the sum of which provides the total impact score where a score over 1 is seen as slightly raised and within the clinical threshold. Cronbach’s alpha for the 25-item total difficulties and 5-item impact scale were questionable to acceptable for both CYP and parent scales (α=.61–.73), corresponding with previous research reporting acceptable internal consistency (α=0.73) (George and Mallery, Reference George and Mallery2003; Goodman, Reference Goodman2001).
Procedure
The 459 CYP who met the criteria for an individual intervention and consented to engage with the MHST service were offered up to 10 sessions of a CB-informed intervention. The intervention type was based on their presenting difficulties and delivered either face to face within school or online via videocall by an MHST practitioner, the majority of which were either trainee or fully qualified EMHPs, in addition to other qualified practitioners delivering CB-informed interventions. The 459 CYP were invited to complete age-appropriate outcome measures (RCADS and SDQ for CYP over 11 years and RCADS only for those 8–11 years) at the start (pre) and end (post) of the intervention. Parents and carers were invited to complete the SDQ for all CYP regardless of age, in addition to the RCADS for those above 8 years old. Outcomes were administered on an online outcome measures data collection platform and stored on an electronic database as part of routine data collection.
Ethics
Data were anonymised and collated on an SPSS data set before analysis. As this was standard service data, consent was not required but all data were stored and analysed on a secure NHS laptop to maintain anonymity and confidentiality. To mitigate any concern from practitioners of being personally evaluated, we ensured that no practitioner-identifiable information was transferred onto the dataset. This service evaluation was approved by the Trusts’ Quality Improvement team before the project commenced.
Analysis plan
Data were collated for the 459 CYP who engaged in an intervention and their parent or carer. Parent and child data were not matched. Records without at least one paired outcome measure (pre- and post-RCADS and/or SDQ) were defined as incomplete and removed from analysis. The final number of CYP and parents used in each analysis is presented in Tables 3 and 4. Two children under the recommended SDQ administration age, and six under the RCADS administration age were checked in sensitivity analyses and showed no impact on the results so are included in the reported findings. All data were found to be approximately normally distributed; the variance was equal across groups and all data were independent from another. As the above assumptions were met, we proceeded to undertake a parametric analytic approach. Due to the unequal sample sizes between boys and girls on the child-rated SDQ measure, which led to a violation of the assumption of homogeneity of variance-covariance matrices (as indicated by a significant Box’s M-test), this dataset was excluded from the gender analysis. Including it was deemed inappropriate as it would compromise the validity and interpretability of the results. Z-scores were calculated to identify outliers (Z>3). Test run including or excluding outliers showed that they had no impact on the final results, and therefore they have been included in the final report.
Overall sample means and standard deviations of child and parent-reported measures used in the MANOVAs
Child self-reported SDQ scale t-test summary table
*p<.001.
As there were only seven CYP who were recorded as a gender other than girl or boy, we could not reliably include them in the above analyses. t-tests were run on each outcome measure inclusive of all CYP as a sensitivity analysis and results were consistent with those reported in the final analysis. For comparison with similar service evaluations, t-tests were conducted using the RCADS T-scores (corresponding with the raw scores used in this analysis). These results were consistent with the raw score analysis and are included in the Supplementary material. CYP and parent outcome measures were not paired, so only group-level analysis was conducted.
Research question 1
To investigate the overall effectiveness of interventions on the RCADS total scores, we conducted two mixed-model MANOVAs on both child- and parent-scores, with time as the within-subjects factor (pre- and post-scores). For the SDQ overall difficulties and impact scales we conducted two paired sample t-tests.
Research question 2
To investigate the effectiveness of interventions for different anxiety profiles, for the overall sample and individually for girls and boys, we conducted two mixed-model MANOVAs on both CYP self-rated and parent-rated anxiety subscales (SP, PD, SAD, GAD and OCD) with time (pre/post) as the within-subjects factor and gender (girl and boy) as the between-subjects factor. To control for multiplicity the cut-off point for significance was adjusted to account for the four tests using the Bonferroni correction (α=.013).
Research question 3
To investigate whether MHST interventions were equally as effective in both boys and girls, we conducted two mixed-model MANOVA with gender (girl vs boy) as the between-subjects factor, assessing group differences across the child- and parent-rated RCADS total and subscale scores and the parent-rated SDQ total difficulties and impact scales.
Sensitivity analysis
The primary analysis was conducted using a complete case analysis. This was because the online platform used to collect data only allowed completed assessments to be submitted. This therefore resulted in each case either having a full set of outcome measures, or no post-outcome measures. To investigate whether the missing data were related to observed or unobserved variables we investigated any difference in baseline symptom severity between those with complete (pre and post) and incomplete (pre-only) outcome measures. We conducted independent samples t-tests on the RCADS (MDD: 341 complete vs 55 incomplete and Total anxiety: 329 complete vs 65 incomplete) and SDQ (Total difficulties: 177 complete vs 64 incomplete and Impact score: 177 complete vs 64 incomplete). If no differences were found, we concluded that the complete case analysis remained unbiased (Enders, Reference Enders2010).
Cohen’s d was calculated for t-tests to indicate effect size and to determine if it was small (0.3), medium (0.5) or large (0.8) (Cohen, Reference Cohen1988). Partial eta-squared was used to estimate the effect size of the MANOVA with values of 0.01, 0.09 and 0.25 representing small, medium and large effect sizes.
Results
Descriptive statistics for all measures can be found in Tables 3 and 4.
Research question 1: Effectiveness of interventions delivered using core outcome measures
There was a statistically significant main effect for time (pre and post) on the CYP-rated MDD scale, F 1,325=187.04, p <.001; and overall anxiety scale, F 1,325=258.39, p<.001. The effect sizes (ηp 2) were 0.37 and 0.44, respectively, indicating a large effect. This was consistent with parent-reported measures with significant main effects for time on the MDD scale, F 1,130=31.83, p<.001, and overall anxiety scale, F 1,130=48.56, p<.001. These showed medium to large effect sizes (ηp 2 = 0.2 and 0.27, respectively). Scores were significantly lower post-treatment, indicative of improvements in difficulties.
As shown in Table 4, the repeated measures t-test showed significant improvements on both the CYP-rated SDQ total difficulties, t 180=10.41 and impact scales, t 154=7.11 (both p<.001). These showed medium effect sizes (Cohen’s d=0.57–0.77). This was consistent with parent-reported measures, with a significant main effect for time on the SDQ total difficulties scale, F 1,65=19.59, p<.001; and impact scale, F 1,65=18.08, p<.001 both showing medium effect size (ηp 2=0.23 and 0.22, respectively).
Research question 2: Are interventions effective in reducing symptomology across the five anxiety profiles assessed on the RCADS?
There were statistically significant main effects for time (pre and post) for all five subscales: SP, F 1,325=178.42; PD, F 1,325=150.87; SAD, F 1,325=151.53; GAD, F 1,325=188.32; and OCD F 1,325=167.86 (all p<.001). Effect sizes were large (ηp 2=0.32–0.37). Post-intervention scores were all significantly lower than pre-intervention scores (Table 3). This was consistent with parent-rated measures which showed a significant main effect of time across all five anxiety profiles: SP, F 1,126=43.48; PD, F 1,126=25.35; SAD, F 1,126=50.03; GAD, F 1,126=42.19; and OCD F 1,126=22.86 (all p<.001). Effect sizes were medium for the PD (ηp 2=0.17) and OCD (ηp 2=0.15) scales, and large for SP (ηp 2=0.26), SAD (ηp 2=0.28) and GAD (ηp 2=0.25) scale.
Research question 3: Are MHST interventions equally as effective in both boys and girls?
There was a statistically significant main effect of gender on the CYP-rated MDD scale, F 1,325=10.33, p=.001; and overall anxiety scale, F 1,325=24.98, p<.001. Girls showed higher scores pre- and post-intervention with a small effect size, ηp 2=0.03 (MDD) and 0.07 (anxiety). Parent-rated outcome measures showed no significant main effect of gender on any of the core scales: MDD scale, F 1,130=0.46, p=.500; overall anxiety; F 1,130=0.64, p=.425; SDQ total difficulties, F 1,65=0.02, p=.879; or SDQ total impact scale, F 1,65=0.10, p=.756. There was no significant interaction between time (pre–post) and gender on the MDD scale, F 1,325=0.26, p=.613; or overall anxiety scale, F 1,325=2.54, p=.112. This corresponded with the parent-rated measures: MDD, F 1,130=0.17, p=.896; overall anxiety, F 1,130=0.00, p=.992; SDQ total difficulties, F 1,65=0.46, p=.500; SDQ impact scale, F 1,65=1.81, p=.183.
There was a statistically significant main effect of gender on the SP subscale, F 1,325=23.498, p<.001; PD subscale, F 1,325=41.09, p<.001; GAD subscale, F 1,325=9.37, p=.002; and SAD subscale, F 1,325=7.483, p=.007. The effect size for the PD subscale was moderate (ηp 2=0.11) but for all other subscales was small; SP (ηp 2=0.07), SAD (ηp 2=0.03) and GAD (ηp 2=0.01). Girls had significantly higher scores on all scales both pre- and post-intervention. There was no significant main effect of gender for the OCD subscales, F 1,325=2.06, p=.152. There was no significant main effect of gender on parent-reported anxiety subscales: SP, F 1,126=0.53, p=.470; PD, F 1,126= 0.01, p=.925; GAD, F 1,126=0.678, p=.412; SAD, F 1,126=0.693, p=.407; or OCD, F 1,126=0.58, p=.447.
There was a significant interaction between gender and time on the PD subscale; F 1,325=8.65, p=.004, with a small effect size (ηp 2=0.03). Girls showed greater improvement over time. There was no significant interaction between time (pre–post) and gender on any of the other subscales; SP subscale, F 1,325=1.50, p=.221; GAD subscale, F 1,325=1.20, p=.274; SAD subscale, F 1,325=0.032, p=.857; or OCD subscale, F 1,325=1.12, p=.291. There was also no significant interaction between time and gender on any of the parent-rated subscale measures: SP, F 1,126=0.07, p=.798; PD, F 1,126=1.87, p=.174; GAD, F 1,126=0.2, p=.6690; SAD, F 1,126=0.28, p=.599; or OCD, F 1,126=0.67, p=.414.
Sensitivity analysis
When comparing datasets of those who had complete versus incomplete outcome measures, there was no significant difference in baseline scores between the two groups on the RCADS MDD t 394=0.48, p=.961 or total anxiety scale t 392=0.512, p=.609. There was also no significant difference between the two groups on the SDQ total difficulties scale t 239=1.06, p=.291 or impact scale t 239=0.59, p=.558.
Discussion
This project aimed to evaluate the effectiveness of CB-informed interventions delivered by MHST practitioners in treating mild to moderate symptoms of emotional and behavioural difficulties in CYP within a school setting. The results showed significant improvements across all standardised measures on both CYP and parent-reported outcome measures. Specifically, there was a significant reduction in symptoms of anxiety and depression post-intervention across both CYP and parent-rated RCADS scores. Furthermore, we found a significant decrease in CYP’s self-reported and parent-reported total emotional and behavioural difficulties on the SDQ total difficulties scale and the CYP and parent-reported impact of these difficulties significantly decreased following an intervention. All child- and parent-reported outcomes showed medium to large effect sizes for the effectiveness of the intervention over time. We found that girls showed higher levels of anxiety and depression both pre- and post-treatment compared with boys on the RCADS; however, there was no difference in the effectiveness of the interventions, with both groups showing improvement post-treatment on the RCADS. As one of the first studies specifically evaluating the role of MHST practitioners within a school context, these results have important meaning for understanding the effectiveness of this newly developed workforce. Within this MHST within the South East of England, interventions being delivered by MHST practitioners appear to be effective in treating mild to moderate mental health conditions in CYP.
The findings from this evaluation build on the practice-based evaluation conducted by Lockhart and colleagues (Reference Lockhart, Jones and Sopp2021) on the effectiveness of CB-informed interventions delivered by trainee practitioners within schools (EMHPs) and in healthcare settings (by CWPs). Their study found significant improvements in symptom severity and impact of symptoms for CYP with mild to moderate mental health difficulties across both child- and parent-rated measures. However, the sample size used in their study was relatively small (n=109) and therefore the researchers were unable to analyse the effectiveness of interventions delivered by CWPs and EMHPs separately. Turnbull and colleagues’ (Reference Turnbull, Kirk, Lincoln, Peacock and Howey2023) evaluation of the CWP role within a CAMHS service in the North East of England found similar results with a significant reduction in symptomology across all RCADS subscales and composite total scales and significant goal progress. The current evaluation is one of the first studies to evaluate the effectiveness of education-based practitioners delivering interventions within a school context and builds upon previous findings by providing additional evidence for the effectiveness of individual low-intensity CB-informed interventions delivered by MHST practitioners within a school setting.
Although it is well established that CB approaches are effective in treating moderate to severe anxiety and depression in CYP, to date there has been less evidence for its effectiveness in treating mild to moderate difficulties and some contraindicatory evidence from a systematic review which reported weak evidence for its use in reducing sub-clinical anxiety and depression in school-based interventions (Caldwell et al., Reference Caldwell, Davies, Hetrick, Palmer, Caro, Lo ṕez-Lo ṕez, Gunnell, Kidger, Thomas, French, Stockings, Campbell and Welton2019). This review included various psychological, educational and physical interventions, all delivered within education settings, and found little efficacy of any universal or targeted intervention in reducing symptoms of anxiety or depression. They concluded weak evidence for a whole school-based approach, but the studies included had a high risk of bias. Furthermore, this review did not include any study evaluating interventions delivered by EMHPs, who are trained with a tailored education-based CB-informed curriculum, and therefore the review may not be truly representative of the current mental health offer within school settings.
The results of this evaluation are important within the wider context of service delivery, with government policy aiming to promote early access to evidence-based psychological support within community settings as part of a stepped-care approach (Department of Health and Department of Education, 2017). It found that the MHSTs were meeting one of its core functions in delivering effective evidence-based interventions to CYP with mild to moderate difficulties. Although based in just one MHST within the South East of England, it provides an early indication that the rollout of MHSTs is effective in reducing emotional and behavioural difficulties in both boys and girls within a school setting and appears to be a positive addition to the delivery of CYP mental health services.
Strengths and limitations
One of the strengths of this evaluation is the relatively large dataset that was used within the analysis. Demographic information was collated from all individual interventions completed between 2021 and 2022 only excluding those without paired outcome measures or who were discharged to an alternative provision or offered only a group-level intervention. The demographic characteristics of children and young people included in this evaluation reflected primary, secondary and college-age children and covered all individual intervention types that the team delivered. The MHST service aims to provide interventions to CYP with mild to moderate difficulties. In our analysis, we observed 68.5% of young people fall into the non-clinical or borderline range for anxiety and 63.5% for depression. Based on this distribution, it appears the CYP accessing the service are predominantly falling within the mild to moderate severity. The smaller number of young people who fell within the ‘borderline’ category is reflective of the fact that this category has a narrower threshold. A recently published paper reporting the characteristics of young people being referred to two MHSTs in the south of England found that the proportion of females and secondary age CYP being referred appear to reflect the demographics of the local community (Robinson et al., Reference Robinson, Chapman, Orchard, Dixon and John2025). This could suggest that the data included in this paper reflects the overall demographics of CYP being accepted into our service. However, given that a large proportion of the referrals into our service were signposted elsewhere, we cannot comment on how well the data in this study reflect the clinical characteristics of all CYP referred to the service, or whether the findings can be generalised to other MHST services across England.
We found that girls showed higher levels of anxiety and depression both pre- and post-treatment compared with boys on the RCADS, consistent with the wider literature (Baron et al., Reference Baron, Hurn, Ainger, Adlington and Shapiro2021). However, there was no difference in the effectiveness of the interventions, with both groups showing improvement post-treatment on the RCADS. Unfortunately, due to the unequal number of boys and girls on the child-rated SDQ measure, we were unable to analyse the impact of gender over time on the CYP-reported SDQ scales. Furthermore, we acknowledge two limitations within this analysis. Firstly, the data available on gender were based on school reports and was a mixture of sex at birth and gender as identified by young person. This therefore introduces inconsistencies in how gender was measured, with an increased risk of misclassification and therefore limits our ability to accurately analyse gender-specific effects and reduces the clarity and generalisability of our findings. Secondly, we were only able to analyse outcomes for two binary genders (girls vs boys) and we are therefore unable to conclude on the effectiveness of interventions with CYP of another gender. Interestingly, there was no difference found between girls and boys on the parent-reported outcome measures for either the RCADS or SDQ scales. This aligns with previous research that has shown that gender differences in internalising and externalising symptoms tend to be less pronounced in parent-reports than in self-reports, likely due to limited parental insight into internal experiences, gendered expectations, and the subtler ways in which symptoms, particularly in girls, may manifest and go unnoticed by adults which can bias parent reports and flatten real gender differences (Rescorla et al., Reference Rescorla, Achenbach and Ivanova2007; Zahn-Waxler et al., Reference Zahn-Waxler, Shirtcliff and Marceau2008).
The present study has been able to build on the findings in previous work by also including the analysis of parent reports, and the anxiety subtypes. Given that existing evidence suggests that different anxiety presentations respond differently to the various interventions, it was important to include these tests within this evaluation to obtain a better understanding of how baseline presentation impacted the response to the intervention (Fonagy et al., Reference Fonagy, Cottrell, Phillips, Bevington, Glaser and Allison2015). Consistent with the findings of Turnbull and colleagues (Reference Turnbull, Kirk, Lincoln, Peacock and Howey2023), we found significant improvements across all anxiety subscales with effect sizes ranging from medium to large. There is a need to undertake a more granular analysis with a larger dataset to consider which of the low-intensity interventions are most effective for a particular symptom pattern. Unfortunately, due to a lack of subscale reliability for the SDQ measure, we were concerned that analysis of individual subscales would be unreliable, and we are therefore unable to comment on the comparative impact of CB interventions on both internalising and externalising difficulties.
Despite encouraging findings there are some additional limitations to this evaluation that need to be held in mind when interpreting the results. With only pre- and post-data, we are not able to definitively say whether effects were due to the interventions as this design lacks a comparison group. We are also unsure of the sustainability of these outcomes. Going forward, an important part of the overall evaluation of this service will be collecting follow-up data to establish if the benefits are maintained over time and to review referral and attrition data. As the service is still in its infancy, this type of data was not routinely collected.
Lastly, we did not have data on who delivered each intervention and whether this was a closely supervised EMHP trainee or a fully qualified member of staff. This therefore introduces uncertainty around treatment fidelity and limits the ability to examine potential moderating effects of provider expertise on intervention outcomes. There is a growing evidence base showing that IAPT CBT trainees obtain excellent client outcomes during their training (Forand et al., Reference Forand, Evans, Haglin and Fishman2011; Öst et al., Reference Öst, Karlstedt and Widén2012), but future research would benefit from comparing outcomes between trainee and qualified EMHPs. Furthermore, the data included in this study reflect referrals received between January 2021 and December 2022 where some COVID-19 restrictions were still in place. Our findings may have limited generalisability to post-pandemic times and should be interpreted in light of this.
Clinical and research implications
The findings of this evaluation have several implications for future research, policy and clinical practice. This study provides evidence for the effectiveness of CB-informed interventions delivered by MHST practitioners in alleviating mild to moderate symptoms of common mental health difficulties and therefore advocates for the expansion of the MHST workforce. It is critical that further research is conducted into the sustainability of these outcomes as this is essential in evaluating whether MHSTs are reducing the need for long-term mental health provision and builds an economic argument for continuing investment. Whilst the interventions were effective for both boys and girls, the fact that the girl’s post-intervention levels remained higher warrants further investigation alongside considering which particular interventions are most effective for various expressions of distress. We also acknowledge that our sample had a higher proportion of girls, which may have limited the strength of our conclusions about treatment response in boys. While the interaction between gender and treatment outcome was not statistically significant, the relatively smaller sample of boys could have contributed to reduced statistical power to detect such an effect. Future studies should also include a comparison of outcomes based on practitioner experience. This study demonstrated the variety of techniques used by EMHPs in interventions, with cognitive restructuring being the most used technique, which could be due to its applicability across several types of difficulties. Furthermore, the consistency in CYP and parent-reported measures suggests that parents and carers can be a helpful source of information when EMHPs are looking to evaluate progress.
Conclusion
This evaluation found that cognitive behavioural informed interventions delivered by MHST practitioners within one service across four MHST sites in the South East of England significantly reduced symptom severity and impact of emotional and behavioural difficulties in children and young people with mild to moderate difficulties. This supports the rollout of MHSTs and the effectiveness of the EMHP training role in providing early intervention. Further research is needed to see if these findings are shown consistently across other MHST services. With the current mental health crisis that children and young people are facing this is a promising outcome for the transformation of services and indicates progress towards a goal of providing all children and young people with the best start in life.
Key practice points
-
(1) MHSTs appear effective in reducing symptoms of emotional and behavioural difficulties in children with mild to moderate difficulties, supporting their role as a low-intensity, accessible intervention.
-
(2) MHST-delivered early interventions can demonstrate symptom reductions even in populations that fall below clinical thresholds.
-
(3) MHST-delivered interventions appear to be equally effective in reducing both emotional and behavioural difficulties in both boys and girls.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1754470X25100111
Data availability statement
The data used within this report were collected as part of routine practice data collection and are not publicly available.
Acknowledgements
The authors wish to thank the research team who supported with gathering and sorting the data used within this service evaluation and the MHST practitioners who delivered the interventions.
Author contributions
Nicole Bebbington: Conceptualization (supporting), Data curation (supporting), Formal analysis (lead), Methodology (equal), Project administration (equal), Validation (equal), Visualization (lead), Writing - original draft (lead), Writing - review & editing (equal); Emilia Robinson: Conceptualization (equal), Data curation (lead), Formal analysis (supporting), Methodology (supporting), Project administration (equal), Resources (equal), Visualization (equal), Writing - review & editing (equal); Faith Orchard: Conceptualization (supporting), Formal analysis (supporting), Methodology (equal), Project administration (equal), Supervision (lead), Validation (equal), Visualization (equal), Writing - review & editing (equal); Chloe Chapman: Data curation (supporting), Project administration (supporting), Visualization (supporting); Clare Dixon: Conceptualization (equal), Methodology (equal), Project administration (supporting), Supervision (equal), Visualization (equal), Writing - review & editing (equal); Mary John: Conceptualization (equal), Formal analysis (supporting), Methodology (equal), Project administration (equal), Supervision (equal), Validation (supporting), Visualization (equal), Writing - review & editing (equal).
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
The authors declare none.
Ethical standards
The authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the British Association for Behavioural and Cognitive Psychotherapies (BABCP) and British Psychological Society (BPS). The SPFT research and audit department advised that ethical approval was not required for this report as it constituted a service evaluation.
Open access
Comments
No Comments have been published for this article.