Child and adolescent mental health services (CAMHS) in the UK’s National Health Service (NHS) have never been well-enough funded to provide a service to all of the 10% of children in the population who at any one time have mental health difficulties. Most NHS mental health services for children and adolescents have therefore historically provided treatment mainly for those young people who have severe and long-lasting mental health difficulties. With the publication of the Health Advisory Service’s document Together We Stand, 1 it became widely accepted that the core business of specialist CAMHS teams (also known in the document as tier 3 teams) was to see children with these relatively complex mental health disorders.
The period between 1996 and 2006 saw development and expansion for many CAMHS in England. Investment from ring-fenced local authority grants brought about the development of new aspects of comprehensive CAMHS (e.g. primary mental health workers and mental health workers in youth offending teams). New treatment approaches, such as eye movement desensitisation and reprocessing (EMDR) were emerging during this time. Reference Cottrell and Kraam2
Epidemiological studies in the UK show that rates of emotional and behavioural disorders in children and young people had increased in this decade. Reference Collishaw, Maughan, Goodman and Pickles3 Family structures and the socioeconomic situation faced by young people in the UK also became more complicated and less advantageous to young people during this time. Reference Collishaw, Maughan, Goodman and Pickles3 Perhaps both of these trends contributed to the findings of a 2007 United Nations Children’s Fund report on child well-being in countries belonging to the Organisation for Economic Co-operation and Development (OECD), in which the UK was in the bottom third of the rankings for 5 of the 6 dimensions of childhood well-being in the 21 nations of the industrial world. 4
From 2000, all services within the NHS were managed in accordance with the vision and targets of The NHS Plan. 5 Child and adolescent mental health services came under increasing pressure to deliver the Plan’s imperative of a 3 months’ maximum wait for out-patient appointments by 2005. In 2006, staff in a combined tier 2 and tier 3 CAMHS serving the county of Lincolnshire, UK (population 700 000), raised the concern in both clinical and managerial discussions that cases being seen in the service appeared to be getting increasingly more complex. Clinicians said it had become difficult to identify ‘straightforward’ cases for trainees to assess and manage. Managers were concerned that complicated cases needed lengthy interventions from multiple members of the clinical teams and that as a result the service’s capacity to take on new work was reduced.
This study, with data collected between 2008 and 2010, aimed to explore staff opinions about change in case complexity and to compare this with objective change in case complexity in cases seen in the decade studied.
Defining case complexity
Complexity is a word commonly used in descriptions of specialist CAMHS. For example, it appears 20 times in Standard Nine of the National Service Framework for Children, Young People and Maternity Services 6 and 36 times in the more recent national independent CAMHS review. 7 However, in neither of these documents is complexity defined. Our understanding from the literature about the definition of case complexity as relevant to CAMHS is that it usually indicates the sum of a child’s clinical and psychosocial needs during their care from CAMHS. The nature of the CAMHS intervention required, the outcome of treatments offered and the demands on the workforce in delivering care are all likely to be influenced by case complexity.
Case complexity and required CAMHS interventions
Children with complex needs are more likely to receive care from more than one professional within a CAMHS team, either sequentially or simultaneously, and as such the child’s care will usually be within tier 3 CAMHS. 1 Paradoxically, some children with complex difficulties may not receive a service from CAMHS at all: poorly resourced CAMHS teams unable to offer a comprehensive service may reject these children overtly at the point of referral, or fail to adequately address their needs by providing services which prove inaccessible, unhelpful or too brief.
Case complexity and treatment outcomes
Outcome measures are becoming a greater priority for CAMHS as services come under scrutiny from new commissioners in 2013 onwards. 8 Clinicians are aware that the outcomes measurable from a CAMHS clinic will inevitably be influenced to some extent by the characteristics of the children referred to the clinic. At first sight, the baseline measures of clinical need of children referred to a CAMHS clinic could be seen as being largely defined by the severity of the children’s index mental health problems. However, the nature of work in CAMHS whereby the child is assessed and managed in the context of their family, their school and their community means that the magnitude of difficulty facing a CAMHS clinician when trying to offer appropriate and effective interventions is influenced not only by the clinical presentation of the child, but also by the child’s psychosocial situation. We suggest that case complexity is likely to influence the time taken for a treatment package to successfully resolve mental health difficulties. Furthermore, if one or more of the adverse factors in a child’s life contributing to case complexity acts as a perpetuating factor in the aetiology of their mental health difficulties, then these children may be less likely to respond to the interventions attempted by CAMHS.
Case complexity and demands on the CAMHS workforce
It seems likely that case complexity has an impact on clinicians’ professional satisfaction and stress. When cases are complex, staff are likely to face more uncertainty in understanding and devising management plans for the multifactorial difficulties faced by these children and families. As suggested earlier, the prospect of bringing about change is more likely to be limited. Clinicians may be uncertain about which are the best interventions for children with complex needs because the clinical research underpinning evidence-based practice is largely conducted in samples of children with well-defined single difficulties. We suggest that these obstacles to successful intervention can be a source of stress for staff and may lead to feelings of decreased professional competency. When cases are very complex, assessments take longer and clinicians are likely to spend more time in multi-agency case conferences and other non-direct clinical activity, potentially detracting from time available for direct clinical contact. The number of cases carried by a practitioner at any one time will therefore be inevitably smaller when the cases are complex. Complexity of care and multifactorial intensity has been included in a model to examine variability in nursing workload. Reference O'Brien-Pallas, Irvine, Peereboom and Murray9
Method
This study received approval from the North Nottinghamshire Regional Ethical Committee. There were three research questions. First, to establish whether or not CAMHS staff felt there had been a change in case complexity over the past 10 years. Second, to measure change in case complexity objectively by scoring case notes using a case complexity rating scale. And third, to evaluate similarities or differences between the subjective and objective ratings of case complexity.
Setting
Lincolnshire is one of the largest and most sparsely populated counties in England, with a mixture of urban and rural communities. In 1999, Lincolnshire had 16 wards in the bottom 1000 of wards ranked by the Child Poverty Index, a subset of Income Deprivation which formed one of eight domains contributing to an Index of Local Deprivation 1999. 15 Between 1991 and 2001, the population increased faster than in any other county in England but the proportion of children and young people reduced. The number of Black and ethnic minority individuals remained low compared with national averages. 16
During the decade over which the study was run, CAMHS in Lincolnshire had four CAMHS teams providing what would now be termed combined tier 2 and tier 3 services, and an in-patient unit with 12 beds and 4 day-patient places. In the national CAMHS mapping exercise undertaken in 2003, Lincolnshire’s region (Trent) had the 11th lowest expenditure on CAMHS of the 28 strategic health authorities, and more new cases were waiting more than 26 weeks for a first appointment in Trent than in any other authority. Reference Glover, Barnes, Dean, Hartley and Wistow17 Our workforce of 43 whole-time equivalent staff in 2004 was judged to represent only two-thirds of the number of staff actually required for the size of the population. 18
Local CAMHS service developments between 1996 and 2006 did not increase the numbers of staff working at tier 3, but led to some expansion of the tier 2 workforce with the recruitment of 12 primary mental health workers, 3 CAMHS nurse specialists working with the youth offending service and two 2 providing CAMHS in-reach to the local authority’s children’s secure unit. The service’s management structure and personnel did, however, change markedly over the decade, with a merger of two mental health trusts within the county and the division of one of the large teams into three.
Measuring case complexity
There are two instruments described in the literature designed to measure case complexity in CAMHS.
The Paddington Complexity Scale gathers 16 items of information across 3 domains (psychiatric, physical/developmental and environmental) and generates a total score and two main subscores. Reference Yates, Garralda and Higginson10 This scale has been validated in comparison with the child version of the Health of the Nation Outcome Scales. Reference Gowers, Harrington, Whitton, Lelliott, Beevor and Wing11 To use the Paddington Complexity Scale, information recorded in the case notes needs to include psychiatric diagnoses and a multi-axial classification of problems.
The second instrument is the Pearce Case Complexity Scale (PCCS). It scores six factors (i.e. extent of comorbidity, extent of psychosocial problems, presence of legal issues in a child’s life, the number of agencies involved with a child, any past unsuccessful treatment, and the extent of the impact of the presenting problems on a child’s life) and generates a total score. Reference Pearce12 We understand the scale was developed from empirical thinking in response to service needs. It has been used for research into waiting times. Reference Foreman and Hanna13 Although Byrne Reference Byrne14 found that the PCCS did not predict treatment outcome well, the objective of our study was not to consider correlation between case complexity and outcome. The scale has face validity for the purpose of grading complexity based on clinical and social factors but it has not been validated in other respects.
Despite these limitations, we chose to use the PCCS in this study for practical reasons. It focuses more on social factors than the Paddington Complexity Scale. Although the Paddington Complexity Scale has more robust psychometric properties, it proved unsuitable for the purposes of this study because a high proportion of CAMHS staff working in multidisciplinary teams are non-medical and do not routinely record diagnoses or multi-axial formulations in case notes. Therefore retrospective data collection using the categories of the Paddington Complexity Scale would not have been possible.
Electronic databases of names of children referred to Lincolnshire CAMHS in 1996, 1998, 2000, 2002, 2004 and 2006 were accessed and an equal number of cases from each year were selected using random number tables. Three experienced child and adolescent psychiatrists independently scored each of the sets of case notes using the PCCS. We predicted that ratings of case complexity would differ over the course of an intervention from CAMHS. Judgements about the complexity of a case are likely to increase from the point of referral through the assessment process as the child’s story unfolds. Once treatment is in progress, some factors contributing to case complexity may be resolved and case complexity may therefore reduce. In this study we rated completed episodes of care using all information from the whole duration of treatment. A factor contributing to case complexity was rated as present if it occurred at any point during the child’s involvement with the service.
To assess subjective views of complexity, we devised a short staff opinions questionnaire (available from the authors on request) based on items from the PCCS. This questionnaire asked staff to indicate their opinion about whether case complexity had changed in the time they had worked in Lincolnshire CAMHS. It also asked them to rate their perceptions of change in five of the six dimensions of complexity used in the PCCS. The item of the PCCS not included in the staff questionnaire made reference to individual patients having had previously unsuccessful treatment from CAMHS; we decided that the meaning of this question could not easily be conveyed in the staff questionnaire and so we omitted it. Responses were collected using a 5-point scale. Information about length of time worked in Lincolnshire CAMHS was also collected. All the clinicians in the community CAMHS teams were invited to complete this survey in 2006.
Statistical methods
The sample size for the survey of staff opinions was constrained by the number of employees in the specialist CAMHS community teams. All the clinicians in the community CAMHS teams (n = 60) were invited to complete this survey. The responses to the staff opinions questionnaire are presented as percentages with 95% confidence intervals. Relationships between responses are assessed using Spearman’s rank correlation coefficient.
Scores on the PCCS range from 0 (very simple case) to 12 (highly complex case). Before data collection, the researchers estimated that a 3-point difference in PCCS scores would reflect a clinically significant difference in case complexity. There was no information available on the variability of the scores, so schematic distributions were used to estimate standard deviation. Reference Deming19 Schematic distributions are methods of estimating standard deviation based on the range and likely distribution of data. The standard deviation of the PCCS scores was estimated to be 2.83. Assuming a 5% significance level and 80% power, 15 case-note reviews were needed at both 10 years and 1 year to detect a difference of 3 points. The data are likely to be skewed so the sample size was adjusted using an asymptotic relative efficiency of 0.867, Reference Landis and Koch20 giving a total of 36 case notes, 18 at each time point. To give a more sensitive assessment of change in complexity over time, it was decided to sample 18 case notes at each of the 6 time points between 1996 and 2006, giving a total of 108 case notes. The mean PCCS scores at 10 years and 1 year were compared using an independent groups t-test. The assumptions underlying the use of the t-test were assessed using the Kolmogorov-Smirnov statistic and Levene’s test for equality of variances.
For the purpose of assessing the interrater reliability of the PCCS, case notes were rated independently by three child and adolescent psychiatrists with at least 8 years’ experience in the specialty. Interrater reliability was estimated using intraclass correlation coefficients. Intraclass correlation coefficients can be considered as an average correlation across all raters; an intraclass correlation coefficient of 1.0 would indicate perfect reliability and an intraclass correlation coefficient of 0 indicates reliability no better than chance. The mean of the scores given by the three raters was used for the analysis and presented along with 95% confidence intervals for each time point.
We give a verbal description of the comparison between the subjective and objective data on case complexity. It was not possible to analyse this comparison statistically.
Results
Staff opinions questionnaire
Two-thirds (46/60, 77%) of clinicians returned completed questionnaires. They showed that 78% of staff thought that case complexity had increased during their professional lives in Lincolnshire (95% CI 64 to 88%); no one thought that cases had become less complex.
More than half of respondents (52%) had worked in Lincolnshire CAMHS for more than 5 years, and nearly half (46%) had worked there for between 1 and 5 years. There was no clear relationship between perceived increases in case complexity and length of service in Lincolnshire CAMHS (Table 1), Spearman’s correlation 0.25 (P = 0.1).
When staff were asked about changes in each of the five factors contained in the PCCS, 68% thought that psychosocial problems had increased, 65% indicated that they thought that comorbidity of cases had increased and 59% thought more agencies were becoming involved over the years with the children they saw. Less than half the staff group thought that the presence of legal issues or the disability caused by presenting problems had increased in their cases.
Interrater reliability of the PCCS
The intraclass coefficients for the total and dimension scores across all three raters using the PCCS are shown in Table 2. Although the reliability for some individual factors is not high, the intraclass coefficient for the total score is 0.61 (95% CI 0.51 to 0.70), which is good and the lower limit of the confidence interval falls to a value which is still moderate. Reference Randles and Wolfe21
Clinicians’ perceived change in case complexity and their length of service
| Time worked, years | Clinicians saying complexity has increased, % (n) |
|---|---|
| <1 | 0 (0) |
| 1-5 | 39 (18) |
| 6-10 | 15 (7) |
| 11-15 | 9 (4) |
| > 15 | 15 (7) |
Pearce Case Complexity Scale intraclass correlation coefficients (ICCs) for the total and dimension scores across all three raters
| Scale | ICC (95% CI) |
|---|---|
| Total | 0.61 (0.51-0.70) |
| Comorbidity | 0.49 (0.38-0.60) |
| Psychosocial problems | 0.48 (0.37-0.59) |
| Legal issues | 0.58 (0.48-0.67) |
| Other agencies | 0.57 (0.47-0.66) |
| Unsuccessful treatment | 0.03 (0.00-0.20) |
| Disability | 0.24 (0.12-0.36) |
Changes in objectively rated case complexity over time
In this sample, there was a mean decrease in the PCCS total score of 0.5 (95% CI −0.9 to 2) from 1996 to 2006. A t-test comparing the mean total scores at these two time points gives t = 0.74, d.f. = 31, P = 0.5. When looking at all time points, there is no obvious trend in average total scores over time (Table 3); Spearman’s correlation 0.04, P = 0.7.
When changes in the individual factors of the PCCS are considered, scores in 1996 and 2006 were the same for legal issues, number of other agencies involved, previous unsuccessful treatment and extent of disability. The degree of comorbidity and the extent of psychosocial factors showed a decrease in the mean PCCS scores between 1996 and 2006.
Comparing subjective and objective data about changes in case complexity
The view of the majority of the staff group that case complexity had increased over time was not supported by the objective rating of case notes. There were no areas of similarity between these two estimates of change in case complexity over 10 years. In fact, over 60% of staff judged the degree of comorbidity of cases and the extent of psychosocial factors to have increased, whereas the PCCS rating of these two factors was found to have decreased over time.
Discussion
The results of this study are unexpected. We shared the views of our colleagues that cases were getting more complex, and our views created an assumed bias in the study. But in spite of this bias we found no statistically significant change in case complexity ratings over a 10-year period.
There are a number of possible reasons for this finding. The construct validity and sensitivity of the PCCS has not been established and this may have some bearing on our finding. However, the scale appeared to have good face validity and the interrater reliability it demonstrated in this study is favourable. Further work is needed to examine the psychometric properties of the PCCS and establish the extent and limitations of its usefulness. This should not detract from the possibility that case complexity in CAMHS has indeed remained unchanged over the years. If we accept that the objective measure of no change in case complexity over time may be valid, then the responses of CAMHS staff to the questionnaire may reflect a view that their working lives had become more pressured due to factors separate from the complexity of the cases they were seeing. We suggest that anxiety in clinicians may have resulted from the impact of a trust merger, frequent management restructuring, the 12-week waiting time directive and, in a decade of widening income inequality, Reference Wilkinson and Pickett22 difficulties arising when medical models are used to improve increasing social stress in children. Staff responses to the questionnaire may have reflected their general sense of dissatisfaction with their working lives. During the time period examined in this study, staff morale in the NHS was generally low. 23
Pearce Case Complexity Scale average total scores 1996-2006
| Year | Mean (95% CI) | s.d. (range) |
|---|---|---|
| 1996 | 3.0 (1.8-4.2) | 2.50 (0.0-8.0) |
| 1998 | 2.5 (1.7-3.3) | 1.66 (0.0-6.3) |
| 2000 | 3.1 (2.4-3.8) | 1.56 (1.0-6.0) |
| 2002 | 3.7 (2.7-4.8) | 2.10 (1.0-7.0) |
| 2004 | 3.7 (2.3-5.1) | 2.75 (0.0-9.3) |
| 2006 | 2.5 (1.6-3.4) | 1.78 (0.3-7.0) |
As the national austerity agenda continues to have impact on the population and service provision across all agencies, NHS staff have to constantly deal with the lack of fit between the needs of their patients and the wider health and social care system. Reference Ballatt and Campling24 Further work is needed to explore the factors which may influence clinicians’ judgements about the complexity of their clinical work. Decisions about case complexity will influence how tariffs of patient care are decided in the payment by results commissioning process when it is introduced in CAMHS. This study suggests that staff anxiety and dissatisfaction may influence this decision-making.
Acknowledgements
We thank Dr Salman Zafar and Dr Guthran Ullah for their help with data collection.
Open access
eLetters
No eLetters have been published for this article.