Clinical setting
Currently, autism presents three challenges to service delivery. First, although Kanner published the paper defining the core syndrome in 1943, the boundaries of the disorder remain contentious and subject to regular updates that may significantly alter classification rules identifying affected populations (Reference McPartland, Reichow and VolkmarMcPartland 2012).
Second, and independent of this, there has been a vast improvement in the detection of autistic disorders, which has contributed to the progressive identification of ever greater proportions of the population, worldwide (Reference Hill, Zuckerman, Fombonne and Robinson-AgramonteHill 2015), from early figures of between 2 and 4 affected individuals per 10 000, to the current estimate of 6.6 per 1000, while rates of the broader classification of autism spectrum disorder (ASD) are around 1 per 100 (Reference Baird, Simonoff and PicklesBaird 2006). Furthermore, in England at least, part of this increase is attributable to an increase in autistic symptomatology as a secular trend (Reference Russell, Collishaw and GoldingRussell 2015).
Third, drug treatments for the core symptoms of autism have recently been found to have no measurable therapeutic benefit, though they remain useful for comorbid psychiatric problems (National Institute for Health and Care Excellence 2016). The entire spectrum of autistic disorders impair the quality of life of those suffering (Reference Ikeda, Hinckson and KrägelohIkeda 2014) and attract significant economic costs (Reference Lavelle, Weinstein and NewhouseLavelle 2014). So, commissioners face a rising tide of a serious, chronic, disabling and costly disorder, with only psychological treatments available to combat its core features. Effective early interventions for ASD are therefore potentially invaluable, and the review featured in this month's Cochrane Corner (Reference Oono, Honey and McConachieOono 2013) provides useful information about the best-known class of such interventions, parent-mediated interventions. Oono et al 's key conclusions are that outcome measures that could be understood as directly measuring the intervention (the authors call these ‘proximal’ measures), such as parent– child interaction measures, showed large effect sizes, but outcomes that estimate the impact of the intervention on ASD reported smaller or absent effect sizes.
Interpreting the studies under review
Making sense of effect size
In general terms, an effect size tries to compute the magnitude of an effect, rather than its statistical significance. There are several different types of effect size, but the review reported three. If all studies used the same continuous measure, the magnitude of the overall effect was simply reported as the mean difference found (averaged across studies), with the importance of the difference interpreted directly from what is known about the measure. When studies used different measures, the differences were converted to a common scale by expressing them in units of the measures’ standard deviation (s.d.). When standardised differences are so calculated, by convention, an average of 0 up to 0.3 s.d. (as the review reported for changes in ASD severity) is considered small, ones around 0.5 (as the review found for parent– child shared attention) are called moderate, while those of 0.8 or above (as the review found for parent–child synchrony) are large. However, as discussed below, these evaluations can vary by context. Sometimes, as in this review, + and – signs are used to indicate whether the effect size refers to, respectively, an increase or decrease in the studies’ measures. If numerical scales were not used in a study, the review reported odds ratios. With this effect size, a value of 1 means that there is no difference between the study groups; values less than 1 mean that the odds are greater that the measured characteristic will be found in the reference group than in the comparison groups(s), whereas values greater than 1 mean the reverse.
Bias and its impact
Treatments intended as ‘early interventions’ present particular challenges in assessing their outcomes. They seek to achieve benefit by altering developmental trajectories, or intervening while the disorder is not fully and finally expressed. The interest in developmental trajectories implies prolonged follow-up across a wide range of possible outcomes, which challenges the conventional randomised controlled trial (RCT) design of having a primary outcome, whose change is measured across the intervention to a clear end point. However, if one simply reports the ‘significant’ improvements that one finds among the many more non-significant outcomes, the result may be outcome measurement bias, as it is rather like shooting an arrow into a tree and then painting the target round it. Intervening while the disorder is still developing implies broad and fluid diagnostic criteria. This increases the importance of careful randomisation, as randomisation is the only way (with sufficient numbers) that unobserved variables can be equivalently assigned to treatment and control groups, and loosening criteria increases the risk of including such variables. So, despite the caveat about end points, RCTs remain the only certain means to infer that an observed change is due to the intervention under study. As this was the focus of their review, Oono et al were bound to restrict their coverage to RCTs.
‘Parent-mediated’ means teaching parents to deliver a treatment. There are obvious advantages to this – almost all parents love their children, are committed to them and will work incredibly hard to do the best for them. As human beings, parents are also subject to all our frailties. Individual differences between parents can be managed by randomisation, as unmeasured variables. However, parents will be motivated (or demotivated) by human perceptual biases, and so these have the potential to confound their interventions. It may be demotivating to find that you are in a control group, particularly when you have signed a consent form which explains, in detail, why the researchers feel their experimental intervention will benefit your child. Therefore, proper masking (‘blinding’) as to which arm of the study participants are in may be as important as effective randomisation, but it is very hard to create ‘control’ conditions that engage parents as much as ‘treatment’ conditions. In these studies, which compared the intervention with various flavours of ‘treatment as usual’, it was impossible, for either therapists or patients. It is a compliment to both parents and researchers that the reviewers found this bias, in practice, to be insignificant. Like parents, researchers, particularly if also clinicians, want to give their patients the best chance. In the context of an RCT, this can lead to allocation bias, through greater concern for worrisome or well-known patients, particularly if assigned to the control group. Therefore, allocation masking, as well as masking of assessors (who otherwise will rate knowing which arm of the study the patient was in) are also important. Finally, participants drop out of studies. As dropping out can be related to the treatment (or lack of treatment) received, correct management of the concomitant data losses is essential.
Oono et al found that 16 out of the 17 studies they included were at risk of biases other than lack of masking for the parents. An example of the potential impact of such biases is reported in the review's assessment of the outcome of the interventions on communication, shown in Fig. 5 of the review. The review reports similar, positive results from Reference Aldred, Green and AdamsAldred et al (2004) and Reference Dawson, Rogers and MunsonDawson et al (2010), but no measurable change was found by Reference Green, Charman and McConachieGreen et al (2010). However, the Aldred et al and Green et al studies were by the same team, using similar methodology in the same region of the UK. The Dawson et al study was by an American group, using a different (though related) therapeutic approach, delivered at much higher intensity.
This unexpected pattern of differences can be better understood when the risk of bias of the three studies is considered: the reviewers reported that the Dawson et al study was most at risk of bias, and had the most positive outcome. Next came Aldred et al, while Green et al, with the least positive outcome, also had the lowest overall risk of bias. This example also reveals a general problem in interpreting meta-analyses: the impact of bias may also affect the summary result; in this case, the overall effect was positive but non-significant.
However, being at risk of bias does not necessarily mean that bias has occurred. This is apparent in the review's consideration of the interventions’ impact on autism severity, reported in Fig. 6 of the review. Here, despite the quantitative and qualitative variation across the studies’ biases, the estimate (an effect size of 0.3) was fairly consistent for all. Such close agreement suggests that, for this result, the studies’ individual biases did not contribute much to the observed outcome.
Seeing past the methodology
Not being certain is not the same as knowing nothing at all, and we have already seen that the need for early intervention in ASD is urgent. Irrespective of bias or methodology, RCTs of parent-mediated early intervention identified a significant reduction in autism severity, with an effect size of 0.3. Such an effect size is typically described as ‘small’, and in absolute terms it is: these interventions provided, on average, an additional 8% chance of improvement in autism compared with treatment as usual (Reference CoeCoe 2002). Although this looks grim from a clinical perspective, a public health perspective may give more grounds for optimism, given the general association between symptom level and population prevalence (Reference Veerman, Dowrick and Ayuso-MateosVeerman 2009; Reference Goodman and GoodmanGoodman 2011). Interpretation of the relationship between population symptomatology and prevalence for autism is complex: Reference Goodman, Heiervang and Fleitlich-BilykGoodman et al (2012) point out its cultural variability, and the detection of autism in Reference Goodman and GoodmanGoodman & Goodman (2011) is partly based on comorbidities, rather than autism's core symptoms (Reference Goodman, Ford and SimmonsGoodman 2000). However, with the UK population prevalence of ASD being 1%, it is possible to calculate from the association reported in Reference Goodman and GoodmanGoodman & Goodman (2011) that, applied as population interventions, parent-mediated early interventions offer the possibility of reducing population prevalence by up to 40%, to 0.6%. Such a dramatic change additionally assumes that all such cases are detected, and that the association between symptomatic improvement and prevalence change will be similar to that found in the gobal population, but even fractions of this figure would produce worthwhile prevalence reductions.
High- v. low-frequency interventions
The added value of high-intensity interventions, typically involving more than 20 hours of therapeutic time per week (Reference Maglione, Gans and DasMaglione 2012) is currently an important issue in delivering parent-mediated interventions for autism. Oono et al 's review was unable to address this, although their finding of reduction in intensity of autism was similar for high- and low-intensity interventions. Older reviews (e.g. Reference Reichow and WoleryReichow 2009) support the value of high-intensity interventions, and a more recent review of reviews (Reference ReichowReichow 2012) suggested effect sizes for intensive interventions of 0.38–1.19 for IQ and 0.30–1.09 for adaptive behaviour. However, the studies that it cites are not all RCTs and, as we have seen above, their high effect sizes may be due to bias, rather than treatment efficacy. A review by Reference Howlin, Magiati and CharmanHowlin et al (2009) observed that the benefits were not uniformly distributed across the ASD population, and that other, lower-intensity interventions had, on occasion, delivered equivalent results. After an extensive review of both RCT and other evidence using a combination of systematic evidence synthesis, expert and service-user voting, an American Technical Expert Panel accepted that higher intensity and longer duration led, on average, to better outcomes (Reference Maglione, Gans and DasMaglione 2012). This conclusion was based on a meta-analysis (Reference Virues-OrtegaVirues-Ortega 2010) that included a much wider range of study types than the RCTs in Oono et al 's review. Intriguingly, despite including studies with a higher risk of bias, the quality of evidence in Maglione et al 's review was rated as ‘moderate’ using a modification of the GRADE scoring system employed by Oono et al, who reported generally ‘low’ quality for the RCT evidence.
The value of component, accessory and intermediate outcomes
This commentary has focused on the use of parent-mediated early interventions for the prevention and mitigation of ASD. However, Oono et al 's review also reported on some specific components of the autistic syndrome (e.g. language; accessory difficulties such as parental stress; and outcomes, especially parent–child interaction) whose optimisation is considered an important mitigator of the developmental trajectory of ASD (Reference DawsonDawson 2008). Parent–child interaction had the largest effect sizes of all the outcome measures reported, supporting the construct validity of the interventions. For example, the 0.9 effect size for parent–child synchrony suggests that these interventions offer an additional 24% chance of improvement in this area. There is surprisingly little research on the association between parent–child synchrony and quality of life, but the importance of attunement in parenting programmes more generally suggests that this is valuable in its own right, as well as being potentially protective in the developmental trajectory of ASD.
Conclusions
The headline result is that parent-mediated early intervention probably does ameliorate the core symptoms of autism, although the magnitude of the effect is too small to recommend its use in routine secondary care settings. However, a case can be made for employing low-intensity interventions in suitably screened populations, in an attempt to reduce population prevalence. While the overall case for the added benefit of high-intensity interventions is moot, it seems clear that they do work especially well in some cases, for example in young children in whom significant autistic symptomatology is present alongside good linguistic and cognitive ability (Reference Magiati, Charman and HowlinMagiati 2007; Reference Smith, Klorman and MruzekSmith 2015). There is thus a case for ensuring that some intensive interventions are available for those who are most likely to benefit from them.
eLetters
No eLetters have been published for this article.