Autism may not be an excessively rare disorder (Reference Volkmar, Lord and BaileyVolkmar et al, 2004). Rather, it could represent the extreme of a quantitative distribution of autistic traits that are present in the general population (e.g. Reference Spiker, Lotspeich and DimiceliSpiker et al, 2002; Reference Constantino, Davis and ToddConstantino & Todd, 2003). Surveys of large unselected samples are required to test and refine this hypothesis. Currently available instruments designed to measure autistic traits suffer limitations that render them unsuitable for large-scale surveys: poor sensitivity (Reference South, Williams and McMahonSouth et al, 2002), little evidence on validity (Reference Ehlers, Gillberg and WingEhlers et al, 1999; Reference Scott, Baron-Cohen and BoltonScott et al, 2002; Reference Cohen, Schmidt-Lackner and RomanczykCohen et al, 2003), applicability to a limited age range of children (Reference Charman, Baron-Cohen and BairdCharman et al, 2001) and time (up to 20 min) taken for completion (Reference Berument, Rutter and LordBerument et al, 1999; Reference Constantino and ToddConstantino & Todd, 2003). We predicted that the Social and Communication Disorders Checklist (SCDC) (Reference Skuse, James and BishopSkuse et al, 1997) would fulfil the need for a brief, simple instrument providing a reliable and valid dimensional measure of heritable autistic traits in children with no learning disability.
METHOD
Informed consent was obtained from all participants in accordance with guidelines from the appropriate hospital ethics committees. Three independent samples participated in heritability, reliability and validity studies. The heritability study was conducted using the Cardiff Study of All Wales and North of England Twins, which is a population-based twin study of all twin births in Wales and Greater Manchester. Data on the heritability of the SCDC have been previously published (Reference Scourfield, Martin and LewisScourfield et al, 1999) but here are re-analysed to examine gender differences in more detail. It is well established that autism is a highly heritable disorder, so any instrument purporting to measure autistic traits should itself show high heritability (however, demonstrating high heritability does not prove the validity of the measure). The SCDC was sent to parents of twins aged 5–17 years from a register of births in south-east Wales (details available in Reference Scourfield, Martin and EleyScourfield et al, 2004). Of the 1109 families contacted, 670 replied, a response rate of 60%. The mean age of the girl twins was 10.6 years (s.d.=3.3) and that of the boys was 10.6 years (s.d.=3.2). There were 278 monozygotic pairs (124 male pairs and 154 female pairs) and 378 dizygotic pairs (198 opposite-gender pairs, 99 male pairs and 81 female pairs).
The test–retest reliability study sample was recruited from a database of females with Turner syndrome, based on a national case register. It comprised 254 individuals (mean age 15.7 years, s.d.=4.2, range 3.3–19.1). Verbal IQ data were available for 72 of these individuals, scored on the Wechsler Intelligence Scale for Children (Reference WechslerWechsler, 1992): the mean verbal IQ was 96.9 (s.d.=17.6, range 58–130).
Participants in the validity study comprised patients from three separate clinics: the social and communication disorders clinic at Great Ormond Street Hospital for Children, London (n=230), a child and adolescent mental health service (CAMHS) clinic in Luton (n=30) and another in Hartlepool (n=23). For the Great Ormond Street recruits, ICD–10 psychiatric diagnoses (World Health Organization, 1992) were established using a novel computerised autism interview, the Developmental, Dimensional and Diagnostic Interview (3di; Reference Skuse, Warrington and BishopSkuse et al, 2004). In the Great Ormond Street clinic the majority of referrals concerned children with neurodevelopmental or language problems. The CAMHS recruits were categorised according to clinician diagnosis. None of the patients recruited into the survey had participated in previous research, and none had previously been assessed with standardised psychiatric interviews for autism. The Great Ormond Street social and communication disorders clinic is a quaternary referral service that specialises in the assessment of children with high-functioning autism and complex presentations. The mean verbal IQ in the Great Ormond Street sample was 94.2 (n=164; range 40–153, s.d.=20.1) and the mean performance IQ was 92.7 (n=118; range 49–143, s.d.=18.7). Data on IQ were not available for the CAMHS participants, but all of them were in mainstream education. The Great Ormond Street sample for recruitment into this study comprised consecutive referrals to the clinic over a period of 4.5 years, from July 1999 to December 2003. The CAMHS recruits were consecutive referrals during January and February 2004. An additional sample of normal controls (n=118) was recruited to enable the assessment of the SCDC's validity as a screening instrument for autistic traits in the general population (Table 1). All members of the control group had intellectual abilities within the normal range, were English-speaking and were in mainstream schooling.
Validity sample characteristics
| Autism and atypical autism (n=208) | Clinic control group (n=76) | Normal control group (n=118) | |
|---|---|---|---|
| Gender: male, %1 | 82 | 88 | 49 |
| Age, years: mean (range) | 8.9 (2.5–18.1) | 10.8 (2.4–17.7) | 13.0 (7.3–17.4) |
| Verbal IQ: mean (s.d.)2 | 93.0 (20.6) | 102.5 (13.8) | |
| Non-verbal IQ: mean (s.d.)3 | 91.9 (19.0) | 99.7 (15.5) | |
| Proportion of verbal IQs below 80, % | 24 | 5 | |
| SCDC score: mean (s.d.) | 16.6 (5.7) | 13.0 (6.1) | 2.9 (4.0) |
The SCDC was sent out for completion by parents. For participants attending the Great Ormond Street Hospital clinic, this questionnaire formed part of the preappointment assessment, and if initially incomplete the omission was rectified prior to the clinical assessment. Unlike previous evaluations of the Social Responsiveness Scale (Reference Constantino and ToddConstantino & Todd, 2003) and the Autism Screening Questionnaire (now known as the Social Communication Questionnaire; Reference Berument, Rutter and LordBerument et al, 1999) in clinical populations, questionnaire completion never followed the standardised interview.
Establishing SCDC psychometric properties
Reliability
In order to assess internal reliability, the SCDC's internal consistency was calculated. External reliability was evaluated in terms of test–retest reliability; parents of 188 participants completed the SCDC for a second time, at a mean retest interval of 2.7 years (s.d.=0.5, range 1.51–5.39).
Validity
We assessed content validity primarily by comparing questions in the SCDC with items in standardised interviews, such as the Autism Diagnostic Interview – Revised (ADI–R; Reference Lord, Rutter and LeCouteurLord et al, 1994) and the 3di (Reference Skuse, Warrington and BishopSkuse et al, 2004), that most strongly discriminate autistic-spectrum disorders from non-autistic conditions. The domains of content of the questions (see Appendix) comprise social reciprocity (questions 1, 2, 3, 6 and 10), non-verbal skills (8) and pragmatic language usage (7, 11 and 12). Three questions concern functional impairment (3–5). There is, however, no explicit question concerning circumscribed interests or stereotyped patterns of motor behaviour.
Social and Communication Disorders Checklist The Social and Communication Disorders Checklist (SCDC) was devised to be simply and quickly rated, comprising just 12 questions. Nine of these serve to measure abnormalities in those aspects of the autistic triad that reflect ‘reciprocal social interaction skills’ and ‘communication skills’. Items 4, 5 and 6 measure behavioural problems in a more general sense, and reflect functional impairment. Each item on the scale is rated according to whether the behaviour has been seen over the past 6 months, and if so whether the associated statements are ‘quite or sometimes true’ or ‘very or often true’. Corresponding scores of 0, 1 and 2 apply, so the maximum possible score is 24. The instrument was originally developed to measure social-behaviour social-behaviour deficits in Turner's syndrome (Reference Skuse, James and BishopSkuse et al, 1997). Checklist For each item, please mark the box that best describes your child's behaviour over the past 6 months.
| Not true | Quite or sometimes true | Very or often true | ||
|---|---|---|---|---|
| 1. | Not aware of other people's feelings | □ | □ | □ |
| 2. | Does not realise when others are upset or angry | □ | □ | □ |
| 3. | Does not notice the effect of his/her behaviour on other members of the family | □ | □ | □ |
| 4. | Behaviour often disrupts family life | □ | □ | □ |
| 5. | Very demanding of other people's time | □ | □ | □ |
| 6. | Difficult to reason with when upset | □ | □ | □ |
| 7. | Does not seem to understand social skills, e.g. persistently interrupts conversations | □ | □ | □ |
| 8. | Does not pick up on body language | □ | □ | □ |
| 9. | Does not appear to understand how to behave when out (e.g. in shops, or other people's homes) | □ | □ | □ |
| 10. | Does not realise if s/he offends people with her/his behaviour | □ | □ | □ |
| 11. | Does not respond when told to do something | □ | □ | □ |
| 12. | Cannot follow a command unless it is carefully worded | □ | □ | □ |
For further validity analysis, the diagnosis of an autistic-spectrum disorder was defined according to ICD–10 criteria, to include autism, atypical autism and Asperger syndrome. In the Great Ormond Street clinic sample this was established from the 3di (see below) according to conventional criteria based on the ADI–R algorithm (Reference Lord, Rutter and LeCouteurLord et al, 1994), combined with findings from the Autism Diagnostic Observation Scale – Generic (Reference Lord, Risi and LambrechtLord et al, 2000). Individuals meeting only ICD–10 diagnostic criteria for pervasive developmental disorder not otherwise specified were categorised as non-cases for the purpose of this study. The 3di is a parental autism interview that can be administered to unselected clinical and general population samples; it measures both symptom intensity and comorbidity across the full range of the autistic spectrum (Reference Skuse, Warrington and BishopSkuse et al, 2004). It is a computerised procedure, for administration by trained interviewers, which generates symptom and diagnostic profiles for both autism and non-autistic conditions. The 3di's test–retest and interrater reliability were assessed in unselected clinical (n=50) and non-clinical (n=30) populations (Reference Skuse, Warrington and BishopSkuse et al, 2004). Concurrent (n=120), discriminant (n=120) and criterion (n=29) validity were evaluated in autistic-spectrum disorder and non-autistic patient groups. Test–retest and interrater reliabilities were excellent (most intraclass correlation coefficients were greater than 0.9). Concurrent validity of the 3di (agreement with independent clinician formulation) was very good (mean k=0.74). Criterion validity of the 3di, in a comparison with the ADI–R (Reference Lord, Rutter and LeCouteurLord et al, 1994), was excellent, and the instrument's ability to discriminate between autistic-spectrum v. non-autistic individuals was almost perfect (sensitivity 1.0, specificity 40.97).
Concurrent validity of the SCDC was assessed by a comparison of mean scores on this measure of children with clinically diagnosed autistic-spectrum disorder (n=208) and children with other clinical diagnoses (n=76). Non-autistic conditions in the comparison samples included conduct disorders, attention-deficit hyperactivity disorder (ADHD), pragmatic disorders of language, Tourette syndrome and obsessive–compulsive disorder, diagnosed by experienced clinicians according to ICD–10 criteria. We expected the mean SCDC scores of these clinical groups to be higher than those of children in the general population, because of their association with autistic features (e.g. Reference Geurts, Verte and OosterlaanGeurts et al, 2004; Reference Gilmour, Hill and PlaceGilmour et al, 2004). Accordingly a second test of concurrent validity was performed, to compare SCDC scores of the clinically identified samples with general population controls (n=118).
Criterion validity of the SCDC was evaluated by determining correlations between the questionnaire total score and the sub-scale scores of algorithms generated by the 3di (Reference Skuse, Warrington and BishopSkuse et al, 2004), which are equivalent to the sub-scale scores of the ADI–R algorithm (Reference Lord, Rutter and LeCouteurLord et al, 1994).
Discriminant validity analysis was conducted using receiver operating characteristic (ROC) analysis (Reference FombonneFombonne, 1991) in which the area under a ROC curve (the AOC) serves as an index of a test's accuracy in discriminating between groups. In this analysis clinical cases of autistic-spectrum disorder were compared with other clinical and non-clinical cases. Subsequently, the sensitivity and specificity of the instrument were determined, based on the optimal cut-off that had been derived for discriminating autistic-spectrum disorder from other clinical conditions and from normal-range behaviour.
Statistical analysis was conducted using the Statistical Package for the Social Sciences (SPSS version 11 for Windows). Test–retest reliability of the SCDC was assessed using intraclass correlation coefficients (ICCs). One-way ICCs were used, to allow for inter-individual variability. Internal consistency was evaluated by calculating Cronbach's a coefficient. Concurrent validity was assessed using one-way analysis of variance (ANOVA), with clinical group as the factor. Tamhane's T2 (Reference TamhaneTamhane, 1979) was used as a post hoc test to see which diagnostic groups differed from each other in terms of mean SCDC score.
RESULTS
Heritability study
Earlier analyses of these data showed no significant difference in the size of genetic influence in males and females and no evidence of separate non-additive genetic effects in females (Reference Scourfield, Martin and LewisScourfield et al, 1999). Here, additional analyses have further examined the question of gender differences and have included the influence of a scalar amplification or dampening of the phenotype in either gender. This allows for comparison with another twin study of an autism trait measure (Reference Constantino and ToddConstantino & Todd, 2003). These additional analyses have shown no evidence of separate additive genetic effects in males or females and no evidence of scalar amplification or dampening of the phenotypic trait in either gender. For both males and females the influence of the shared environment was non-significant and the best fitting model showed, for both genders, a heritability of 0.74 and non-shared environmental influence of 0.26 (summary of data presented as a data supplement to the online version of this paper).
Internal and external reliability of the SCDC
Cronbach's α coefficient for the SCDC was 0.93, showing that the content of the instrument has high internal consistency. The ICC for test–retest SCDC scores on a clinical sample of 188 children (Reference Skuse, James and BishopSkuse et al, 1997) with a mean retest interval of 2.7 years (s.d.=0.5, range 1.51–5.39) was 0.81 (95% CI 0.76–0.86).
Validity of the SCDC
Concurrent validity
The mean SCDC score for the autistic-spectrum group was 16.6 (s.d.=5.7), which was significantly higher than that of the clinical control group (mean score 11.6, s.d.=6.6) and the community control group (mean score 2.9, s.d.=4.0); one-way ANOVA (F (2,346)=258.72, P<0.001). However, it should be noted that Levene's test showed that the assumption of equality of variances had been violated for this analysis (Levene statistic 19.6, P<0.001). Tamhane's T2 was therefore used for post hoc comparisons between groups, as this test is specially designed for situations in which population variances differ, and is conservative in relation to type 1 error. This post hoc analysis showed that significant differences in SCDC scores exist between all three clinical groups: for each group comparison, P<0.001.
Discriminant validity
Discriminant validity was then assessed by determining the power of the SCDC to distinguish participants with autistic-spectrum disorder from non-autistic participants, using ROC analysis. This analysis was done in two parts. First, we found that the SCDC showed impressive accuracy in discriminating children with an autistic-spectrum disorder from (clinical plus non-clinical) controls (AOC=0.86, P<0.001). Maximal discrimination between all pervasive developmental disorder (PDD) diagnoses and non-PDD diagnoses/normal comparisons was obtained at a cut-off score of 9 points (a score of 9 or above implied a case). Sensitivity was 0.90 and specificity was 0.69 with this cut-off; the positive predictive validity was 0.75 and the negative predictive validity was 0.86. Of the 61 false positives obtained with this cut-off, 19 (31%) were clinical control cases selected from children attending the Great Ormond Street clinic. These were cases of social communication difficulty on referral that had already been assessed locally and were referred to our national centre for a second (or even a third) opinion. Their presence is likely to have raised the false-positive rate; among comparisons from the general population the false-positive rate was only 9%. We repeated the ROC analysis excluding data from the general population sample. The sensitivity of the instrument (with the identical cut-off) was the same (0.9) but the specificity was reduced to 0.35.
Criterion validity
Finally, criterion validity was assessed by comparing total scores on the SCDC with the ADI–R equivalent algorithm output generated by the 3di, for the Great Ormond Street sample (n=230), comprising 73 children with autism, 131 with other PDD diagnoses and 26 without a PDD diagnosis. These correlations were modest, which is unsurprising in view of the fact that the items that make up the SCDC were not derived from the ADI–R and are designed to measure autistic traits rather than for diagnostic purposes. Correlation with the social interaction sub-scale was 0.41 (P<0.001), correlation with the language/communication sub-scale was 0.30 (P<0.001) and correlation with the repetitive and stereotyped behaviour sub-scale was 0.21 (P<0.01). The correlation between the SCDC total and the 3di total score was 0.38 (P<0.001).
DISCUSSION
Population screening for autistic traits
‘Is autism one end of the normal spectrum of behaviour, or is it an abnormal condition?’ (Medical Research Council, 2001). To date, no whole population screen for autistic traits has been published. We simply do not know whether autistic behaviours are continuous with a normal distribution of severity, as the work of Spiker et al (Reference Spiker, Lotspeich and Dimiceli2002) and Constantino & Todd (Reference Constantino and Todd2003) implies; or whether autism is distinct, either in terms of bimodality of quantitative trait distribution or in qualitative difference from normal development. The SCDC may be used for a first-stage screen of school-aged populations in order to provide an answer to these questions.
Heritability of the SCDC
Autism is a highly heritable disorder, but of the various screening instruments available only the Social Responsiveness Scale has been evaluated in terms of formal heritability by means of a twin study. Constantino & Todd (Reference Constantino and Todd2003) report a best fitting model heritability estimate of 0.48, with a sample size of 788 twin pairs. The best fitting model for SCDC data showed no significant gender differences and, using similar analyses, a substantially greater heritability of 0.76 with unique environmental influences of 0.24. A heritability of 0.76 is close to the heritability estimates (about 0.9) reported in twin studies of clinical cases of autism (Reference Bailey, Le Couteur and GottesmanBailey et al, 1995). No significant influence of the shared environment emerged in our study, although the upper 95% confidence limit of these estimates was 0.26 in females and 0.45 in males; therefore, it is possible that a larger sample size might have detected a more significant effect.
Comparative reliability and validity of the SCDC
The instrument compares well with existing autism screening tools in terms of its psychometric properties. Internal consistency of the SCDC is very high (0.93), indicating it has a simple factorial structure, both in studies of symptomatic cases and in the general population. The derivation of this 12-item questionnaire from principal components analysis of a longer instrument was described by Skuse et al (Reference Skuse, James and Bishop1997). Test–retest reliability (established with an interval of nearly 3 years) has not, to the best of our knowledge, been evaluated with comparable instruments. The issue of validity was established in terms of content, concurrent validity, discriminant and criterion validity. In common with the Social Responsiveness Scale (Reference Constantino, Davis and ToddConstantino et al, 2003), total scores on the SCDC were independent of IQ. The sensitivity and specificity values obtained by Berument et al (Reference Berument, Rutter and Lord1999) for the questionnaire now known as the Social Communication Questionnaire (0.85 and 0.75 respectively) were closely similar to our own estimates of 0.90 and 0.69 respectively, which were based on a ROC analysis of a sample that contained a high proportion of children with no psychiatric diagnosis. Because the inclusion of the latter sample might have led to inflated estimates of the SCDC performance, we subsequently conducted a further ROC analysis in which cases of autistic-spectrum disorder were compared with other social communication and neurodevelopmental disorders. The high sensitivity of the instrument was replicated (0.90), which is appropriate for a screening instrument, but specificity was substantially reduced. This is no doubt because autistic traits are strongly correlated with common problems such as ADHD (Reference Geurts, Verte and OosterlaanGeurts et al, 2004) and conduct disorder (Reference Gilmour, Hill and PlaceGilmour et al, 2004). We did not anticipate that the SCDC would be suitable for making discriminations within the autistic spectrum of disorders: such discrimination was a problem even for the Autism Screening Questionnaire (Reference Berument, Rutter and LordBerument et al, 1999), which was designed specifically for application to clinical populations (Reference Volkmar, Lord and BaileyVolkmar et al, 2004).
Autism as a dimensional disorder
Increasing evidence supports the hypothesis that autism is a quantitative or dimensional spectrum, with no clear qualitative distinction between traits found among individuals with the disorder and the general population. The majority of people with autism probably have IQ scores in the normal range, although autistic behaviours may be proportionately more common among those with learning disabilities (Medical Research Council, 2001). Is autism unidimensional, as claimed by Constantino & Todd (Reference Constantino and Todd2003) and Spiker et al (Reference Spiker, Lotspeich and Dimiceli2002)? Contrary evidence is provided by Silverman et al (Reference Silverman, Smith and Schmeidler2002), who found that social and language deficits in autistic disorders were not closely correlated with stereotyped and repetitive behaviours. Screening questionnaires are generally not sensitive to the latter dimension of autistic impairment, which has proved to be problematic, in terms of weak diagnostic differentiation, in studies of autistic individuals with IQ scores in the normal range (e.g. Reference Berument, Rutter and LordBerument et al, 1999). Items concerning such traits are virtually absent from similar screening instruments (Reference Constantino, Davis and ToddConstantino & Todd, 2003).
Implications
Recent surveys of the prevalence of autism in the community indicate not only an increase in the number of cases meeting conventional criteria, but a disproportionate increase in the number of milder cases that fail to reach full ICD–10 or DSM–IV (American Psychiatric Association, 1994) criteria (Reference Chakrabarti and FombonneChakrabarti & Fombonne, 2001; Reference Yeargin-Allsopp, Rice and KarapurkarYeargin-Allsopp et al, 2003). Subclinical cases of autism may present indirectly, for example with conduct problems at school (Reference Gilmour, Hill and PlaceGilmour et al, 2004). The burgeoning recognition of autistic disorders is putting a great strain on local services. Rational planning for the likely number of as-yet-unrecognised cases requires a better estimate than we currently have of where the boundaries of the autistic spectrum lie. The SCDC, a brief, reliable and valid screening questionnaire, should finally allow this question to be answered in the context of a whole-population survey of school-age children.
Clinical Implications and Limitations
CLINICAL IMPLICATIONS
-
▪ The Social and Communication Disorders Checklist (SCDC) is a brief and effective screening measure for pervasive developmental disorders.
-
▪ It will be useful in studies that aim to estimate the boundaries of the autistic spectrum by measuring autistic traits in large community samples.
-
▪ The SCDC can be used to further our understanding of the role of sub-threshold autistic traits in behavioural difficulties and conditions such as attention-deficit hyperactivity and conduct disorders.
LIMITATIONS
-
▪ Interrater reliability data are needed.
-
▪ The SCDC was designed for large-scale surveys and is not suitable for providing clinical diagnoses. It has excellent sensitivity, but low specificity with regard to the diagnosis of autism itself.
-
▪ The SCDC is a parent-report measure. A self-rated version for adults has not been validated in adult populations of individuals with autism.
Acknowledgements
We are grateful to the families who assisted in the gathering of data for this study, and to the consultants who assisted with the collection of clinical material, especially Dr Uttom Chowdhury and Dr Maurice Place.
eLetters
No eLetters have been published for this article.