The National Institute for Health and Clinical Excellence (NICE) guidelines 1 on the diagnosis and management of attention-deficit hyperactivity disorder (ADHD) in children, young people and adults state that the complexity of the ADHD assessment requires a multimodal, multiprofessional and multi-agency approach. Standardised rating scales are commonly used in clinical practice to help obtain information relating to the severity and pervasiveness of behavioural and social problems associated with ADHD. Weaknesses in interrater reliability depend to a degree on the subjective differences in expectations and tolerance of certain behaviours subject to scoring by the rater. Reference Hale, Fiorello and Brown2 Since cognitive impairments around attention and impulse control are very likely to be present in ADHD, attempts have been made to include psychometric tests such as the Continuous Performance Test (CPT) Reference Rosvold, Mirsky, Sarason, Bransome and Beck3,Reference Barkley and Murphy4 to provide objective measurements especially associated with selective and sustained attention, vigilance and impulsiveness (response inhibition). At present the NICE guidelines recommend the need for further research on the extend to which neuropsychological tests can effectively be used to guide psychological interventions. New infrared motion detection has successfully been combined with the CPT to measure a test person's motoric activity during CPT. This combined system of measuring objectively activity, attention and impulse control measures was found to be an effective means of quantifying hyperactivity and correlated significantly with commonly used teacher rating scales for children with ADHD. Reference Teicher, Ito, Glod and Barber5 The aim of this audit was to appraise how our routine standard ADHD assessments compare with routine standard ADHD assessments that include objective measurements of activity, attention and impulse control (QbTest system, www.qbtech.se/products/qbanalysis). For this purpose we compared the outcomes of the clinical assessment with scores from the Strengths and Difficulties Questionnaire (SDQ) Reference Goodman6 and the objective measures.
Method
The notes of 108 individuals with a referral for ADHD made to our local generic child and adolescent mental health services (CAMHS) clinic over 2 years were reviewed. All assessments undertaken followed the NICE guidelines. The essential components of a full assessment process included a clinical interview by the child and adolescent psychiatrists (five) at the clinic at the time of collecting data for the audit, a medical examination and the administration of rating scales by parents and teachers. The SDQ was used to look at a wide variety of childhood behaviours, possible psychopathology and to evaluate psychosocial functioning. As a rating scale specifically for the symptoms of ADHD our clinic routinely administered the Conners Rating Scales - Revised Reference Conners7 (short or long version) or the Conners Abbreviated Teacher Rating Scale, depending on the clinician's individual preference. Because there was no uniform application of the Connors Rating Scales data, analysis of the this was not carried out. Direct observation in educational settings was undertaken when following the clinical interview some questions remained with regard to the severity and pervasiveness of the behavioural problems and further information was required to determine specific scholastic difficulties and the outcome of strategies applied to address the child and adolescent's behavioural difficulties. Reports of developmental and literacy skills assessments were provided when indicated.
The ADHD assessments in the year prior to using objective measurements (2006-2007 control group, n = 46) were compared with ADHD assessments in the first year of adding objective measures to the assessment (2007-2008 QbTest group, n = 62). The same child and adolescent psychiatrists conducted the assessments for both groups using the same protocol.
In terms of objective measurements a system combining computer-administered CPT with infrared motion capture (QbTest) was used (for further information and a visual description of the system, go to www.qbtech.se/products/qbanalysis). The test provides an objective analysis by comparing the test scores from the child referred to the clinic with the scores from normative data of schoolchildren of the same age and gender. Separate measurements for activity, attention and impulse control are presented in the test report. Measurements ranging two standard deviations from the population mean were considered to be a result outside the normal range. A QbTest Behaviour Rating Scale was completed with information about the child's motivation and engagement during the test. The QbTest takes 15 min for children aged 6-12 years and 20 min for young people aged 13-18 years to complete. The test can be administered by medical clinicians, psychologists and specialist nurses. The additional objective information obtained from this test assisted clinicians in the QbTest group to confirm or reject a diagnosis of ADHD.
The SDQ is a commonly used questionnaire in clinical practice. Diagnostic predictions based on the SDQ agree well with clinical diagnoses; Reference Goodman, Renfrew and Mullick8 however, sensitivity is known to be substantially poorer with single-informant rather than multi-informant SDQs. Reference Goodman, Ford, Simmons, Gatward and Meltzer9 The clinical decision to diagnose ADHD or not by the child and adolescent psychiatrist was compared in both the control and QbTest group with the hyperactivity scores from parent and teacher SDQs.
At 1-year follow-up the notes of the assessed individuals were reviewed for revised diagnoses, allocated interventions and those lost to the clinic. The initiation of treatment with stimulant medication and its continuation/discontinuation over 1 year was recorded. From the newly diagnosed children and adolescents with ADHD in the QbTest group, those consenting to treatment with stimulant medication repeated the QbTest on a test dose of methylphenidate prior to commencing a trial on stimulant medication.
Results
The mean age of children and adolescents in the control group was 9 years (mode 10 years, median 9 years) and 10.5 years for the QbTest group (mode 8 years, median 10 years). Table 1 compares the outcome of the clinical assessments with the hyperactivity scores of the SDQ. A positive SDQ reflects both parent and teacher ratings for hyperactivity in the abnormal range, whereas a negative SDQ includes individuals whose parent and teacher ratings were both below the cut-off point and in the normal range. Positive/negative SDQ represents ratings for hyperactivity with either one of the parent or teacher rating in the abnormal range and the other rating below the cut-off point or in the normal range producing a mixed result. Among those with a positive/negative SDQ in both the control and QbTest groups the majority of parents’ SDQs (10/13, 77%) agreed with the clinician's diagnosis of ADHD, whereas the majority of teacher's SDQs (13/18, 72%) agreed with the clinician's rejection of a diagnosis.
Comparison between clinical diagnosis and hyperactivity scores from the Strengths and Difficulties Questionnaire
| Strengths and Difficulties Questionnaire, hyperactivity: n | ||||
|---|---|---|---|---|
| n (%) | Positive | Positive/negative | Negative | |
| Control group | 46 | |||
| ADHD | 27 (59) | 19 | 7 | 1 |
| No ADHD | 19 (41) | 5 | 12 | 2 |
| QbTest group | 62 | |||
| ADHD | 43 (69) | 33 | 6 | 4 |
| No ADHD | 19 (31) | 10 | 6 | 3 |
ADHD, attention-deficit hyperactivity disorder.
At 1-year follow-up seven participants in the control group had their assessment revised (Table 2). All of these had clinical assessments where a diagnosis of ADHD was initially rejected despite a positive SDQ (three individuals) or a positive/negative SDQ (four individuals, all positive scores from parents). The clinician's decision to initially reject a diagnosis was based on a lack of pervasiveness in the child's history and presentation of the core signs of ADHD or the symptoms were attributed to coexisting emotional difficulties (anxiety disorder, emotional disorder unspecified, early trauma). The majority of the revised assessments were for girls (n = 4). In comparison, none of the clinical assessments in the QbTest group with a rejected diagnosis had their clinical assessment changed at 1-year follow-up (P>0.0035, Fisher's exact test).
Follow-up over 1 year of the participants referred for an attention-deficit hyperactivity disorder (ADHD) assessment with a diagnosis rejected at the initial assessment
| Did not attend/lost to clinic | Reassessed and diagnosed with ADHD at 1-year follow-up | |
|---|---|---|
| Control group (n = 19) | 3 | 7 |
| QbTest group (n = 19) | 1 | 0Footnote a |
a. Two-tailed Fisher's exact test P = 0.0035.
Of the 19 participants in the QbTest group with a rejected diagnosis the scores for the three core signs of ADHD (overactivity/inattention/impulsivity) were either in the normal range or in some cases only one of the three core signs was abnormal. A typical finding would be increased activity scores with normal attention and impulse control scores in six participants. Five out of the six participants had a positive SDQ and one had a positive/negative SDQ. In these individuals the findings triggered a referral to occupational therapy for further assessment for a possible developmental coordination disorder. The occupational therapy assessment confirmed difficulties in static balance, manual dexterity and muscle tone.
Five participants with normal attention and impulse control measurements among the QbTest group with a rejected diagnosis received further assessment for coexisting social communication difficulties confirming a diagnosis of autism-spectrum disorder with sensory integration difficulties.
The majority of participants in the control group who were not diagnosed with ADHD received interventions for emotional difficulties. One individual with a known history of developmental coordination disorder was referred back to occupational therapy and two participants received further assessments of their social communication difficulties.
All seven girls in the QbTest group diagnosed with ADHD (16%) presented with QbTest scores outside the normal range for activity as well as attention and impulse control suggesting ADHD of the combined type. There were six individuals in the QbTest group (14%) that according to their QbTest result met criteria for ADHD inattentive type. Three individuals in the control group (11%) met criteria for ADHD inattentive type.
There was a trend towards a higher rate of children and adolescents on stimulant medication continuing to take their medication at 1-year follow-up (Table 3). The difference however was not statistically significant. Among individuals with a confirmed diagnosis of ADHD despite a mixed positive/negative SDQ the trend to continue with medication at 1-year follow-up was also better in the QbTest group (5/6) compared with the control group (2/7).
Follow-up over 1 year of the participants diagnosed with attention-deficit hyperactivity disorder (ADHD)Footnote a
| n (%) | |||||
|---|---|---|---|---|---|
| Medication trial |
Continued medication |
Discontinued medication |
Did not attend/ lost to clinic |
Diagnosis changed |
|
| Control group (n = 27) | 22 (81) | 13 (59) | 9 (41) | 3 (11) | 1 (4) |
| QbTest group (n = 43) | 38 (88) | 28 (74)Footnote b | 9 (24) | 4 (9) | 1 (2) |
a. In each group the diagnosis of one participant was altered from ADHD to oppositional defiant disorder.
b. Two-tailed Fisher's exact test P = 0.24.
Discussion
The results from this audit suggest that through greater symptom specification with the use of objective measurements clinical decisions remained more consistent and were less likely to be revised over 1 year. Children with high levels of ADHD symptoms differ significantly from those in a community-based control group with regard to neurocognitive variables. Reference Wahlsted, Thorell and Bohlin10 Adding a CPT to the assessment allows for further evaluation of neurocognitive variables, such as a primary impairment in the ability to sustain attention during a task that does not require the child to shift strategies. This test condition allows children with autism to demonstrate an unremarkable primary ability to sustain attention (compared with children with ADHD) when the clinical presentation of attention difficulties may be attributable to other factors such as motivational contingencies or deficits in executive functions with externally imposed tasks being too complex. Reference Garretson, Fein and Waterhouse11 The simultaneous capture of motion-activity during CPT yields the opportunity to analyse the interplay between performing a cognitive task (CPT) and the range of body movement while in a seated position. Children with postural control difficulties as part of a developmental coordination disorder are known to demonstrate greater activity while performing a cognitive task. Reference Laufer, Ashkenazi and Josman12 Thus these children's activity levels may become more noticeable during the test when at the same time they deliver normal scores for attention and impulse control on the CPT. Assessments in the QbTest group that had a diagnosis of ADHD rejected despite positive or mixed SDQs presented with normal objective measures or raised scores in one of the three core signs of activity, attention and impulsivity. Raised scores in activity measures alone matched in particular with coinciding developmental coordination disorder and autism-spectrum disorder with sensory overresponsivity, a sensory modulation disorder characterised by responses to sensory stimuli that are faster, longer or more intense than what would be expected with typical sensory responsivity. Reference Miller, Anzalone, Lane, Cermak and Osten13 The overlap between developmental coordination disorder and ADHD has repeatedly been noted in cross-sectional and longitudinal studies. Reference Kadesjo and Gillberg14,Reference Rasmussen and Gillberg15 Sensory overresponsivity as well as frequent motor control problems are commonly found in autism-spectrum disorder. Reference Kopp, Beckung and Gillberg16,Reference Reynolds and Lane17 From our audit it appears that the additional neurocognitive information provided by the QbTest measures was useful in determining differences in the degree and quality of attentional and impulse control difficulties when the behavioural manifestations in these conditions are difficult to differentiate clinically. Improved distinction and differentiation helped to determine the appropriate choice of treatment intervention.
Clinical decisions made without additional information from objective measures were more prone to change within a year as compared with clinical decisions in the QbTest group. In particular there were assessments for a number of individuals in the control group where despite positive or mixed SDQs the clinicians rejected a diagnosis of ADHD but that were later on converted to a diagnosis of ADHD; this did not occur in the QbTest group. This difference was statistically significant. One possible explanation could be that the clinician's decision to reject a diagnosis in the control group was initially influenced by the child's normal appearance and behaviour during the clinical interview, whereas further information generated through the objective measures in the QbTest group helped to highlight activity, attention and impulse control difficulties and put them into context with the information from the referral, questionnaires and behavioural reports.
By monitoring the child/adolescent's behaviour during the QbTest, the opportunity to observe behavioural differences between the clinical interview and the behaviour during the test also appeared to improve the quality of the assessment especially in those children and adolescents whose symptoms may not stand out during an interview. Earlier criticism about CPT results generated in laboratory settings providing a greater risk of false negative assessments because of a low perceptual load Reference Nigg18 were therefore not found in this audit. In contrast, because of the challenging and demanding nature of the test on attention and impulse control more symptoms surfaced on the behavioural rating scales used by the examiner during the QbTest.
The relatively high rate of girls having their initial assessment changed at 1-year follow-up to a diagnosis of ADHD reflects findings suggesting an underidentification of hyperactive girls with ADHD. Reference Taylor, Heptinstall, Sonuga-Barke, Sandberg and Bowyer19 This did not seem to occur in the QbTest group suggesting adequate identification of girls with ADHD. Gender differences in activity levels may influence the appraisal of activity levels in girls by clinicians, carers and third-party informants. Significant differences have been reported suggesting that the average boy is more active than about 69% of girls. Reference Campbell and Eaton20,Reference Eaton and Enns21 The QbTest scores are compared with normative data of the same gender and age thus highlighting differences in activity levels that would otherwise be more difficult to distinguish if gender differences in activity levels were not taken into account.
There was a trend towards more children and adolescents on stimulant medication continuing to take their medication at 1-year follow-up in the QbTest group compared with the control group. All participants opting for a trial of stimulant medication in the QbTest group repeated the QbTest on a single test dose of methylphenidate shortly after their baseline test. Treatment effects were presented by demonstrating the pre- and post-test reports to the child and their carer. Showing the visible effect of the medication in the report together with the experience of having less difficulties during the test may provide a better understanding of the medication effect. This could be one possible factor contributing to a higher trend of continued medication at 1-year follow-up in the QbTest group with the difference not reaching statistical significance. However, the numbers in this study are too small to control for confounding factors such as side-effects and lack of treatment efficacy. Further studies with larger samples are required to evaluate the impact of medication response biofeedback on treatment continuation.
Open access
eLetters
No eLetters have been published for this article.