Sample

The sample comprised all 226 individuals who were referred over a 3-year period to a specialist mental health service for people with intellectual disabilities. Of these 226 individuals, 140 (62%) were male and 86 (38%) were female. The average age was 34 years (s.d.=13.5). Most (71%) of those referred were White, 19% were African–Caribbean, 6% were Asian and 4% were classed as other ‘non-White’. More than two-thirds (68%) had mild intellectual disability, 20% had moderate intellectual disability and 12% had severe intellectual disability. All lived in the community: 49% lived with their family, 31% lived in supported housing for four or more people which was not health-service funded, 16% lived independently and 4% lived in a health service residence for eight or more people which was funded by the health service. Table 1 lists the psychiatric diagnoses made by the clinician at assessment.

Table 1

Psychiatric diagnoses made by the clinician at assessment (n=226)

Diagnostic category (ICD-10 code)	% (n)
No psychiatric diagnosis	35 (78)
Schizophrenia spectrum disorder	19 (42)
(F20-29)
Depressive disorder (F32-39)	12 (27)
Anxiety (F40-48)	8 (19)
Adjustment reaction (F43)	8 (18)
Dementia (F00-03)	4 (10)
Personality disorder (F60-69)1	8 (18)
Other: delirium (F05) eating disorders	6 (14)
(F50) and hyperkinetic disorders (F90)1

Procedure

Reliability

Item analysis

Table 2 summarises the results of the item analysis. Scales A and B and total scores 1 (affective/neurotic disorders) and 2 (possible organic disorders) had alpha values equal to or greater than 0.7. Scales C, D and E and the total score 3 (psychotic disorders) had alpha values equal to 0.6. The median item–total (minus item) point biserial correlations were greater than 0.3. Every scale had a number of rogue items with item–total correlations of less than 0.3. In some cases there were scales, such as scale B, where 4 out of 11 items had item–total correlations of less than 0.3.

Table 2

Values of Cronbach's alpha, item—total point biserial correlations and number of rogue items for the PAS—ADD Checklist scales and scores

	Items n	α	Item—total point serial correlation Median (range)	Rogue items n
Scale
A	7	0.7	0.36 (0.10-0.54)	2
B	11	0.7	0.33 (0.07-0.45)	4
C	4	0.6	0.37 (0.22-0.53)	2
D	2	0.6	0.47 (0.47)	0
E	5	0.6	0.30 (0.24-0.50)	1
Total score
1 Affective/neurotic	22	0.8	0.41 (0.08-0.61)	5
2 Organic	6	0.7	0.52 (0.21-0.56)	2
3 Psychotic	5	0.6	0.30 (0.24-0.50)	1

Factor analysis

The results of the factor analysis are shown in Table 3. The first nine factors had eigen-values greater than 1.0 and these factors accounted for 64% of the variance. An examination of a scree plot suggested a single factor structure, since the first factor accounted for 20% of the variance and the subsequent eight factors accounted for 4–8% of the variance. The first factor was characterised primarily by items related to mood, such as loss of interest and energy, sadness, avoiding conversation, low self-esteem, loss of appetite and confidence, and poor concentration. The second factor was characterised by three items related to sleep disturbance. The third factor was characterised by three items related to psychotic symptoms. It was difficult to characterise subsequent factors clearly.

Table 3

Exploratory factor analysis for the PAS—ADD Checklist

	Factor
Item	1	2	3	4	5	6	7	8	9
Loss of interest	0.80
Loss of energy	0.74
Feeling sad or ‘down'	0.63	0.37
Avoiding conversation	0.65							0.31
Low self-esteem	0.57						0.44		0.53
Loss of appetite	0.59
Loss of confidence	0.55								0.36
Concentration	0.59				0.40	0.40
Sleep broken		0.88
Sleep delayed		0.79
Waking		0.71
Beliefs			0.84
Experience			0.77
Suspicious			0.61
Gesture				0.79	0.80
Language				0.75	0.81
Too happy				0.55
Irritable					0.74
Startled				0.56	0.44				0.32
Restless			0.32		0.64	0.33
Forgets	0.34				0.36	0.39
Self-care	0.41				0.39	0.65	0.42		0.31
General fear			0.64				0.80
Sudden fear					0.82	0.32	0.31	0.62
Incapacitated								0.79
Repeated actions			0.32		0.36			0.57
Suicide							0.79		0.89
Eigenvalue	5.33	2.20	1.83	1.69	1.50	1.44	1.18	1.11	1.02
Variance (%)	19.7	8.1	6.8	6.2	5.6	5.3	4.4	4.1	3.8

Validity

Table 4 shows the PAS–ADD Checklist scores for people who were diagnosed by a clinician as having ‘no diagnosis’, schizophrenia spectrum disorder, personality disorder, anxiety disorder, depressive disorder or adjustment reaction. There was a significant difference between individuals on total score 1 (affective/neurotic disorder), in which people who had depressive disorder scored higher than those with no diagnosis, and all other psychiatric diagnoses. There was a significant difference between individuals on total score 2 (possible organic disorder), in which those with depressive disorder scored higher than those with no diagnosis. There was a significant difference between individuals on total score 3 (psychotic disorder), in which people with schizophrenia spectrum disorder scored higher than those with no psychiatric diagnosis and all other diagnoses.

Table 4

Diagnostic group scores on the three total scores of the PAS—ADD Checklist

Variable	No psychiatric disorder	Schizophrenia spectrum disorder	Personality disorder	Anxiety disorder	Depressive disorder	Adjustment reaction disorder	F	Post hoc Scheffé tests
n	78	42	18	18	28	18
Total score: mean (s.d.) 1 Affective disorders	2.53 (3.47)	4.14 (3.97)	3.39 (4.15)	4.83 (4.00)	9.75 (6.37)	3.78 (4.58)	12.04***	Depression > no pathology, schizophrenia, personality disorder, adjustment reaction, anxiety
2 Organic disorders	1.18 (1.65)	1.50 (1.71)	1.00 (1.50)	1.06 (1.11)	2.54 (2.44)	1.50 (2.09)	2.84*	Depression > no pathology
3 Psychotic disorders	0.60 (1.10)	2.05 (1.99)	0.61 (1.15)	0.44 (0.78)	0.32 (0.72)	0.44 (1.10)	9.71***	Schizophrenia > no pathology, personality disorder, adjustment reaction, anxiety, depression

Sensitivity and specificity

Table 5 shows the numbers of people who crossed any PAS–ADD Checklist threshold in relation to the numbers who had a clinical psychiatric diagnosis covered by the Checklist. The sensitivity of the PAS–ADD Checklist was 66% and the specificity was 70%.

Table 5

Numbers of people who crossed any PAS—ADD Checklist threshold in relation to the numbers who had a clinical psychiatric disorder covered by the Checklist

Presence of disorder	Checklist thresholds crossed
	Yes	No	Total
Psychiatric disorder covered by Checklist	76	40	116
Psychiatric disorder not covered by Checklist/no	33	77	110
psychiatric disorder present
Total	109	117	226

Reliability

Validity

The validity of the PAS–ADD Checklist appears to be good when considering the scores of people who have different psychiatric diagnoses. People who had depressive disorder scored higher on total score 1 (affective/neurotic disorder) than those who did not have this disorder, demonstrating that in terms of affective/neurotic disorders the Checklist performed well and identified the correct people. Individuals with depressive disorder also scored significantly higher than those without this disorder on total score 2 (possible organic disorder), although the significance was relatively low. This is not surprising, because there is some overlap between the scales that contribute to total score 1 and total score 2. Also, no individual in this section of the analysis had an organic disorder, so we would not expect the scores of the people with the disorders that are included to vary significantly on this organic disorder threshold.

People with schizophrenia spectrum disorder scored significantly higher on total score 3 (psychotic disorder) than people with any other diagnosis, confirming that the Checklist performs well on this disorder.

Sensitivity

Any screening tool must be assessed in relation to sensitivity. The main criticism of the PAS–ADD Checklist in this study relates to this measure.

The sensitivity (proportion of people with a psychiatric disorder covered by the Checklist who are correctly classified by the instrument as having a psychiatric disorder) of the PAS–ADD Checklist was 66%. This is lower than the figure of 78% calculated from the findings of the developers of the Checklist (Reference Moss, Prosser and CostelloMoss et al, 1998) and is also lower than other screening measures such as the 12-item General Health Questionnaire, which has a sensitivity of 76% (Reference Goldberg, Gater and SartoriusGoldberg et al, 1997). There are several possible explanations for the presence of false negatives. Moss et al (Reference Moss, Prosser and Costello1998) found that the likelihood of crossing the thresholds rose with severity of the illness. Although in our study the severity of clinician diagnosis was not recorded, it might have been the case that some of these people did not have symptoms that were severe enough to be picked up by the Checklist. Of the people not crossing any threshold, 14 had schizophrenia spectrum disorder, which is a chronic disorder. At assessment these people's symptoms might have been absent and therefore not identified by the Checklist, if controlled through medication or if the person's disorder was in remission. Unfortunately, these data were not available, so this can only be proposed as a possible explanation.

The large number of people diagnosed as having an affective disorder but not crossing any of the thresholds (n=25) might be due to the nature of these diagnoses. Although some aspects may be observable, and the Checklist focuses mainly on these elements, scoring highly on the checklist and crossing a threshold is reliant to some extent on the person being able to communicate how he or she is feeling. This may be easier to elicit from people with intellectual disabilities in a clinical assessment rather than by use of a Checklist that is not completed by the patients themselves.

The breakdown of level of intellectual disability in those who had a diagnosis covered by the Checklist but who did not cross the threshold was similar to the breakdown of the total sample.

Although the above explanations may very well be valid, the data are not available to prove them, and the fact remains that the sensitivity of the PAS–ADD Checklist in this study was fairly low. A further consideration raised by this analysis is that 14% of this sample had a psychiatric diagnosis that the PAS–ADD Checklist was not designed to identify and therefore could not be expected to pick up.

If anything, we would expect a screening instrument to be overinclusive rather than underinclusive. In this study 15% of the total sample had no psychiatric disorder, or a psychiatric disorder that was not covered by the Checklist but crossed at least one of its thresholds. This is higher than the 8% of false positives calculated from findings of the Checklist's developers (Reference Moss, Prosser and CostelloMoss et al, 1998). For the purpose of screening people for further psychiatric assessment, it is preferable to have false positives rather than false negatives: people with intellectual disability may find going to a psychiatric out-patient clinic very upsetting and a high rate of false positives would be costly. Therefore we would hope for a low false positive rate. The specificity of the PAS–ADD Checklist was 70%, indicating that 70% of people who did not have a psychiatric disorder or had a psychiatric disorder that was not covered by the PAS–ADD Checklist were correctly identified.

We did not explore the sensitivity and false positive rates of the PAS–ADD Checklist with lower threshold scores. However, this may be something to consider in the future.

In summary, the PAS–ADD Checklist had acceptable internal consistency, one main factor characterised by mood items was sensitive to differences between diagnostic groups, and had an overall sensitivity of 66%.

Limitations of the study

There was only a small number of people with an organic disorder in this sample. Consequently, it was difficult to determine how successful the PAS–ADD Checklist was at identifying these disorders. It would also have been useful to have had some measure of the severity of the disorders as clinically diagnosed, as this would have enabled us to comment further on the issue of severity of symptoms affecting the crossing of the threshold scores.

The PAS–ADD Checklist has been revised since our study was completed, and although the items in the two versions differ only slightly, there is some variation in the order the items are presented. Whether this revision will affect the PAS–ADD Checklist's psychometric properties remains to be seen in future research.

CLINICAL IMPLICATIONS

1. ▪ The Psychiatric Assessment Schedule for Adults with Developmental Disabilities (PAS–ADD) Checklist should not be used as the sole screening method for identifying possible psychiatric diagnoses in people with intellectual disabilities; however, it is the best psychometric measure available.

2. ▪ Appropriateness of referral to mental health services for full assessment, diagnosis and treatment might improve through use of the PAS–ADD Checklist.

3. ▪ The PAS–ADD Checklist appears to have the potential to reduce the level of undetected mental health problems among people with intellectual disabilities.

LIMITATIONS

1. ▪ The number of people with organic disorders was small, making it difficult to determine the success of the PAS–ADD Checklist in identifying them.

2. ▪ There was no measure of the severity of symptoms.

3. ▪ The Checklist has been revised since this study.