Validation of the Social Satisfaction Questionnaire for outcome evaluation in substance use disorders

AIMS AND METHOD To develop a scale to measure social satisfaction in people with substance use disorders and to test its psychometric properties. The rationale is that social satisfaction is more universal and relevant to treatment planning than assessing social problems. The new Social Satisfaction Questionnaire (SSQ) was derived from an existing social problems questionnaire and validation was undertaken on two large clinic populations.


D U N C A N R A I S T R I C K , G I L L I A N T O B E R , N I C K H E AT H E R A N D J E N N I F E R A . C L A R K
Validation of the Social Satisfaction Questionnaire for outcome evaluation in substance use disorders

AIMS AND METHOD
To develop a scale to measure social satisfaction in people with substance use disorders and to test its psychometric properties. The rationale is that social satisfaction is more universal and relevant to treatment planning than assessing social problems. The new Social Satisfaction Questionnaire (SSQ) was derived from an existing social problems questionnaire and validation was undertaken on two large clinic populations.

RESULTS
An eight-item SSQ was tested and found to have good psychometric properties in terms of test-retest reliability, internal consistency, distribution of responses and concurrent validity.

CLINICAL IMPLICATIONS
The SSQ is suitable for use as the social domain element of an outcome measures package.
Social functioning has consistently been associated with help-seeking and entry into treatment for alcohol and drug problems (e.g. Moos & Moos, 2004) and is an important domain to measure in the evaluation of treatment outcome; social problems are a key component of current models of the treatment process (Gifford et al, 2006;Orford et al, 2006). Poor levels of social functioning are associated with poorer treatment outcomes and risk of relapse (Moos & Moos, 2006). Substance use disorders span the socio-economic spectrum, and the task of identifying norms to measure social functioning components of treatment outcome is fraught with difficulty.
Standard measurement tools include the European Quality of Life (EuroQoL; Brooks et al, 2003), which is useful for comparisons of health status across populations but is insensitive for specific social outcomes, and the Short Form Health Survey (SF-36;Ware et al, 2000), which is acceptable to service users (Garratt et al, 1993) and suggested as a standard for the National Health Service (NHS) but is too lengthy to be included in routine, substance misuse specific outcome measurement and does not inform treatment goals. Criteria suitable for outcome measurement in routine clinical practice (Raistrick et al, 1994) are universality of scale items, change potential, items written in plain English, self-completion format to avoid therapist bias, satisfactory psychometric properties and brevity of completion. To include these and tap into the subjective experience of help seekers, we decided to measure satisfaction with social circumstances. Thus, the individual's objective social circumstances are less important than how they feel about them both for measurement and as the focus of treatment planning.
The relativity of social functioning was addressed by Corney & Clare (1985) who developed a 33-item Social Problems Questionnaire (SPQ) as a compact, comprehensible, valid and reliable self-report questionnaire for screening individuals in primary care or related settings who are particularly at risk for manifesting social maladjustment and dysfunction (Corney & Clare, 1985: p. 638).
Recognising the difficulties inherent in attempts to measure and compare social circumstances, the SPQ combines items enquiring about objective circumstances with items enquiring about the degree of satisfaction experienced with reference to these. The domains covered are: housing, occupation, finances, social activities and relationships, relatives, marital status, children, other domestic relationships, legal matters, and questions for those living alone. The scale was piloted and tested in primary care samples (including the responses of relatives), in psychiatric and epilepsy out-patient departments and in a social work case-load, and found to be a valid and reliable measure which is readily acceptable for use either by postal or face-to-face contact.
This paper describes the development and validation of the Social Satisfaction Questionnaire (SSQ), which was adapted from the SPQ (Corney & Clare, 1985) as a brief measure of satisfaction and designed for use in routine assessment and treatment outcome evaluation (Tober et al, 2000).

Method
The SSQ was developed in four steps: . step 1 -item reduction from the SPQ to create the eight-item SSQ . step 2 -piloting the SSQ . step 3 -main validation in clinical practice on two large samples . step 4 -test-retest reliability.
Items which did not measure satisfaction directly, those measuring actual circumstances, failing to meet inclusion criteria, or referring to single occupational groups were removed from the SPQ, and some rephrasing of the remaining items was undertaken. This produced an eight-item questionnaire (see data supplement to the online version of this paper). The SSQ was piloted with 20 service users and minor adjustments made to wording and layout.
The psychometric properties of the scale were tested in three separate samples. Samples 1 and 2 were used for testing validity and internal consistency. Sample 1 consisted of 1665 new service users attending NHS services in Leeds and Newcastle. The assessment package comprised self-reported substance use, dependence as measured by the Leeds Dependence Questionnaire (LDQ; Raistrick et al, 1994) and psychopathology as measured by the General Health Questionnaire (GHQ; Goldberg & Williams, 1988). Sample 2 consisted of 6732 new service users attending the Leeds service. For this sample the assessment package had been updated to measure psychopathology using the Clinical Outcomes in Routine Evaluation -Outcome Measure (CORE-OM; Evans et al, 2002). Sample characteristics are presented in Table 1.
Sample 3 was used for the purpose of estimating test-retest reliability. The scale was administered as part of a package of three questionnaires to 61 respondents attending an NHS clinic for treatment of their heroin use. The sample consisted of 40 males (65.6%) and 21 females (34.4%), with a mean age of 25.7 years (s.d.=4.9, range 17-37). The mean time between first and second completion of the scale was 5.3 days (s.d.=3.75) with a minimum of 2 days and a maximum of 14 days.

Item analysis
An item analysis of samples 1 and 2 was conducted to examine the homogeneity and internal consistency of the scale. Each of the eight items correlated with total SSQ score (Spearman's rho ranged from 0.57 to 0.66 and 0.60 to 0.68 for samples 1 and 2 respectively) and all were significant at the P50.001 level. Cronbach's a was 0.73 and 0.79 respectively and similar values held with each item deleted. In order to examine the factor structure of the scale, responses given by sample 2 were subjected to exploratory principal components analysis, which yielded three factors with an eigen value 41. The first of these had an eigen value of 3.3 accounting for 41% of the variance with item loadings ranging from 0.59 to 0.69.
Factor 2 was specific to accommodation and factor 3 to employment and finance, therefore it was felt that these items did not make sense as stand-alone factors (see Table 2) and so a single factor is a better interpretation of the data.

Test-retest reliability
A test-retest reliability study was conducted with sample 3. The correlation between the total SSQ scores at the two data collection points was 0.85 and the mean correlation between individual item scores was 0.72. Mean scores at the first and second data collection points were compared in order to eliminate the possibility that scores have changed but were still correlated. No significant difference was found in the mean total score for the scale between the first and second scale completion (9.3 and 8.7 respectively) indicating that the responses had remained constant.

Validity
The relationships between the SSQ and other variables were examined, using a correlation technique to demonstrate construct validity. The expectation was that there Raistrick et al Social Satisfaction Questionnaire for substance use disorders original papers  would be significant but modest correlations given the distinctiveness of the domains of dependence, psychological health and social satisfaction. This was found to be the case. For sample 1 the SSQ was negatively and significantly correlated with measures of psychopathology (r=0.38, P50.001) and dependence (r=0.28, P50.001). The above findings were confirmed by a stepwise multiple regression analysis, with SSQ score as the dependent variable and age, gender, GHQ score (psychopathology), LDQ score (dependence) and main reason for referral (alcohol or other drug) as independent variables. Results of this analysis showed that GHQ score and LDQ score predicted SSQ score (GHQ: t=9.01, P50.0001; LDQ: t=3.26, P50.001). Thus, service users with higher psychopathology and dependence were most likely to have a low score for social satisfaction. As expected gender, main reason for referral and treatment site were not predictors of SSQ.
The correlations were reassessed on sample 2 which was large enough to analyse according to the drug groups of opiates, stimulants, cannabis and alcohol. Again the expectation was that there would be low-order but significant correlations between the SSQ and other assessment measures. This was found to be the case. The SSQ was negatively correlated with the LDQ in the range 70.16 to 70.28 and with CORE-OM in the range 70.40 to 70.52 (P=0.001) ( Table 3). The scale achieves a close to normal distribution across sample 2, showing a capability to measure the full range of scores.

Discussion
The aim of this study was to develop and validate an instrument to measure a dimension of social functioning that was meaningful in a help-seeking population and suitable for use in routine practice. We have presented evidence that the SSQ has robust internal psychometric properties. Validation of a subjective construct such as satisfaction is problematic, however, we have shown that the SSQ is a unidimensional construct which measures a separate domain to that of dependence and psychological morbidity and is independent of unrelated variables.
The case for a new scale is based on the need for a social domain measure that has clinical utility and acceptability in a busy clinic environment. The scale has face validity in pointing to social domains that are likely to be important to and engage service users in treatment. In sample 2 we combined the SSQ with the LDQ, the CORE-OM and the substance use section of the Maudsley Addiction Profile (MAP; Marsden et al, 1998) to create a comprehensive outcome measurement package which takes approximately 15-20 min to complete. Services less concerned with psychological morbidity can save time by replacing CORE-OM with the MAP measure for this domain. There is a strong argument for using a standard assessment package. For example, the Addiction Severity Index (McLellan et al, 1980) is popular but its reliability and validity have been questioned (MÌkelÌ, 2004). There may be a stronger case for selecting from a menu of standardised outcome measures, each with strong reliability and validity, which can be packaged to suit local need, and it was with this approach in mind that we developed the SSQ.
The SSQ has been extensively used and found to have clinical utility in two NHS substance misuse services, however, there needs to be further testing of concurrent validity and in different service user groups, beyond problems of substance misuse. Therapists familiar with scale items can incorporate responses into treatment, for example to elicit motivational statements (Ford, 2003) and include service user concerns expressed through the SSQ in care plans. A critical test of the validity and utility of the SSQ will be its ability to predict treatment outcome in combination with other measures in the package and this will be reported in due course.

Declaration of interest
None.  Despite extensive evidence that attention-deficit hyperactivity disorder (ADHD) and developmental coordination disorder commonly present as overlapping disorders, it is not clear whether clinicians routinely enquire about movement difficulties when assessing children with suspected ADHD. We describe a survey that examines knowledge and practice of child and adolescent psychiatrists (n=107) and paediatricians (n=51) in this area.

RESULTS
Results show that 67.3% of child and adolescent psychiatrists compared with 15.7% of paediatricians claimed to have poor or very poor knowledge of developmental coordination disorder, and 28% compared with 5.9% respectively reported that they never or only occasionally ask about motor difficulties.

CLINICAL IMPLICATIONS
Child and adolescent psychiatrists should consider routine screening for developmental coordination disorder when assessing for ADHD. Further training in assessment of developmental coordination disorder is recommended to facilitate this.
Greater awareness of attention-deficit hyperactivity disorder (ADHD) among schools, parents and primary care has increased the flow of referrals into specialist child and adolescent mental health services (CAMHS) across the UK. In the UK Office of National Statistics study, a diagnosis of ADHD was shown to be the most common reason for follow-up in specialist CAMHS (Meltzer et al, 2000). This is in line with research from the USA that showed that 30-50% of referrals to CAMHS were specifically related to ADHD (Barkley, 1996). There is clear evidence of association or comorbidity of ADHD with a number of other psychiatric conditions, including oppositional defiant disorder, conduct disorder, and depression and anxiety disorders (Loeber, 1982;Barkley et al, 1990;Taylor et al, 1991), and these should be routinely considered at the time of assessment.
In addition, it has been shown that approximately half of children with ADHD have developmental coordination disorder (Kadejs˛& Gillberg, 1999). Developmental coordination disorder is a term used to describe motor coordination difficulties that have an impact on daily living and education. Children who meet DSM-IV criteria for ADHD (any of the subtypes) and developmental coordination disorder (American Psychiatric Association, 1994) are sometimes also described as having disorders of attention, motor control and perception or 'DAMP' ; this is a Scandinavian construct that attempts to make sense of the comorbidity issues (Gillberg, 2003). Children with disorders of attention, motor control and perception have clinically significant impairment in social or academic functioning, or both, but do not have severe learning disability or cerebral palsy.
There is evidence to support the rationale for asking about motor difficulties in ADHD clinics. For example, Tervo et al (2002) showed that children with ADHD and developmental coordination disorder were more likely to have the severe combined type of ADHD and other neurodevelopmental and behavioural problems than children with ADHD alone. In addition, Hellgren et al (1993) in a long-term follow-up study showed that patients with the combination of ADHD and developmental coordination disorder had a greater risk of long-term psychiatric morbidity than those with developmental coordination disorder alone. Recognising the combination of ADHD and developmental coordination disorder might also be important in terms of responsiveness to different treatment approaches. For example, Blondis (1999)  ORFORD, J., HODGSON, R., COPELLO, A., et al (2006) The clients'perspective on change during treatment for an alcohol problem: qualitative analysis on follow-up interviews in the UK Alcohol TreatmentTrial. Addiction, 101, 60-68. RAISTRICK, D., BRADSHAW, J.,TOBER, G., et al (1994) Development of the Leeds Dependence Questionnaire (LDQ): a questionnaire to measure alcohol and opiate dependence in the context of a treatment evaluation package. Addiction, 89, 563-572. TOBER, G., BREARLEY, R., KENYON, R., et al (2000) Measuring outcomes in a health service addiction clinic. Addiction Research,8,[169][170][171][172][173][174][175][176][177][178][179][180][181][182]