Skip to main content Accessibility help
×
Home
Hostname: page-component-5bf98f6d76-rtbc9 Total loading time: 29.569 Render date: 2021-04-22T11:53:51.915Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false, "newCitedByModal": true }

Validation of the Short Obsessive–Compulsive Disorder Screener (SOCS) in children and adolescents

Published online by Cambridge University Press:  02 January 2018

José A. Piqueras
Affiliation:
Universidad Miguel Hernández de Elche, Alicante, Spain
Tíscar Rodríguez-Jiménez
Affiliation:
Universidad Miguel Hernández de Elche, Alicante, Spain
Ana G. Ortiz
Affiliation:
Department of Child and Adolescent Psychiatry and Psychology, Hospital Clínic de Barcelona, Barcelona, Spain
Elena Moreno
Affiliation:
Department of Child and Adolescent Psychiatry and Psychology, Hospital Clínic de Barcelona, Barcelona, Spain
Luisa Lázaro
Affiliation:
Department of Child and Adolescent Psychiatry and Psychology, Hospital Clínic de Barcelona, IDIBAPS. CIBERSAM, University of Barcelona, Barcelona, Spain
Antonio Godoy
Affiliation:
Universidad de Málaga, Málaga, Spain
Corresponding
E-mail address:
Rights & Permissions[Opens in a new window]

Abstract

Background

The Short Obsessive–Compulsive Disorder Screener (SOCS) is recommended by the National Institute for Health and Care Excellence as a suitable and validated screening tool for 11- to 15-year olds. Despite its excellent sensitivity and specificity in detecting obsessive–compulsive disorder (OCD), it has limitations.

Aims

To empirically examine whether the SOCS is suitable for assessing OCD symptoms across a wide age range of children and adolescents and to provide new data about its psychometric properties.

Method

Participants were 94 patients (9–19 years) with OCD, and 880 healthy controls.

Results

The results supported the SOCS' unidimensional factor structure and metric invariance across samples. It showed good reliability in terms of internal consistency and temporal stability. Furthermore, it had significantly high correlations with other OCD measures and an acceptable sensitivity and specificity for detecting OCD.

Conclusions

The SOCS is a brief screening tool suitable for detecting OCD in children and adolescents.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © The Royal College of Psychiatrists 2015

Obsessive–compulsive disorder (OCD) symptoms in children and adolescents are an underestimated and understudied problem. Recently, OCD has become part of a new chapter – entitled ‘Obsessive compulsive and related disorders’ – in the DSM-5. 1 OCD is a mental disorder characterised by the presence of obsessions and/or compulsions that produce intense distress and disability. 1 According to epidemiological studies, OCD often begins in childhood or early adolescence. Reference Farrell and Barrett2 Prevalence rates in children and adolescents can reach up to 3% for OCD Reference Canals, Hernández-Martínez, Cosi and Voltas3 and up to 19% for the subclinical symptoms. Reference Canals, Hernández-Martínez, Cosi and Voltas3Reference Orgilés, Méndez, Espada, Carballo and Piqueras5

Therefore, the development and dissemination of effective interventions for OCD are critically important, as is the need for early detection and intervention to improve outcomes. For example, regarding assessment and treatment of OCD, the National Institute for Health and Care Excellence (NICE) recommends the routine screening of young people at risk in general practice or other settings where they may present for help. Reference Krebs and Heyman6 Such screening requires short, easy-to-use, highly sensitive, reasonable and widely available measures. Reference Uher, Heyman, Mortimore, Frampton and Goodman7

Thus, recently, there has been a considerable increase in assessment tools for OCD and obsessive–compulsive symptoms in children and adolescents. Reference Overduin and Furnham8 For example, according to a recent review, Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 the Children's Yale-Brown Obsessive–Compulsive Scale (CY-BOCS) Reference Scahill, Riddle, McSwiggin-Hardin, Ort, King and Goodman10 is the gold standard in the field of psychiatric clinical assessment, because of its excellent psychometric properties; however, it is a lengthy, clinician-administered, interview-based instrument, making it unsuitable for use in community settings; moreover, its self-report format is too lengthy. Other measures such as the Obsessive Compulsive Inventory – Child Version (OCI-CV), Reference Foa, Coles, Huppert, Pasupuleti, Franklin and March11 the Children's Obsessional Compulsive Inventory (CHOCI), Reference Shafran, Frampton, Heyman, Reynolds, Teachman and Rachman12 the Children's Florida Obsessive–Compulsive Inventory (C-FOCI), Reference Storch, Khanna, Merlo, Loew, Franklin and Reid13 the Obsessive–Compulsive Subscale (OCS) of the Child Behavior Checklist (CBCL) Reference Achenbach14 and the Child Saving Inventory (CSI) Reference Storch, Muroff, Lewin, Geller, Ross and McCarthy15 are promising self-report measures; however, all of them present some limitations. These limitations concern features such as the tools’ length, ease of use and availability. Reference Rodríguez-Jiménez, Godoy, Piqueras, Gavino, Martínez-González and Foa16 Furthermore, most researchers assert that evidence-based assessment (EBA) is a first step towards the development of psychopathological studies and assessment of the effectiveness of empirically validated treatments. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9,Reference Rodríguez-Jiménez, Godoy, Piqueras, Gavino, Martínez-González and Foa16 Specifically, the NICE recommends the Short Obsessive–Compulsive Disorder Screener (SOCS). Reference Krebs and Heyman6,Reference Uher, Heyman, Mortimore, Frampton and Goodman7

Uher et al Reference Uher, Heyman, Mortimore, Frampton and Goodman7 designed and validated the SOCS to provide a simple screening tool suitable for general practice and community settings. The SOCS is a seven-item self-report tool with a three-point response format (0 [‘no’], 1[‘a bit’] and 2 [‘a lot’]). The first five items were adapted from the five most discriminant items of the 44-item child version of the Leyton Obsessional Inventory (LOI) Reference Berg, Rapoport and Flament17 and assess common symptoms including checking, touching, cleanliness/washing, repeating and exactness. Two additional questions assess the associated impairment and resistance. The SOCS score is calculated by summing the scores for all seven items. With a sample of 11- to 15-year-old youth, Uher et al found that the tool had good internal consistency, a unidimensional factor structure and excellent sensitivity for detecting OCD in children; furthermore, its specificity was good for children without psychiatric diagnoses but poor for a psychiatric sample. Although Uher et al's study highlights the tool's uses, the SOCS still has limitations concerning certain psychometric properties, such as factorial structure invariance, test–retest reliability, and the lack of evidence of convergent/divergent validity. Furthermore, all psychometric properties must be investigated in a broader age range than 11–15 years.

Consequently, our main objective was to further validate the SOCS while empirically examining whether it is useful in assessing obsessive–compulsive symptoms across a wide age range of clinical and community samples of children and adolescents.

Method

Participants

The clinical sample comprised 94 consecutive child and adolescent patients (46 males; 48 females) with a current diagnosis of OCD according to the DSM-IV criteria. 18 Their mean age was 14.62 years (s.d.=2.65; range 9–19). All patients were recruited from the Child and Adolescent Psychiatry and Psychology Unit of Hospital Clínic de Barcelona, and from the Child and Adolescent Psychological Clinical Unit of Universidad Miguel Hernández de Elche. Axis I diagnoses were ascertained using the Schedule for Affective Disorders and Schizophrenia for School-Age Children – Present and Lifetime Version (K-SADS-PL), Reference Kaufman, Birmaher, Brent, Rao, Flynn and Moreci19 which has been validated in Spanish. Reference Ulloa, Ortiz, Higuera, Nogales, Fresán and Apiquian20 The children and their parents were informants. Exclusion criteria were psychiatric comorbidity with a psychotic disorder or autism spectrum disorder, and an IQ of less than 70. Patients could have another comorbid Axis I disorder, but OCD needed to be the primary disorder. Specifically, we found the following rates of associated secondary disorders: generalised anxiety disorder 16%, Tourette's disorder 4.3%, attention-deficit hyperactivity disorder 4.3%, social phobia 2.1% and major depression 2.1%.

Written informed consent was obtained from all participants and families following an explanation of the procedures involved.

The community sample consisted of 880 students (437 males; 443 females) from five schools in Levante, eastern Spain. Their mean age was 13.21 years (s.d.=2.19; range 10–18). All the schools volunteered for the study. After obtaining permission from the principals and psycho-pedagogical departments of the schools, all students agreed to participate.

Measures

SOCS Reference Uher, Heyman, Mortimore, Frampton and Goodman7

This was translated into Spanish with translation and backtranslation. Reference Hambleton, Merenda and Spielberger21 The Spanish version of the SOCS was administered to a pilot sample of 10-year-olds to ensure that the item content could be understood by young children. This Spanish translation was authorised in 2009 by Isobel Heyman, an author of the original scale.

Ad hoc sociodemographic sheet

This was designed to collect data on age, gender and geographic area of residence.

Spanish version of the OCI-CV

This comprises 21 items scored on a 3-point Likert scale (0 [‘never’], 1 [‘sometimes’] and 2 [‘always’]). The OCI-CV provides six subscale scores – doubting/checking, obsessing, hoarding, washing, ordering and neutralising – and a total score. Reference Foa, Coles, Huppert, Pasupuleti, Franklin and March11 The psychometric properties of the English Reference Foa, Coles, Huppert, Pasupuleti, Franklin and March11 and Spanish Reference Rodríguez-Jiménez, Godoy, Piqueras, Gavino, Martínez-González and Foa16 versions have been reported.

C-FOCI Reference Storch, Khanna, Merlo, Loew, Franklin and Reid13

This was developed from the original version of the FOCI for adults to assess obsessive–compulsive symptoms in 7- to 20-year-old children and adolescents. It consists of 22 items in two subscales: symptom checklist and OCD severity scale. It has shown adequate psychometric properties both for English Reference Storch, Khanna, Merlo, Loew, Franklin and Reid13 and Spanish speaking children and adolescents (further details of the Spanish validation study available from the authors on request).

Revised Child Anxiety and Depression Scale (RCADS-30) Reference Sandin, Chorot, Valiente, Chorpita and Sandín22

This is a reduced 30-item version of the RCADS, Reference Chorpita, Yim, Moffitt, Umemoto and Francis23 a self-report for evaluating anxiety and depression in children and adolescents. The scale comprises 30 items and in six subscales for evaluating symptoms of the following conditions: panic disorder, social phobia, separation anxiety disorder, generalised anxiety disorder, OCD and major depressive disorder. The scale ranges from 0 to 3 (corresponding to never, sometimes, often and always, respectively). The scale has shown excellent psychometric properties, equivalent to the full version, with a Spanish population. Reference Sandin, Chorot, Valiente, Chorpita and Sandín22

CY-BOCS

This is a clinician-administered scale involving a semi-structured interview for assessing the severity of obsessive–compulsive symptoms in children. Reference Scahill, Riddle, McSwiggin-Hardin, Ort, King and Goodman10 The scale includes a list of subdomains used to identify the content of the patient's obsessions and compulsions. These symptoms are then rated on a 5-point scale (0 [‘no symptoms’] to 4 [‘extreme symptoms’]) covering five areas: time spent on the symptoms per day, interference caused by symptoms, level of distress, level of resistance and control. Summing these scores for each symptom produces scores for the severity of various obsessions and compulsions. A total severity score (ranging from 0 to 40) can also be obtained, with a higher score indicating greater severity. A total severity score of 16 serves as the cut-off point for an OCD diagnosis.

Procedure

The study was approved by the Research and Ethics Committees of the Universidad Miguel Hernández de Elche and the Hospital Clínic de Barcelona. Experienced psychologists with master's degree or doctorate in psychology administered the SOCS and the other self-administered questionnaires to the clinical and community samples at the time of study entry, along with the K-SADS-PL and CY-BOCS exclusively to the clinical sample. They administered the questionnaires 4 weeks after the first evaluation again. The psychologists gave the participants clear instructions concerning how to indicate their responses and they were reassured that it was not a test and that there were no right or wrong answers, and individual assistance to those students who needed it. In order to prevent the introduction of biases in the collection of information, four different balanced protocols were applied. For it, the questionnaires were randomly ordered in each protocol. The average time of application of the assessment was about 50 minutes and no cases of fatigue were detected.

Data analysis

Six sets of analyses based on the ordinal nature of the variables (in all cases, we used ordinal and not ratio scales) were conducted.

First, we examined the item distribution and frequency as well as the factor structure of the SOCS for the clinical and community samples. Then, based on the results of Uher et al, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 the hypothesis that the SOCS has a single-factor structure was tested by confirmatory factor analysis (CFA) performed simultaneously for both the clinical and community samples. We also tested the alternative hypothesis that the SOCS items are independent, and tested whether the SOCS exhibits metric invariance. Testing metric invariance was conducted once, not systematically (i.e. we did not check the configuration invariance first; instead, we first checked the invariance of the factor loadings, after which we checked the measurement errors followed by the intercepts). Therefore, in all cases, we introduced all the restrictions simultaneously, which enabled us to check the metric invariance between the samples with a single calculation. We used polychoric correlation matrices and the diagonally weighted least squares (DWLS) method in all cases (LISREL 8.8, DWLS procedure). Last, we used the following indices as goodness-of-fit measures Reference Schermelleh-Engel, Moosbrugger and Müller24 : the Satorra-Bentler chi-square, a root mean square error of approximation (RMSEA) less than or equal to 0.8 and a confirmatory factor index (CFI), goodness of fit index (GFI) and non-normed fit index (NNFI) (or Tucker-Lewis index) higher than 0.95. The CFI was also used to compare the basic model (total sample) with the metric invariance model in the case of the two samples. According to Cheung and Rensvold, Reference Cheung and Rensvold25 the invariance between samples is admissible when the difference between the CFIs (ΔCFI) is less than or equal to 0.01. They also assert that ΔCFI is a better estimator of invariance admissibility than Δχ Reference Farrell and Barrett2 , which is the index used by some researchers.

McDonald's omega Reference McDonald26 and the greatest lower bound to reliability index (GLB) Reference Woodhouse and Jackson27 were used to estimate the internal consistency of the SOCS, as they are better estimators of reliability than the Cronbach's alpha is. Reference Schweizer28 We also administered the SOCS twice after an interval of 4 weeks to examine test–retest reliability.

Discriminative validity was examined using the Mann–Whitney U-test and the probability of superiority Reference Erceg-Hurn and Mirosevich29 was used to estimate the effect size of the differences between the samples.

We also conducted analyses of convergent/divergent validity by calculating Spearman's rho coefficients. We used Cohen's criteria Reference Cohen30 to evaluate the effect sizes of the correlations: small (0.20) and large (≥0.50). Reference Cohen30,Reference Lipsey and Wilson31

To determine the accuracy of the scales in terms of their ability to correctly classify children and adolescents with or without OCD, we conducted a receiver operating characteristic (ROC) curve analysis, wherein subscales with areas under the curve (AUCs) between 0.70 and 0.80 were considered adequate.

Then, the predictive validity of the SOCS for a K-SADS-PL diagnosis of OCD was calculated by binary logistic regression analyses. The diagnoses according to the K-SADS-PL were the outcome variables (OCD or non-OCD), and the predictor variable was SOCS score.

Results

Item analysis

The means of the item scores for the clinical sample ranged from 0.66 (item 4) to 1.31 (item 1), and the standard deviations ranged from 0.59 (item 7) to 0.80 (item 2). The distribution for almost all the item scores showed skewness indices (−0.57 to 0.70) and kurtosis indices (−1.33 to 0.46) near zero. Overall, the average percentage of each response across the items scores was as follows: ‘no’ (26%), ‘a lot’ (34.1%) and ‘a bit’ (39.9%). In the community sample, the means of the items scores ranged from 0.55 (item 6) to 1.05 (item 5), and the standard deviations ranged from 0.63 (item 1) to 0.75 (item 3). For almost all the items scores, skewness (−0.09 to 0.80) and kurtosis indices (−1.20 to 0.46) were near zero. Overall, the average percentage of each response across the items scores was as follows: ‘a lot’ (18.4%), ‘no’ (37.8%) and ‘a bit’ (43.8%).

CFA

As can be seen in Table 1, for the total sample (clinical and community), the fit indices indicated that the model fits the data reasonably well, and we found metric invariance between the samples (configurational invariance and invariance of factor loadings, measurement errors and intercepts). The RMSEA was <0.8, and the CFI, GFI and NNFI were >0.90; the ΔCFI was 0.01 (0.98–0.97).

Table 1 Confirmatory factor analysis (total sample: N=974) and multi-group confirmatory factor analyses (diagonally weighted least squares; polychoric correlation matrix) for children from clinical setting (n=94) and children from the community (n=880).

Sample/model χ2 df RMSEA (90% CI) CFI GFI NNFI IFI RFI
Total sample
 Independent 1969.59 21
  One factor 55.55 14 0.06 (0.04–0.07) 0.98 0.99 0.97 0.98 0.96
Metric invariance
Clinical community
 Independent 1779.37 42
 One factor 109.59 55 0.05 (0.03–0.06) 0.97 Clinical: 0.90
Community: 0.99
0.98 0.97 0.95

χ2, Satorra-Bentler's chi-squared; RMSEA, root mean square error of approximation; CFI, comparative fix index; GFI, good-of-fit index; NNFI, non-normed fit index; IFI, incremental fit index; RFI, relative fit index.

Table 2 shows the degree of the relationship (standardised lambda weights) for each item score with a single factor. As is evident, all item weights are above 0.50, except for the weight of item 2 (0.35).

Table 2 Item content, item factor loading (lambda). Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).

Item Item content Lambda
1 Does your mind often make you do things like checking or touching things or counting things, even though you know you don't really have to? 0.50
2 Are you particularly fussy about keeping your hands clean? 0.35
3 Do you ever have to do things over and over a certain number of times before they seem quite right? 0.61
4 Do you ever have trouble finishing your school work or chores because you have to do something over and over again? 0.56
5 Do you worry a lot if you've done something not exactly the way you like? 0.54
6 Do these things interfere with your life? 0.78
7 Do you try to stop them? 0.51

Reliability

The internal consistency (McDonald's omega and GLB) of the SOCS for both samples is provided in Table 3. The reliability indices for the total score were higher in the clinical sample.

Table 3 Means (s.d.), differences between clinical and community samples and reliability (McDonald's omega and GLB) for the SOCS. Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).

Sample Statistic Total score
Total sample Omega 0.75
GLB 0.80
Mean 5.82
s.d. 2.92
Clinical sample Omega 0.79
GLB 0.88
Mean 7.56
s.d. 3.15
Community sample Omega 0.74
GLB 0.79
Mean 5.64
s.d. 2.83
Mann–Whitney U-test 27093*
Z −5.53
PS effect size 0.67

GLB, greatest lower bound to reliability; PS, probability that a randomly sampled score from one population is larger than a randomly sampled score from a second population.

* P=0.01.

Regarding test–retest reliability, the two mean SOCS scores of the clinical sample were 7.56 (s.d.=3.15) and 6.58 (s.d.=3.18), with a rho of 0.79. In the community sample, the mean scores were 5.64 (s.d.=2.83) and 4.61 (s.d.=2.83), with a rho of 0.63. In both samples, the correlations were statistically significant (P<0.01).

Descriptive statistics of the scales and discriminative validity

Table 3 shows the means and standard deviations of the SOCS for the total sample, for the clinical sample and for the community sample. In general, the students from the clinical sample obtained higher scores on the SOCS than did those from the school sample (probability of superiority 0.67, which is considered a medium effect size according to Grissom Reference Nunnally and Bernstein33 ).

Analysis of convergent/divergent validity

The results indicated that in both samples, SOCS score was significantly and positively correlated with the scores on well-established measures of child OCD (see Table 4). The effect sizes were large in the clinical sample (r=0.50–0.80) and medium to large in the community sample (r=0.47–0.72). The only exception, in both samples, was the OCI-CV's Hoarding scale, which had a moderate correlation with the SOCS. In the clinical sample, the total severity scale of the CY-BOCS showed a moderate association with the SOCS.

Table 4 Convergent/divergent validity of the SOCS. Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).

Measures SOCS
Clinical Community
CY-BOCS total severity score 0.38**
C-FOCI symptom checklist 0.65** 0.61**
C-FOCI severity scale 0.57** 0.49**
OCI-CV checking 0.68** 0.62**
OCI-CV obsessing 0.53** 0.47**
OCI-CV hoarding 0.43** 0.43**
OCI-CV washing 0.53** 0.54**
OCI-CV ordering 0.50** 0.49**
OCI-CV neutralising 0.58** 0.47**
OCI-CV total score 0.80** 0.72**
RCADS-30 separation anxiety 0.26* 0.36**
RCADS-30 social phobia 0.43** 0.40**
RCADS-30 generalised anxiety 0.46** 0.43**
RCADS-30 panic disorder 0.44** 0.42**
RCADS-30 obsessive–compulsive disorder 0.63** 0.52**
RCADS-30 major depression 0.49** 0.41**
RCADS-30 total score 0.61** 0.57**

SOCS, Short OCD Screener; OCI-CV, Obsessive–Compulsive Inventory – Child Version; RCADS-30, Revised Child Anxiety and Depression Scale; C-FOCI, Children's Florida Obsessive–Compulsive Inventory; CY-BOCS, Children's Yale-Brown Obsessive Compulsive Scale.

* P<0.05; **P<0.01.

With regard to related construct measures such as anxiety and depression symptoms, all the correlations were significant with medium effect sizes (r=0.26–0.49). The only exception in both samples was the total score on the RCADS, which showed a large effect size (see Table 4).

Sensitivity and specificity of the SOCS

The AUC of the SOCS was 0.67 (s.e.=0.03, P<0.01), suggesting that there is a 67% probability that a child or adolescent with OCD will have a higher score on the SOCS than a non-OCD child/adolescent. Table 5 shows the sensitivity and specificity of the SOCS.

Table 5 Sensitivity, specificity, Youden index and accuracy of the SOCS. Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).

Cut-off Sensitivity Specificity PPV NPV Youden index Accuracy
1 0.98 0.02 0.11 0.90 0.00 0.89
2 0.97 0.08 0.04 0.90 0.05 0.84
3 0.95 0.15 0.04 0.89 0.09 0.78
4 0.90 0.26 0.04 0.89 0.16 0.68
5 0.83 0.37 0.05 0.88 0.20 0.59
6 0.76 0.49 0.05 0.86 0.25 0.49
7 0.61 0.60 0.07 0.86 0.21 0.40
8 0.51 0.73 0.07 0.83 0.24 0.29
9 0.37 0.82 0.08 0.82 0.19 0.22
10 0.30 0.91 0.08 0.74 0.21 0.15
11 0.21 0.96 0.08 0.62 0.18 0.11
12 0.12 0.98 0.09 0.56 0.10 0.10
13 0.04 1.00 0.09 0.50 0.04 0.10
14 0.02 1.00 0.09 0.00 0.02 0.09

PPV, positive predictive value; NPV, negative predictive value; accuracy (informedness), percentage of children correctly classified.

We selected cut-off scores to provide the best balance between sensitivity and specificity for each measure. A SOCS cut-off score of 7 showed a sensitivity of 61% and a specificity of 60%, a Youden index of 0.21 and an accuracy or informedness (percentage of participants correctly classified) of 40%. However, according to the Youden index and accuracy percentage, a score of 6 was more sensitive but less specific, making it a possibly more suitable cut-off score for detecting OCD.

Predictive validity

The predictive model accounted for 8% (Nagelkerke's R 2) of the variance in OCD diagnoses and resulted in a classification accuracy of 90.3%. The SOCS was related to diagnostic status (Wald (1)=34.66, P<0.01) such that higher SOCS scores were better predictors of OCD diagnoses. The odds ratio was significant, OR=1.26 (95% CI 1.17–1.36).

Discussion

The NICE recommends the SOCS as a suitable and validated screening tool for youth between 11 and 15 years. In spite of its excellent sensitivity and specificity in detecting OCD in non-clinical children, the research on SOCS does show some gaps. For example, the invariance of the SOCS’ factorial structure is unknown, there is some ambiguity regarding test–retest reliability estimations and there is a lack of evidence concerning its convergent/divergent validity. Moreover, all such properties must be determined for a broader age range than 11- to 15-year-olds. Thus, we provided new data about these psychometric properties to determine whether the SOCS is suitable for assessing OCD symptoms across a wider age range of children and adolescents.

Our results supported the factorial validity of the SOCS, confirming the unidimensional structure previously found. Reference Uher, Heyman, Mortimore, Frampton and Goodman7 The CFA showed that the data fitted adequately to the unidimensional measurement model, although the factor loadings of item 2 were below 0.40. This small load factor for item 2 may be due to hypothetical understanding problem by the youngest children.

With regard to invariance, the SOCS presented the expected metric invariance between the samples, as indicated by an RMSEA below 0.06 and CFI, GFI and NNFI values over 0.90; these values also indicated acceptable fit according to Schermelleh-Engel et al Reference Schermelleh-Engel, Moosbrugger and Müller24 Thus, in both samples, we found the same factor and equivalent factor loadings, measurement errors and intercepts.

With respect to reliability estimation, the internal consistency coefficients of the SOCS were between moderate and high (0.74–0.88), namely, above the recommended value of 0.70 proposed by Nunnally & Bernstein. Reference Nunnally and Bernstein33 These data indicate that the scale has a high internal consistency, which supports the existence of a single dimension. The data are consistent with a previous study carried out with the SOCS, which also indicated good internal consistency (Cronbach's alpha 0.85). Our data are also consistent with those concerning other EBA measures for OCD, such as the CY-BOCS, OCI-CV, CHOCI, C-FOCI, OCS and CSI. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9

The SOCS also showed an adequate temporal stability, as we found positive correlations for both the clinical (0.79) and community (0.63) samples after 4 weeks. According to reliability studies, values between 0.50 and 0.70 indicate moderate reliability, and values between 0.80 and 0.90 indicate good reliability. Other studies on self-report measures for OCD show similar short-term temporal stability. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9

The mean score of the clinical sample was higher than that of the community sample (M=5.64; s.d.=2.83), with a medium effect size, supporting the discriminative validity of the SOCS. In addition, the mean score for the OCD group was lower than that reported by Uher et al Reference Uher, Heyman, Mortimore, Frampton and Goodman7 for an OCD group (M=9.7, s.d.=2.2) but higher than the mean for Uher et al's healthy group (M=3, s.d.=2.3), mixed community group (M=3.3, s.d.=2.5) and psychiatric control group (M=5.8, s.d.=2.8). Compared with the results of Uher et al, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 our results revealed somewhat low levels of obsessive–compulsive symptoms in the clinical sample but high levels in the community sample. This result could be because we obtained SOCS scores from samples with mean ages and an age range higher than was the case in Uher et al Reference Uher, Heyman, Mortimore, Frampton and Goodman7 and from a community sample larger than that in Uher et al. Reference Uher, Heyman, Mortimore, Frampton and Goodman7 Furthermore, we calculated the mean of the community sample without eliminating the scores of children who could have met the criteria for OCD (because we did not evaluate these criteria in this sample), whereas Uher et al eliminated children who met OCD criteria from their community sample. Therefore, in our sample, children with potential OCD were also considered when calculating the mean SOCS score. However, our outcomes are similar to findings concerning other OCD screening instruments (i.e. the LOI-CV), which displayed sensitivity values between poor and adequate. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 This conclusion can be defended as SOCS scores have metric invariance and the same intercepts in clinical and non-clinical populations, an issue supported in our study.

Regarding the concurrent validity, the confirmed relationships between the SOCS and the three measures with the highest empirical support – the CY-BOCS, OCI-CV and C-FOCI – as indicated by Iniesta-Sepúlveda et al, Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 also supports the validity of the SOCS. A large positive relationship was found between the SOCS and the total scores on the OCI-CV and the severity scale of the C-FOCI, whereas a moderate association was found between the SOCS and the total severity scale of the CY-BOCS in the clinical sample, following Cohen's effect size criteria. Reference Cohen30,Reference Lipsey and Wilson31 However, there were also moderate associations between the SOCS and specific anxiety and depression symptom measures. One explanation for this finding is the symptom overlap between depression, anxiety and OCD disorders, which other studies have also found. Reference Sun, Li, Buys and Storch34

Finally, we examined the sensitivity and specificity of the SOCS by using ROC analysis. Overall, the results indicated an AUC of 0.67. Values around 0.70 are considered to represent acceptable discriminatory ability. According to our data, a SOCS score above 7 was initially recommended because it provides an optimal balance between the percentages of true positives and negatives in real cases. This cut-off score led to an acceptably low percentage of adolescents incorrectly identified as having OCD (specificity 60%) and an adequate proportion of adolescents being overlooked in terms of heightened OCD symptoms (sensitivity 61%). However, because the main purpose of the SOCS is to serve as a screening measure of OCD, the best cut-off score is 6, with a Youden index of 0.25. This SOCS score could be advantageous for screening adolescents in clinical settings, as it would preclude the overlooking of adolescents requiring further assessment. However, depending on the intended purpose of the SOCS, administrators could choose a higher or lower cut-off score. For example, a more conservative cut-off score of 8–9 might be justified when using the subscales of the SOCS for research purposes in order to avoid false positives (see Table 5). Our sensitivity/specificity results were lower than those reported by Uher et al, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 but again, the sample differences could explain this discrepancy. However, our outcomes are similar to findings concerning other OCD screening instruments (i.e. the LOI-CV), which displayed sensitivity values between poor and adequate. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 However, this aspect of OCD measures presents limited psychometric evidence according to Iniesta-Sepúlveda et al. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 Furthermore, although the community sample was assumed to have no OCD cases, it is possible that this sample contained such cases, which could explain our relatively low sensitivity and specificity.

Regarding the predictive validity of the SOCS, the results of a binary logistic regression analysis indicated that the scale was a significant predictor of OCD diagnosis (classification accuracy of 90.3%). This result is consistent with the findings of Uher et al's study with the SOCS, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 which resulted in the development of a screening tool suitable for accurately classifying OCD.

Limitations

Some limitations should be noted. First, our study did not examine the psychometric properties of SOCS with a large clinical sample. Second, the sensitivity and specificity in differentiating children with and without OCD with a sample of children with the disorder and a non-clinically interviewed community-based sample. And third, its convergent and discriminant validity with other self-report measures as well as the relationship of its scores with those from other assessment procedures (e.g. information deriving from parents or teachers).

Use of the SOCS

In conclusion, the SOCS is shorter than other self-report tools (for further details, see the review by Iniesta-Sepúlveda et al Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 ) and has comparable psychometric properties. Following the classification of EBAs, Reference Cohen, Cohen, La Greca, La Greca, Blount and Blount35 which are based on three levels of empirical support, and according to the results of this and Uher et al's study, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 the SOCS can be considered a well-established, or at least a promising, paediatric OCD assessment instrument.

Acknowledgements

We thank Isobel Heyman for permitting us to use the SOCS, David Mataix-Cols and Miquel A. Fullana for their assistance in translating the tool into Spanish and Agustin E. Martínez-González for his drive and dedication.

Footnotes

Declaration of interest

None.

References

1 American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (5th edn). APA, 2013.Google Scholar
2 Farrell, L, Barrett, PM. Obsessive-compulsive disorder across developmental trajectory: cognitive processing of threat in children, adolescents and adults. Br J Psychol 2006; 97: 95114.CrossRefGoogle Scholar
3 Canals, J, Hernández-Martínez, C, Cosi, S, Voltas, N. The epidemiology of obsessive-compulsive disorder in Spanish school children. J Anxiety Disord 2012; 26: 746–52.CrossRefGoogle ScholarPubMed
4 Godoy, A, Gavino, A, Carrillo, F, Cobos, MP, Quintero, C. Composición factorial de la version española de la Spence Children Anxiety Scale (SCAS). Psicothema 2011; 23: 289–94.Google Scholar
5 Orgilés, M, Méndez, X, Espada, JP, Carballo, JL, Piqueras, JA. Anxiety disorder symptoms in children and adolescents: differences by age and gender in a community sample. Rev Psiquiatr y Salud Ment (English Ed) 2012; 5: 115–20.Google Scholar
6 Krebs, G, Heyman, I. Obsessive-compulsive disorder in children and adolescents. Arch Dis Child 2014; 100: 495–9.Google ScholarPubMed
7 Uher, R, Heyman, I, Mortimore, C, Frampton, I, Goodman, R. Screening young people for obsessive–compulsive disorder. Br J Psychiatry 2007; 191: 353–4.CrossRefGoogle ScholarPubMed
8 Overduin, MK, Furnham, A. Assessing obsessive-compulsive disorder (OCD): a review of self-report measures. J Obsessive Compuls Relat Disord 2012; 1: 113.Google Scholar
9 Iniesta-Sepúlveda, M, Rosa-Alcázar, AI, Rosa-Alcázar, Á, Storch, EA. Evidence-based assessment in children and adolescents with obsessive-compulsive disorder. J Child Fam Stud 2013; 23: 1455–70.Google Scholar
10 Scahill, L, Riddle, MA, McSwiggin-Hardin, M, Ort, SI, King, RA, Goodman, WK, et al. Children's Yale-Brown Obsessive Compulsive Scale: reliability and validity. J Am Acad Child Adolesc Psychiatry 1997; 36: 844–52.CrossRefGoogle ScholarPubMed
11 Foa, EB, Coles, ME, Huppert, JD, Pasupuleti, RV, Franklin, ME, March, JS. Development and validation of a child version of the obsessive compulsive inventory. Behav Ther 2010; 41: 121–32.CrossRefGoogle ScholarPubMed
12 Shafran, R, Frampton, I, Heyman, I, Reynolds, M, Teachman, B, Rachman, S. The preliminary development of a new self-report measure for OCD in young people. J Adolesc 2003; 26: 137–42.CrossRefGoogle ScholarPubMed
13 Storch, EA, Khanna, M, Merlo, LJ, Loew, BA, Franklin, M, Reid, JM. Children's Florida obsessive compulsive inventory: psychometric properties and feasibility of a self-report measure of obsessive-compulsive symptoms in youth. Child Psychiatry Hum Dev 2009; 40: 467–83.CrossRefGoogle ScholarPubMed
14 Achenbach, TM. Manual for the Child Behavior Checklist/4-18 and 1991 Profile. University of Vermont Department of Psychiatry, 1991.Google Scholar
15 Storch, EA, Muroff, J, Lewin, AB, Geller, D, Ross, A, McCarthy, K, et al. Development and preliminary psychometric evaluation of the children's saving inventory. Child Psychiatry Hum Dev 2011; 42: 166–82.CrossRefGoogle ScholarPubMed
16 Rodríguez-Jiménez, T, Godoy, A, Piqueras, JA, Gavino, A, Martínez-González, AE, Foa, E. Factor structure and measurement invariance of the Obsessive-Compulsive Inventory – Child version (OCI-CV) in general population. Eur J Psychol Assess 2015, in press (doi: 10.1027/1015-5759/a000276).CrossRefGoogle Scholar
17 Berg, CJ, Rapoport, JL, Flament, M. The leyton obsessional inventory-child version. J Am Acad Child Psychiatry 1986; 25: 8491.CrossRefGoogle ScholarPubMed
18 Association AP. Diagnostic and Statistical Manual of Mental Disorder (DSM-IV) (4th edn). American Psychiatric Association, 1994.Google Scholar
19 Kaufman, J, Birmaher, B, Brent, D, Rao, U, Flynn, C, Moreci, P, et al. Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 1997; 36: 980–8.CrossRefGoogle ScholarPubMed
20 Ulloa, RE, Ortiz, S, Higuera, F, Nogales, I, Fresán, A, Apiquian, R, et al. Interrater reliability of the Spanish version of schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL). Actas Esp Psiquiatr 2006; 34: 3640.Google Scholar
21 Hambleton, RK, Merenda, P, Spielberger, C (eds). Adapting Educational and Psychological Tests for Cross-cultural Assessment. Lawrence Erlbaum, 2005.Google Scholar
22 Sandin, B, Chorot, P, Valiente, RM, Chorpita, BF, Sandín, B. Development of a 30-item version of the revised child anxiety and depression scale. Rev Psicopatología y Psicol Clínica 2010; 15: 165–78.Google Scholar
23 Chorpita, BF, Yim, L, Moffitt, CE, Umemoto, LA, Francis, SE. Assessment of symptoms of DSM-IV anxiety and depression in children: a revised child anxiety and depression scale. Behav Res Ther 2000; 38: 835–55.CrossRefGoogle ScholarPubMed
24 Schermelleh-Engel, K, Moosbrugger, H, Müller, H. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online 2003; 8: 2374.Google Scholar
25 Cheung, GW, Rensvold, RB. Evaluating goodness-of-fit indexes for testing measurement invariance. Struct Equ Model A Multidiscip J 2002; 9: 233–55.Google Scholar
26 McDonald, R. Test Theory: A Unified Treatment. Lawrence Erlbaum, 1999.Google Scholar
27 Woodhouse, B, Jackson, PH. Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: a search procedure to locate the greatest lower bound. Psychometrika 1977; 42: 579–91.CrossRefGoogle Scholar
28 Schweizer, K. On the changing role of Cronbach's α in the evaluation of the quality of a measure. Eur J Psychol Assess 2011; 27: 143–4.CrossRefGoogle Scholar
29 Erceg-Hurn, DM, Mirosevich, VM. Modern robust statistical methods: an easy way to maximize the accuracy and power of your research. Am Psychol 2008; 63: 591601.CrossRefGoogle ScholarPubMed
30 Cohen, J. Statistical Power Analysis for the Behavioral Science (2nd edn). Lawrence Erlbaum, 1988.Google Scholar
31 Lipsey, M, Wilson, D. Practical Meta-analysis. Sage, 2001.Google Scholar
32 Grissom, RJ. Probability of the superior outcome of one treatment over another. J Appl Psychol 1994; 79: 314–16.CrossRefGoogle Scholar
33 Nunnally, JC, Bernstein, IH. Psychometric Theory. McGraw-Hill, 1994.Google Scholar
34 Sun, J, Li, Z, Buys, N, Storch, EA. Correlates of comorbid depression, anxiety and helplessness with obsessive-compulsive disorder in Chinese adolescents. J Affect Disord 2015; 174: 31–7.CrossRefGoogle ScholarPubMed
35 Cohen, LL, Cohen, LL, La Greca, AM, La Greca, AM, Blount, RL, Blount, RL, et al. 2007. Introduction to special issue: evidence-based assessment in pediatric psychology. J Pediatr Psychol 2008; 33: 911–5.CrossRefGoogle Scholar
Submit a response

eLetters

No eLetters have been published for this article.

Altmetric attention score

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 226
Total number of PDF views: 437 *
View data table for this chart

* Views captured on Cambridge Core between 02nd January 2018 - 22nd April 2021. This data will be updated every 24 hours.

You have Access
Open access

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Validation of the Short Obsessive–Compulsive Disorder Screener (SOCS) in children and adolescents
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Validation of the Short Obsessive–Compulsive Disorder Screener (SOCS) in children and adolescents
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Validation of the Short Obsessive–Compulsive Disorder Screener (SOCS) in children and adolescents
Available formats
×
×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *