Obsessive–compulsive disorder (OCD) symptoms in children and adolescents are an underestimated and understudied problem. Recently, OCD has become part of a new chapter – entitled ‘Obsessive compulsive and related disorders’ – in the DSM-5. 1 OCD is a mental disorder characterised by the presence of obsessions and/or compulsions that produce intense distress and disability. 1 According to epidemiological studies, OCD often begins in childhood or early adolescence. Reference Farrell and Barrett2 Prevalence rates in children and adolescents can reach up to 3% for OCD Reference Canals, Hernández-Martínez, Cosi and Voltas3 and up to 19% for the subclinical symptoms. Reference Canals, Hernández-Martínez, Cosi and Voltas3–Reference Orgilés, Méndez, Espada, Carballo and Piqueras5
Therefore, the development and dissemination of effective interventions for OCD are critically important, as is the need for early detection and intervention to improve outcomes. For example, regarding assessment and treatment of OCD, the National Institute for Health and Care Excellence (NICE) recommends the routine screening of young people at risk in general practice or other settings where they may present for help. Reference Krebs and Heyman6 Such screening requires short, easy-to-use, highly sensitive, reasonable and widely available measures. Reference Uher, Heyman, Mortimore, Frampton and Goodman7
Thus, recently, there has been a considerable increase in assessment tools for OCD and obsessive–compulsive symptoms in children and adolescents. Reference Overduin and Furnham8 For example, according to a recent review, Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 the Children's Yale-Brown Obsessive–Compulsive Scale (CY-BOCS) Reference Scahill, Riddle, McSwiggin-Hardin, Ort, King and Goodman10 is the gold standard in the field of psychiatric clinical assessment, because of its excellent psychometric properties; however, it is a lengthy, clinician-administered, interview-based instrument, making it unsuitable for use in community settings; moreover, its self-report format is too lengthy. Other measures such as the Obsessive Compulsive Inventory – Child Version (OCI-CV), Reference Foa, Coles, Huppert, Pasupuleti, Franklin and March11 the Children's Obsessional Compulsive Inventory (CHOCI), Reference Shafran, Frampton, Heyman, Reynolds, Teachman and Rachman12 the Children's Florida Obsessive–Compulsive Inventory (C-FOCI), Reference Storch, Khanna, Merlo, Loew, Franklin and Reid13 the Obsessive–Compulsive Subscale (OCS) of the Child Behavior Checklist (CBCL) Reference Achenbach14 and the Child Saving Inventory (CSI) Reference Storch, Muroff, Lewin, Geller, Ross and McCarthy15 are promising self-report measures; however, all of them present some limitations. These limitations concern features such as the tools’ length, ease of use and availability. Reference Rodríguez-Jiménez, Godoy, Piqueras, Gavino, Martínez-González and Foa16 Furthermore, most researchers assert that evidence-based assessment (EBA) is a first step towards the development of psychopathological studies and assessment of the effectiveness of empirically validated treatments. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9,Reference Rodríguez-Jiménez, Godoy, Piqueras, Gavino, Martínez-González and Foa16 Specifically, the NICE recommends the Short Obsessive–Compulsive Disorder Screener (SOCS). Reference Krebs and Heyman6,Reference Uher, Heyman, Mortimore, Frampton and Goodman7
Uher et al Reference Uher, Heyman, Mortimore, Frampton and Goodman7 designed and validated the SOCS to provide a simple screening tool suitable for general practice and community settings. The SOCS is a seven-item self-report tool with a three-point response format (0 [‘no’], 1[‘a bit’] and 2 [‘a lot’]). The first five items were adapted from the five most discriminant items of the 44-item child version of the Leyton Obsessional Inventory (LOI) Reference Berg, Rapoport and Flament17 and assess common symptoms including checking, touching, cleanliness/washing, repeating and exactness. Two additional questions assess the associated impairment and resistance. The SOCS score is calculated by summing the scores for all seven items. With a sample of 11- to 15-year-old youth, Uher et al found that the tool had good internal consistency, a unidimensional factor structure and excellent sensitivity for detecting OCD in children; furthermore, its specificity was good for children without psychiatric diagnoses but poor for a psychiatric sample. Although Uher et al's study highlights the tool's uses, the SOCS still has limitations concerning certain psychometric properties, such as factorial structure invariance, test–retest reliability, and the lack of evidence of convergent/divergent validity. Furthermore, all psychometric properties must be investigated in a broader age range than 11–15 years.
Consequently, our main objective was to further validate the SOCS while empirically examining whether it is useful in assessing obsessive–compulsive symptoms across a wide age range of clinical and community samples of children and adolescents.
Method
Participants
The clinical sample comprised 94 consecutive child and adolescent patients (46 males; 48 females) with a current diagnosis of OCD according to the DSM-IV criteria. 18 Their mean age was 14.62 years (s.d.=2.65; range 9–19). All patients were recruited from the Child and Adolescent Psychiatry and Psychology Unit of Hospital Clínic de Barcelona, and from the Child and Adolescent Psychological Clinical Unit of Universidad Miguel Hernández de Elche. Axis I diagnoses were ascertained using the Schedule for Affective Disorders and Schizophrenia for School-Age Children – Present and Lifetime Version (K-SADS-PL), Reference Kaufman, Birmaher, Brent, Rao, Flynn and Moreci19 which has been validated in Spanish. Reference Ulloa, Ortiz, Higuera, Nogales, Fresán and Apiquian20 The children and their parents were informants. Exclusion criteria were psychiatric comorbidity with a psychotic disorder or autism spectrum disorder, and an IQ of less than 70. Patients could have another comorbid Axis I disorder, but OCD needed to be the primary disorder. Specifically, we found the following rates of associated secondary disorders: generalised anxiety disorder 16%, Tourette's disorder 4.3%, attention-deficit hyperactivity disorder 4.3%, social phobia 2.1% and major depression 2.1%.
Written informed consent was obtained from all participants and families following an explanation of the procedures involved.
The community sample consisted of 880 students (437 males; 443 females) from five schools in Levante, eastern Spain. Their mean age was 13.21 years (s.d.=2.19; range 10–18). All the schools volunteered for the study. After obtaining permission from the principals and psycho-pedagogical departments of the schools, all students agreed to participate.
Measures
SOCS Reference Uher, Heyman, Mortimore, Frampton and Goodman7
This was translated into Spanish with translation and backtranslation. Reference Hambleton, Merenda and Spielberger21 The Spanish version of the SOCS was administered to a pilot sample of 10-year-olds to ensure that the item content could be understood by young children. This Spanish translation was authorised in 2009 by Isobel Heyman, an author of the original scale.
Ad hoc sociodemographic sheet
This was designed to collect data on age, gender and geographic area of residence.
Spanish version of the OCI-CV
This comprises 21 items scored on a 3-point Likert scale (0 [‘never’], 1 [‘sometimes’] and 2 [‘always’]). The OCI-CV provides six subscale scores – doubting/checking, obsessing, hoarding, washing, ordering and neutralising – and a total score. Reference Foa, Coles, Huppert, Pasupuleti, Franklin and March11 The psychometric properties of the English Reference Foa, Coles, Huppert, Pasupuleti, Franklin and March11 and Spanish Reference Rodríguez-Jiménez, Godoy, Piqueras, Gavino, Martínez-González and Foa16 versions have been reported.
C-FOCI Reference Storch, Khanna, Merlo, Loew, Franklin and Reid13
This was developed from the original version of the FOCI for adults to assess obsessive–compulsive symptoms in 7- to 20-year-old children and adolescents. It consists of 22 items in two subscales: symptom checklist and OCD severity scale. It has shown adequate psychometric properties both for English Reference Storch, Khanna, Merlo, Loew, Franklin and Reid13 and Spanish speaking children and adolescents (further details of the Spanish validation study available from the authors on request).
Revised Child Anxiety and Depression Scale (RCADS-30) Reference Sandin, Chorot, Valiente, Chorpita and Sandín22
This is a reduced 30-item version of the RCADS, Reference Chorpita, Yim, Moffitt, Umemoto and Francis23 a self-report for evaluating anxiety and depression in children and adolescents. The scale comprises 30 items and in six subscales for evaluating symptoms of the following conditions: panic disorder, social phobia, separation anxiety disorder, generalised anxiety disorder, OCD and major depressive disorder. The scale ranges from 0 to 3 (corresponding to never, sometimes, often and always, respectively). The scale has shown excellent psychometric properties, equivalent to the full version, with a Spanish population. Reference Sandin, Chorot, Valiente, Chorpita and Sandín22
CY-BOCS
This is a clinician-administered scale involving a semi-structured interview for assessing the severity of obsessive–compulsive symptoms in children. Reference Scahill, Riddle, McSwiggin-Hardin, Ort, King and Goodman10 The scale includes a list of subdomains used to identify the content of the patient's obsessions and compulsions. These symptoms are then rated on a 5-point scale (0 [‘no symptoms’] to 4 [‘extreme symptoms’]) covering five areas: time spent on the symptoms per day, interference caused by symptoms, level of distress, level of resistance and control. Summing these scores for each symptom produces scores for the severity of various obsessions and compulsions. A total severity score (ranging from 0 to 40) can also be obtained, with a higher score indicating greater severity. A total severity score of 16 serves as the cut-off point for an OCD diagnosis.
Procedure
The study was approved by the Research and Ethics Committees of the Universidad Miguel Hernández de Elche and the Hospital Clínic de Barcelona. Experienced psychologists with master's degree or doctorate in psychology administered the SOCS and the other self-administered questionnaires to the clinical and community samples at the time of study entry, along with the K-SADS-PL and CY-BOCS exclusively to the clinical sample. They administered the questionnaires 4 weeks after the first evaluation again. The psychologists gave the participants clear instructions concerning how to indicate their responses and they were reassured that it was not a test and that there were no right or wrong answers, and individual assistance to those students who needed it. In order to prevent the introduction of biases in the collection of information, four different balanced protocols were applied. For it, the questionnaires were randomly ordered in each protocol. The average time of application of the assessment was about 50 minutes and no cases of fatigue were detected.
Data analysis
Six sets of analyses based on the ordinal nature of the variables (in all cases, we used ordinal and not ratio scales) were conducted.
First, we examined the item distribution and frequency as well as the factor structure of the SOCS for the clinical and community samples. Then, based on the results of Uher et al, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 the hypothesis that the SOCS has a single-factor structure was tested by confirmatory factor analysis (CFA) performed simultaneously for both the clinical and community samples. We also tested the alternative hypothesis that the SOCS items are independent, and tested whether the SOCS exhibits metric invariance. Testing metric invariance was conducted once, not systematically (i.e. we did not check the configuration invariance first; instead, we first checked the invariance of the factor loadings, after which we checked the measurement errors followed by the intercepts). Therefore, in all cases, we introduced all the restrictions simultaneously, which enabled us to check the metric invariance between the samples with a single calculation. We used polychoric correlation matrices and the diagonally weighted least squares (DWLS) method in all cases (LISREL 8.8, DWLS procedure). Last, we used the following indices as goodness-of-fit measures Reference Schermelleh-Engel, Moosbrugger and Müller24 : the Satorra-Bentler chi-square, a root mean square error of approximation (RMSEA) less than or equal to 0.8 and a confirmatory factor index (CFI), goodness of fit index (GFI) and non-normed fit index (NNFI) (or Tucker-Lewis index) higher than 0.95. The CFI was also used to compare the basic model (total sample) with the metric invariance model in the case of the two samples. According to Cheung and Rensvold, Reference Cheung and Rensvold25 the invariance between samples is admissible when the difference between the CFIs (ΔCFI) is less than or equal to 0.01. They also assert that ΔCFI is a better estimator of invariance admissibility than Δχ Reference Farrell and Barrett2 , which is the index used by some researchers.
McDonald's omega Reference McDonald26 and the greatest lower bound to reliability index (GLB) Reference Woodhouse and Jackson27 were used to estimate the internal consistency of the SOCS, as they are better estimators of reliability than the Cronbach's alpha is. Reference Schweizer28 We also administered the SOCS twice after an interval of 4 weeks to examine test–retest reliability.
Discriminative validity was examined using the Mann–Whitney U-test and the probability of superiority Reference Erceg-Hurn and Mirosevich29 was used to estimate the effect size of the differences between the samples.
We also conducted analyses of convergent/divergent validity by calculating Spearman's rho coefficients. We used Cohen's criteria Reference Cohen30 to evaluate the effect sizes of the correlations: small (0.20) and large (≥0.50). Reference Cohen30,Reference Lipsey and Wilson31
To determine the accuracy of the scales in terms of their ability to correctly classify children and adolescents with or without OCD, we conducted a receiver operating characteristic (ROC) curve analysis, wherein subscales with areas under the curve (AUCs) between 0.70 and 0.80 were considered adequate.
Then, the predictive validity of the SOCS for a K-SADS-PL diagnosis of OCD was calculated by binary logistic regression analyses. The diagnoses according to the K-SADS-PL were the outcome variables (OCD or non-OCD), and the predictor variable was SOCS score.
Results
Item analysis
The means of the item scores for the clinical sample ranged from 0.66 (item 4) to 1.31 (item 1), and the standard deviations ranged from 0.59 (item 7) to 0.80 (item 2). The distribution for almost all the item scores showed skewness indices (−0.57 to 0.70) and kurtosis indices (−1.33 to 0.46) near zero. Overall, the average percentage of each response across the items scores was as follows: ‘no’ (26%), ‘a lot’ (34.1%) and ‘a bit’ (39.9%). In the community sample, the means of the items scores ranged from 0.55 (item 6) to 1.05 (item 5), and the standard deviations ranged from 0.63 (item 1) to 0.75 (item 3). For almost all the items scores, skewness (−0.09 to 0.80) and kurtosis indices (−1.20 to 0.46) were near zero. Overall, the average percentage of each response across the items scores was as follows: ‘a lot’ (18.4%), ‘no’ (37.8%) and ‘a bit’ (43.8%).
CFA
As can be seen in Table 1, for the total sample (clinical and community), the fit indices indicated that the model fits the data reasonably well, and we found metric invariance between the samples (configurational invariance and invariance of factor loadings, measurement errors and intercepts). The RMSEA was <0.8, and the CFI, GFI and NNFI were >0.90; the ΔCFI was 0.01 (0.98–0.97).
Table 1 Confirmatory factor analysis (total sample: N=974) and multi-group confirmatory factor analyses (diagonally weighted least squares; polychoric correlation matrix) for children from clinical setting (n=94) and children from the community (n=880).
| Sample/model | χ2 | df | RMSEA (90% CI) | CFI | GFI | NNFI | IFI | RFI |
|---|---|---|---|---|---|---|---|---|
| Total sample | ||||||||
| Independent | 1969.59 | 21 | ||||||
| One factor | 55.55 | 14 | 0.06 (0.04–0.07) | 0.98 | 0.99 | 0.97 | 0.98 | 0.96 |
| Metric invariance | ||||||||
| Clinical community | ||||||||
| Independent | 1779.37 | 42 | ||||||
| One factor | 109.59 | 55 | 0.05 (0.03–0.06) | 0.97 | Clinical: 0.90 Community: 0.99 |
0.98 | 0.97 | 0.95 |
χ2, Satorra-Bentler's chi-squared; RMSEA, root mean square error of approximation; CFI, comparative fix index; GFI, good-of-fit index; NNFI, non-normed fit index; IFI, incremental fit index; RFI, relative fit index.
Table 2 shows the degree of the relationship (standardised lambda weights) for each item score with a single factor. As is evident, all item weights are above 0.50, except for the weight of item 2 (0.35).
Table 2 Item content, item factor loading (lambda). Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).
| Item | Item content | Lambda |
|---|---|---|
| 1 | Does your mind often make you do things like checking or touching things or counting things, even though you know you don't really have to? | 0.50 |
| 2 | Are you particularly fussy about keeping your hands clean? | 0.35 |
| 3 | Do you ever have to do things over and over a certain number of times before they seem quite right? | 0.61 |
| 4 | Do you ever have trouble finishing your school work or chores because you have to do something over and over again? | 0.56 |
| 5 | Do you worry a lot if you've done something not exactly the way you like? | 0.54 |
| 6 | Do these things interfere with your life? | 0.78 |
| 7 | Do you try to stop them? | 0.51 |
Reliability
The internal consistency (McDonald's omega and GLB) of the SOCS for both samples is provided in Table 3. The reliability indices for the total score were higher in the clinical sample.
Table 3 Means (s.d.), differences between clinical and community samples and reliability (McDonald's omega and GLB) for the SOCS. Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).
| Sample | Statistic | Total score |
|---|---|---|
| Total sample | Omega | 0.75 |
| GLB | 0.80 | |
| Mean | 5.82 | |
| s.d. | 2.92 | |
| Clinical sample | Omega | 0.79 |
| GLB | 0.88 | |
| Mean | 7.56 | |
| s.d. | 3.15 | |
| Community sample | Omega | 0.74 |
| GLB | 0.79 | |
| Mean | 5.64 | |
| s.d. | 2.83 | |
| Mann–Whitney U-test | 27093* | |
| Z | −5.53 | |
| PS effect size | 0.67 | |
GLB, greatest lower bound to reliability; PS, probability that a randomly sampled score from one population is larger than a randomly sampled score from a second population.
* P=0.01.
Regarding test–retest reliability, the two mean SOCS scores of the clinical sample were 7.56 (s.d.=3.15) and 6.58 (s.d.=3.18), with a rho of 0.79. In the community sample, the mean scores were 5.64 (s.d.=2.83) and 4.61 (s.d.=2.83), with a rho of 0.63. In both samples, the correlations were statistically significant (P<0.01).
Descriptive statistics of the scales and discriminative validity
Table 3 shows the means and standard deviations of the SOCS for the total sample, for the clinical sample and for the community sample. In general, the students from the clinical sample obtained higher scores on the SOCS than did those from the school sample (probability of superiority 0.67, which is considered a medium effect size according to Grissom Reference Nunnally and Bernstein33 ).
Analysis of convergent/divergent validity
The results indicated that in both samples, SOCS score was significantly and positively correlated with the scores on well-established measures of child OCD (see Table 4). The effect sizes were large in the clinical sample (r=0.50–0.80) and medium to large in the community sample (r=0.47–0.72). The only exception, in both samples, was the OCI-CV's Hoarding scale, which had a moderate correlation with the SOCS. In the clinical sample, the total severity scale of the CY-BOCS showed a moderate association with the SOCS.
Table 4 Convergent/divergent validity of the SOCS. Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).
| Measures | SOCS | |
|---|---|---|
| Clinical | Community | |
| CY-BOCS total severity score | 0.38** | – |
| C-FOCI symptom checklist | 0.65** | 0.61** |
| C-FOCI severity scale | 0.57** | 0.49** |
| OCI-CV checking | 0.68** | 0.62** |
| OCI-CV obsessing | 0.53** | 0.47** |
| OCI-CV hoarding | 0.43** | 0.43** |
| OCI-CV washing | 0.53** | 0.54** |
| OCI-CV ordering | 0.50** | 0.49** |
| OCI-CV neutralising | 0.58** | 0.47** |
| OCI-CV total score | 0.80** | 0.72** |
| RCADS-30 separation anxiety | 0.26* | 0.36** |
| RCADS-30 social phobia | 0.43** | 0.40** |
| RCADS-30 generalised anxiety | 0.46** | 0.43** |
| RCADS-30 panic disorder | 0.44** | 0.42** |
| RCADS-30 obsessive–compulsive disorder | 0.63** | 0.52** |
| RCADS-30 major depression | 0.49** | 0.41** |
| RCADS-30 total score | 0.61** | 0.57** |
SOCS, Short OCD Screener; OCI-CV, Obsessive–Compulsive Inventory – Child Version; RCADS-30, Revised Child Anxiety and Depression Scale; C-FOCI, Children's Florida Obsessive–Compulsive Inventory; CY-BOCS, Children's Yale-Brown Obsessive Compulsive Scale.
* P<0.05; **P<0.01.
With regard to related construct measures such as anxiety and depression symptoms, all the correlations were significant with medium effect sizes (r=0.26–0.49). The only exception in both samples was the total score on the RCADS, which showed a large effect size (see Table 4).
Sensitivity and specificity of the SOCS
The AUC of the SOCS was 0.67 (s.e.=0.03, P<0.01), suggesting that there is a 67% probability that a child or adolescent with OCD will have a higher score on the SOCS than a non-OCD child/adolescent. Table 5 shows the sensitivity and specificity of the SOCS.
Table 5 Sensitivity, specificity, Youden index and accuracy of the SOCS. Total sample (N=974), children from clinical setting (n=94) and children from the community (n=880).
| Cut-off | Sensitivity | Specificity | PPV | NPV | Youden index | Accuracy |
|---|---|---|---|---|---|---|
| 1 | 0.98 | 0.02 | 0.11 | 0.90 | 0.00 | 0.89 |
| 2 | 0.97 | 0.08 | 0.04 | 0.90 | 0.05 | 0.84 |
| 3 | 0.95 | 0.15 | 0.04 | 0.89 | 0.09 | 0.78 |
| 4 | 0.90 | 0.26 | 0.04 | 0.89 | 0.16 | 0.68 |
| 5 | 0.83 | 0.37 | 0.05 | 0.88 | 0.20 | 0.59 |
| 6 | 0.76 | 0.49 | 0.05 | 0.86 | 0.25 | 0.49 |
| 7 | 0.61 | 0.60 | 0.07 | 0.86 | 0.21 | 0.40 |
| 8 | 0.51 | 0.73 | 0.07 | 0.83 | 0.24 | 0.29 |
| 9 | 0.37 | 0.82 | 0.08 | 0.82 | 0.19 | 0.22 |
| 10 | 0.30 | 0.91 | 0.08 | 0.74 | 0.21 | 0.15 |
| 11 | 0.21 | 0.96 | 0.08 | 0.62 | 0.18 | 0.11 |
| 12 | 0.12 | 0.98 | 0.09 | 0.56 | 0.10 | 0.10 |
| 13 | 0.04 | 1.00 | 0.09 | 0.50 | 0.04 | 0.10 |
| 14 | 0.02 | 1.00 | 0.09 | 0.00 | 0.02 | 0.09 |
PPV, positive predictive value; NPV, negative predictive value; accuracy (informedness), percentage of children correctly classified.
We selected cut-off scores to provide the best balance between sensitivity and specificity for each measure. A SOCS cut-off score of 7 showed a sensitivity of 61% and a specificity of 60%, a Youden index of 0.21 and an accuracy or informedness (percentage of participants correctly classified) of 40%. However, according to the Youden index and accuracy percentage, a score of 6 was more sensitive but less specific, making it a possibly more suitable cut-off score for detecting OCD.
Predictive validity
The predictive model accounted for 8% (Nagelkerke's R 2) of the variance in OCD diagnoses and resulted in a classification accuracy of 90.3%. The SOCS was related to diagnostic status (Wald (1)=34.66, P<0.01) such that higher SOCS scores were better predictors of OCD diagnoses. The odds ratio was significant, OR=1.26 (95% CI 1.17–1.36).
Discussion
The NICE recommends the SOCS as a suitable and validated screening tool for youth between 11 and 15 years. In spite of its excellent sensitivity and specificity in detecting OCD in non-clinical children, the research on SOCS does show some gaps. For example, the invariance of the SOCS’ factorial structure is unknown, there is some ambiguity regarding test–retest reliability estimations and there is a lack of evidence concerning its convergent/divergent validity. Moreover, all such properties must be determined for a broader age range than 11- to 15-year-olds. Thus, we provided new data about these psychometric properties to determine whether the SOCS is suitable for assessing OCD symptoms across a wider age range of children and adolescents.
Our results supported the factorial validity of the SOCS, confirming the unidimensional structure previously found. Reference Uher, Heyman, Mortimore, Frampton and Goodman7 The CFA showed that the data fitted adequately to the unidimensional measurement model, although the factor loadings of item 2 were below 0.40. This small load factor for item 2 may be due to hypothetical understanding problem by the youngest children.
With regard to invariance, the SOCS presented the expected metric invariance between the samples, as indicated by an RMSEA below 0.06 and CFI, GFI and NNFI values over 0.90; these values also indicated acceptable fit according to Schermelleh-Engel et al Reference Schermelleh-Engel, Moosbrugger and Müller24 Thus, in both samples, we found the same factor and equivalent factor loadings, measurement errors and intercepts.
With respect to reliability estimation, the internal consistency coefficients of the SOCS were between moderate and high (0.74–0.88), namely, above the recommended value of 0.70 proposed by Nunnally & Bernstein. Reference Nunnally and Bernstein33 These data indicate that the scale has a high internal consistency, which supports the existence of a single dimension. The data are consistent with a previous study carried out with the SOCS, which also indicated good internal consistency (Cronbach's alpha 0.85). Our data are also consistent with those concerning other EBA measures for OCD, such as the CY-BOCS, OCI-CV, CHOCI, C-FOCI, OCS and CSI. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9
The SOCS also showed an adequate temporal stability, as we found positive correlations for both the clinical (0.79) and community (0.63) samples after 4 weeks. According to reliability studies, values between 0.50 and 0.70 indicate moderate reliability, and values between 0.80 and 0.90 indicate good reliability. Other studies on self-report measures for OCD show similar short-term temporal stability. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9
The mean score of the clinical sample was higher than that of the community sample (M=5.64; s.d.=2.83), with a medium effect size, supporting the discriminative validity of the SOCS. In addition, the mean score for the OCD group was lower than that reported by Uher et al Reference Uher, Heyman, Mortimore, Frampton and Goodman7 for an OCD group (M=9.7, s.d.=2.2) but higher than the mean for Uher et al's healthy group (M=3, s.d.=2.3), mixed community group (M=3.3, s.d.=2.5) and psychiatric control group (M=5.8, s.d.=2.8). Compared with the results of Uher et al, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 our results revealed somewhat low levels of obsessive–compulsive symptoms in the clinical sample but high levels in the community sample. This result could be because we obtained SOCS scores from samples with mean ages and an age range higher than was the case in Uher et al Reference Uher, Heyman, Mortimore, Frampton and Goodman7 and from a community sample larger than that in Uher et al. Reference Uher, Heyman, Mortimore, Frampton and Goodman7 Furthermore, we calculated the mean of the community sample without eliminating the scores of children who could have met the criteria for OCD (because we did not evaluate these criteria in this sample), whereas Uher et al eliminated children who met OCD criteria from their community sample. Therefore, in our sample, children with potential OCD were also considered when calculating the mean SOCS score. However, our outcomes are similar to findings concerning other OCD screening instruments (i.e. the LOI-CV), which displayed sensitivity values between poor and adequate. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 This conclusion can be defended as SOCS scores have metric invariance and the same intercepts in clinical and non-clinical populations, an issue supported in our study.
Regarding the concurrent validity, the confirmed relationships between the SOCS and the three measures with the highest empirical support – the CY-BOCS, OCI-CV and C-FOCI – as indicated by Iniesta-Sepúlveda et al, Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 also supports the validity of the SOCS. A large positive relationship was found between the SOCS and the total scores on the OCI-CV and the severity scale of the C-FOCI, whereas a moderate association was found between the SOCS and the total severity scale of the CY-BOCS in the clinical sample, following Cohen's effect size criteria. Reference Cohen30,Reference Lipsey and Wilson31 However, there were also moderate associations between the SOCS and specific anxiety and depression symptom measures. One explanation for this finding is the symptom overlap between depression, anxiety and OCD disorders, which other studies have also found. Reference Sun, Li, Buys and Storch34
Finally, we examined the sensitivity and specificity of the SOCS by using ROC analysis. Overall, the results indicated an AUC of 0.67. Values around 0.70 are considered to represent acceptable discriminatory ability. According to our data, a SOCS score above 7 was initially recommended because it provides an optimal balance between the percentages of true positives and negatives in real cases. This cut-off score led to an acceptably low percentage of adolescents incorrectly identified as having OCD (specificity 60%) and an adequate proportion of adolescents being overlooked in terms of heightened OCD symptoms (sensitivity 61%). However, because the main purpose of the SOCS is to serve as a screening measure of OCD, the best cut-off score is 6, with a Youden index of 0.25. This SOCS score could be advantageous for screening adolescents in clinical settings, as it would preclude the overlooking of adolescents requiring further assessment. However, depending on the intended purpose of the SOCS, administrators could choose a higher or lower cut-off score. For example, a more conservative cut-off score of 8–9 might be justified when using the subscales of the SOCS for research purposes in order to avoid false positives (see Table 5). Our sensitivity/specificity results were lower than those reported by Uher et al, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 but again, the sample differences could explain this discrepancy. However, our outcomes are similar to findings concerning other OCD screening instruments (i.e. the LOI-CV), which displayed sensitivity values between poor and adequate. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 However, this aspect of OCD measures presents limited psychometric evidence according to Iniesta-Sepúlveda et al. Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 Furthermore, although the community sample was assumed to have no OCD cases, it is possible that this sample contained such cases, which could explain our relatively low sensitivity and specificity.
Regarding the predictive validity of the SOCS, the results of a binary logistic regression analysis indicated that the scale was a significant predictor of OCD diagnosis (classification accuracy of 90.3%). This result is consistent with the findings of Uher et al's study with the SOCS, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 which resulted in the development of a screening tool suitable for accurately classifying OCD.
Limitations
Some limitations should be noted. First, our study did not examine the psychometric properties of SOCS with a large clinical sample. Second, the sensitivity and specificity in differentiating children with and without OCD with a sample of children with the disorder and a non-clinically interviewed community-based sample. And third, its convergent and discriminant validity with other self-report measures as well as the relationship of its scores with those from other assessment procedures (e.g. information deriving from parents or teachers).
Use of the SOCS
In conclusion, the SOCS is shorter than other self-report tools (for further details, see the review by Iniesta-Sepúlveda et al Reference Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-Alcázar and Storch9 ) and has comparable psychometric properties. Following the classification of EBAs, Reference Cohen, Cohen, La Greca, La Greca, Blount and Blount35 which are based on three levels of empirical support, and according to the results of this and Uher et al's study, Reference Uher, Heyman, Mortimore, Frampton and Goodman7 the SOCS can be considered a well-established, or at least a promising, paediatric OCD assessment instrument.
Acknowledgements
We thank Isobel Heyman for permitting us to use the SOCS, David Mataix-Cols and Miquel A. Fullana for their assistance in translating the tool into Spanish and Agustin E. Martínez-González for his drive and dedication.
Open access
eLetters
No eLetters have been published for this article.