Significant outcomes
-
∙ The Audio Recorded Cognitive Screen (ARCS) may be able to detect the cognitive deficits associated with schizophrenia. The memory and fluency domains are the main areas of discrimination.
-
∙ The ARCS and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) may be more appropriate cognitive screening tools for individuals with schizophrenia than the Mini Mental State Examination (MMSE), which was unable to discriminate between the two groups after demographic variables were controlled for.
-
∙ Performance on the ARCS may be associated with employment status.
Limitations
-
∙ RBANS and other clinical and diagnostic data were obtained from stored data held by the Australian Schizophrenia Research Bank (ASRB) collected at varying times and may not have been representative of contemporaneous cognitive functioning or clinical state.
-
∙ The small sample size limited the precision of statistical tests.
-
∙ The sample was recruited from a research register and is likely to be a more highly functioning subsection of individuals with schizophrenia living in the community, with limited generalisability to more acutely ill patients.
Introduction
Cognitive impairment in individuals with schizophrenia is a well-established finding. Individuals with schizophrenia have been found to have deficits in the areas of memory (Reference Tracey, Mattson, King, Bundick, Celenza and Glosser1–Reference Lee and Park5), executive functioning (Reference Chan, Chen and CW6), and attention (Reference Kurtz, Ragland, Bilker, Gur and Gur7,Reference Kamio, Nakagome and Murakami8). These deficits are relatively stable over time and largely independent of changes in clinical status (Reference Censits, Ragland, Gur and Gur9–Reference Addington, Saaedi and Addinton11). Cognitive deficits in schizophrenia have been shown to be associated with independent living skills (Reference Revheim, Schechter and Kim12), social functioning (Reference Cohen, Forbes, Mann and Blanchard13), and employment status (Reference Mcgurk and Meltzer14,Reference Hofer, Baumgartner and Bodber15). These findings suggest that cognitive impairment is a predictor of functional outcomes in individuals with schizophrenia and should be viewed as an important assessment and treatment target if the overall level of disability is to be reduced (Reference Gold16).
Comprehensive neuropsychological test batteries represent the gold standard in cognitive assessment in schizophrenia. However, such batteries may be problematic in clinical settings because of the time, cost, and specialist training required. Therefore, there is a need for an easily applied, widely available cognitive screening instrument that would not be expensive in terms of clinician time yet could provide a sensitive evaluation of cognitive strengths and weaknesses in such patients. Because of its brevity and familiarity to many clinicians the MMSE (Reference Folstein, Folstein and Mchugh17) is one of the most widely used screening instruments used in clinical practice and it has been recommended by some for its utility in detecting impairment in patients with schizophrenia (Reference Moore, Palmer and Jeste18). Other options for cognitive assessment in schizophrenia include the Brief Assessment of Cognition in Schizophrenia (BACS) (Reference Keefe, Goldberg, Harvey, Gold, Poe and Coughenour19) and the RBANS (Reference Gold, Queern, Iannone and Buchanana20). Both of these instruments require specialist training and take ∼30 min to administer. The BACS and RBANS have both been shown to be sensitive to the cognitive impairment in schizophrenia (Reference Keefe, Goldberg, Harvey, Gold, Poe and Coughenour19,Reference Wilk, Gold, Humber, Dickerson, Fenton and Buchanana21), well correlated with other cognitive measures (Reference Gold, Queern, Iannone and Buchanana20,Reference Wilk, Gold, Humber, Dickerson, Fenton and Buchanana21), and associated with functional outcomes (Reference Gold, Queern, Iannone and Buchanana20,Reference Keefe, Poe, Walker and Harvey22). However, research suggests that the MMSE may not be sensitive enough to detect cognitive deficit associated with schizophrenia (Reference Faustman, Moses and Csernansky23,Reference Manning, Wanigaratne, Best, Strathdee, Schrover and Gossop24).
The current study investigated the use of the ARCS with individuals with schizophrenia (Reference Schofield, Lee and Lewin25). The ARCS offers a novel way of screening for impairment in individuals with schizophrenia that is particularly efficient and suitable for community-based settings. The ARCS is unique in that it is administered via an audio device such as an MP3 or CD player and participants record their answers in a response booklet. The test was normed on a large sample of individuals (Reference Schofield, Lee and Lewin25). All domain and overall scores are adjusted for age, sex, and education level. The administration of the tool requires minimal clinician supervision, no special training, and takes only 35 min. The minimal resources required to administer this test may make it attractive for many community-based clinical settings such as community mental health and disability employment services.
There has only been one published study investigating the ability of the ARCS to detect the cognitive deficits associated with psychosis (Reference Loughland, Allen and Gianacas26). The authors administered the ARCS and the RBANS to a sample of 25 individuals with psychosis and 25 healthy controls. The psychosis group performed significantly worse than the control group on the ARCS memory, fluency, and overall scores. Positive correlations were found between several corresponding RBANS and ARCS domain scores. In addition, some of the ARCS scores were positively correlated with scores on the Global Assessment of Functioning in the psychosis group (Reference Loughland, Allen and Gianacas26). These findings suggest that the ARCS is sensitive to the cognitive deficits associated with psychosis and compares well with another well-standardised measure. They also provide preliminary evidence that the ARCS may predict functioning in individuals with psychosis.
Aims of the study
First, the current study aimed to determine whether the ARCS was better able to detect cognitive deficits in a sample of high-functioning, community-dwelling individuals with schizophrenia than the MMSE. We also wanted to determine if ARCS performance would be positively correlated with performance on the RBANS, which measures similar cognitive domains and has been shown to be sensitive to cognitive impairment in this population (Reference Gold, Queern, Iannone and Buchanana20). Last, the study aimed to explore the relationship between performance on the ARCS and functional outcomes, namely social functioning and employment status.
Methods
Participants
The participants of the study were 19 community-dwelling individuals with a diagnosis of schizophrenia (n=12) or schizoaffective disorder (n=7) and 19 healthy controls. Individuals with a diagnosis of schizoaffective disorder were included in the schizophrenia group to increase the number of potential participants. Schizoaffective disorder is part of the schizophrenia spectrum and has also been associated with cognitive deficits, although they are milder than those in schizophrenia (Reference Torniainen, Suvisaari and Partonen27). The schizophrenia group comprised 13 males and six females (mean age=50.21, SD=8.07). The control group comprised five males and 14 females (mean age=49.32, SD=12.53). All participants were recruited from the ASRB volunteer pool (Reference Loughland, Draganic and Mccabe28). Participants in the schizophrenia group had their diagnoses confirmed by the Diagnostic Interview for Psychosis (Reference Castle, Jablenskey and Mcgrath29) and met DSM-IV-TR criteria for schizophrenia or schizoaffective disorder. Individuals aged between 18 and 65 were eligible to participate in the research. Individuals with an intellectual disability, traumatic brain injury, or drug and alcohol dependence within the last 12 months were excluded from the study. All participants were reimbursed a minimal amount to cover expenses associated with their participation in the research.
Materials and apparatus
Participants in this study were required to complete two cognitive tests: the MMSE and the ARCS. The MMSE requires individuals to answer questions and complete tasks that aim to assess orientation, learning, memory, attention, language, and visuospatial ability. The MMSE provides one total score out of a maximum of 30.
The ARCS was administered via an MP3 player and headphones and the participants recorded their answers in the response booklet. The ARCS consists of eight subtests that are designed to measure five cognitive domains: memory, verbal fluency, language, visuospatial ability, and attention (Reference Schofield, Lee and Lewin25). The subtests include a 12-word list-learning task (three trials), clock drawing task, verbal fluency tasks (category, verb, letter), delayed recall, delayed recognition, picture naming, and the Hunter Attentional Tasks (HAT) A and B. In HAT A, individuals are instructed to convert 30 lowercase letters to upper case. In HAT B, individuals are instructed to convert 30 lowercase letters to the capital form when the letter is circled and replicate the letter in lower case when the letter is not circled. Individuals have 30 s for each task (i.e. HAT A and HAT B). The ARCS also includes a speed of writing task to identify any physical limitations the individual may have in completing the test. The last three pages of the ARCS response booklet consist of a brief questionnaire about the acceptability of the test.
The ARCS package includes an excel spreadsheet that automatically converts raw scores to scaled scores that are adjusted for age, sex, and education. The spreadsheet provides five domain scores as well as an overall ARCS score and a QuickARCS score with a mean of 100 and standard deviation of 15. The QuickARCS is a score that clinicians can calculate in minutes without the need to put the raw scores into the ARCS calculator. The QuickARCS is calculated by summing the raw scores of delayed recall, category fluency, visuospatial ability, and language, and then doubling the score and adding it to the raw score for HAT B. An integer constant is then added to the score based on age, sex, and education level (Reference Schofield, Lee and Lewin25).
Participants also completed three self-report measures: Depression Anxiety Stress Scales (DASS21) (Reference Lovibond and Lovibond30), Social Functioning Scale (SFS) (Reference Birchwood, Smith, Cochrane, Wetton and Copestake31), and employment questionnaire. The DASS21 is a well-validated 21-item measure that assesses an individuals’ experience of the states of depression, anxiety, and stress over the past week. The SFS is a well-validated measure of social functioning that has been specifically designed to be relevant to individuals with schizophrenia. The SFS measures seven domains of social functioning: withdrawal, interpersonal communication, independence performance, independence competence, recreation, prosocial behaviour, and employment/occupation. The SFS consists of two parts: a self-report questionnaire and a relative/carer questionnaire. Only the self-report element was used in the current study. The employment questionnaire is a novel, two-page questionnaire that asks for details regarding an individual’s employment situation including employment status, hours of work per week, job description, source of income, and work performance problems. Participants were classified as employed if they were currently engaged in any form of paid employment or full-time higher education.
RBANS and the Scale for the Assessment of Negative Symptoms (SANS) (Reference Andreasen32) data were obtained from stored data held by the ASRB for each participant (Reference Loughland, Draganic and Mccabe28). The RBANS measures five domains of cognitive functioning: immediate memory, visuospatial ability, language, attention, and delayed memory. The measure provides domain and total scores with an average of 100 and standard deviation of 15. The SANS rates negative symptoms in individuals with schizophrenia. There are five symptoms complexes on the SANS, of which four were available on the ASRB databank: blunted affect, alogia, avolition, and asociality. SANS ratings are scored on a six-point scale (0=not at all to 5=severe).
Procedure
The ASRB staff contacted all participants that met the inclusion criteria from their volunteer pool on behalf of the researchers. Interested volunteers, with their consent, had their contact details forwarded to the research team. Appointments were organised with each volunteer and prior to the testing session participants were sent an information statement, consent form, DASS21, SFS, and employment questionnaire. They were instructed to complete these items and bring them to the pre-arranged testing session.
The MMSE was administered first by the researcher and took ∼10 min to complete. After establishing the participants’ ability to read and write, and providing a brief preamble, the participants completed the ARCS independently. The audio recorded portion of the test ran for 34 min.
Data analysis
JMP version 11 statistical software from SAS was used for all statistical analyses. The characteristics of the schizophrenia and control groups were compared with t-tests for continuous variables and χ2-tests for categorical variables. Between-group differences on ARCS, RBANS, and MMSE scores were assessed with t-tests. General linear models, specifically effects tests, were used to examine the relationships between scores on the cognitive measures and demographic variables. Relationships between continuous variables were assessed using pairwise correlations, while relationships between ordinal and continuous variables were assessed with Spearman’s rank order correlations. Logistic regression was used to examine the association between overall ARCS score and both group membership and employment status. The likelihood ratio test was used for statistical significance. A p-value of 0.05 was considered statistically significant on all statistical tests.
Results
Sample characteristics
Table 1 reports the sample characteristics of each group. The time elapsed between the participants completing their RBANS on intake onto the ASRB register and the date of ARCS testing ranged from 11 to 72 months (M=40.4, SD=19.2). There was no significant difference between the schizophrenia and control group on time between testing sessions. There was no significant difference between the mean age of the schizophrenia and control groups. There was a significantly higher percentage of males in the schizophrenia group (68.4%) than in the control group (26.3%). The groups differed significantly in education level with the majority of the control group having completed university education (89.5%) compared with 31.6% of the schizophrenia group. A significantly higher proportion of participants in the control group (89.5%) were employed compared with the schizophrenia group (26.3%).
Table 1 Sample characteristics
ARCS, Audio Recorded Cognitive Screen; ASRB, Australian Schizophrenia Research Bank; DASS, Depression Anxiety Stress Scales; SANS, Scale for the Assessment of Negative Symptom.
The majority of participants in the schizophrenia group were taking atypical antipsychotics (n=16). One participant in the schizophrenia group was taking a typical antipsychotic and two participants were not taking any psychotropic medication at all. Eight of the participants in the schizophrenia group were also taking one or more other medications (Lithium=3, Venlafaxine=2, Escitalopram=2, Sodium Valproate=2, Reboxetine=1, Alprazolam=1, Nitrazepam=1, Diazepam=1). None of the control group was currently taking any psychotropic medications.
Measures of clinical status indicate the schizophrenia group were experiencing low to moderate levels of negative symptoms and emotional disturbance at time of testing. The schizophrenia group’s SANS scores put them in the questionable to mild range for blunted affect, alogia, avolition, and asociality. The schizophrenia group had significantly higher levels of depression, anxiety, and stress than the control group on the DASS21. The schizophrenia group’s mean DASS21 scores put them in the moderate range for depression, anxiety, and stress. The mean DASS21 scores for the control group put them in the normal range for depression and stress, and the mild range for anxiety.
Acceptability of the ARCS
All participants in the schizophrenia and control groups were able to complete the ARCS, suggesting the instrument is feasible to conduct with individuals with psychotic disorders. On the basis of their responses to the questionnaire at the end of the ARCS response booklet, all participants in both groups indicated that they could hear the audio instructions clearly, experienced no technical difficulties, and did not experience any external distractions. All participants in the control group and all but one participant (94.7%) in the schizophrenia group indicated that the directions were not confusing or difficult to follow. All participants in the control group and the majority of participants in the schizophrenia group (89.5%) reported that they gave their best effort on the test. The majority of participants in the schizophrenia group indicated that they felt they performed satisfactorily or better (94.7%) and there was no significant difference between the two groups on subjective evaluation of their test performance [t(36)=1.85, p=0.07], suggesting that the ARCS is acceptable to individuals with psychotic disorders. A higher proportion of participants in the schizophrenia group (22.2%) than the control group (0%) indicated that they believed anxiety affected their performance on the test [χ2(1)=6.28, p=0.01].
Between-group differences in performance on cognitive measures
Table 2 shows mean and standard deviations for ARCS scaled scores, RBANS index scores, and MMSE score for the schizophrenia and control groups, differences between the means, 95% confidence intervals, and the statistical significance of group differences. All ARCS variables appeared normally distributed apart from the visuospatial domain, which had an abnormal distribution for the schizophrenia group. The distribution was abmornal in that most of the scores were clustered within 1 SD of the mean (74%), but four participants (21.05%) scored extremely low (below 2 SD from the mean) giving the distribution a long tail.
Table 2 Means and standard deviations on cognitive measures and group comparisons
ARCS, Audio Recorded Cognitive Screen; MMSE, Mini Mental State Examination; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status.
As expected, the schizophrenia group performed significantly worse than the control group on the ARCS memory, fluency, QuickARCS, and overall scores. However, the schizophrenia group performed better than expected on the ARCS attention domain and there was no significant difference between the two groups on this domain. In line with the general pattern of cognitive impairment in schizophrenia (Reference Heinrichs and Zakzanis33), there was no significant difference between the two groups on the ARCS visuospatial and language domains. For a more detailed analysis of the pattern of memory impairment, we also compared the difference between the two groups on scaled immediate recall, delayed recall, and delayed recognition scores. The schizophrenia group performed significantly worse on immediate recall and delayed recall. There was no significant difference between the two groups on delayed recognition, suggesting this process is relatively spared compared with free recall in individuals with schizophrenia. As expected the control group performed within 0.5 SD of the population mean on all ARCS domains (Reference Schofield, Lee and Lewin25). Meanwhile, the schizophrenia group performed more than 1 SD below the population mean on the QuickARCS and ARCS overall score, suggesting that the ARCS is sensitive to the cognitive deficit associated with schizophrenia.
Given the differences between the two groups on sex and education level, a general linear model was used to investigate whether these factors had any additional influence on ARCS score unaccounted for by the ARCS norming process. The model revealed no effect for age [F(1,32)=0.01, p=0.91], sex [F(1,32)=0.12, p=0.73], or education level [F(2,32)=0.26, p=0.77] on ARCS overall score. There was a significant effect for the schizophrenia group [F(1,32)=5.22, p=0.03] on ARCS overall score providing support for the ARCS ability to discriminate between schizophrenia and healthy controls.
On the RBANS, the schizophrenia group performed significantly worse than the control group on the immediate memory, language, attention, delayed memory, and total scores. There was no significant difference between the two groups on the visuospatial domain. The control group was within 0.5 SD of the population mean on all RBANS index scores except for language where they scored more than 0.5 SD higher than the population mean, perhaps reflecting their high level of education. The schizophrenia group performed more than 1 SD below the population mean on the RBANS immediate memory index. All other scores in the schizophrenia group were within the normal range. As previous studies show associations between RBANS scores and education level in schizophrenia groups (Reference Wilk, Gold, Humber, Dickerson, Fenton and Buchanana21) we used a general linear model to investigate the impact of demographic factors on RBANS scores. There was a significant effect for group on RBANS total score [F(1,32)=6.63, p=0.01]. There was no significant effect for age [F(1,32)=3.92, p=0.06], sex [F(1,32)=0.30, p=0.59], or education level [F(2,32)=2.24, p=0.12] on RBANS total score.
The mean performance of the schizophrenia group was significantly worse than the control group on the MMSE on t-test, however, these differences were found to be driven by differences in the groups’ demographic variables rather than cognitive functioning. The MMSE has been shown to be associated with age and education level (Reference Crum, Anthony, Bassett and Folstein34). A general linear model revealed significant effects for age [F(1,32)=4.91, p=0.03] on MMSE score. There was no effect for sex [F(1,32)=3.44, p=0.07], education [F(2,32)=0.19, p=0.83], or group [F(1,32)=0.58, p=0.45] on MMSE score. When we removed the least significant variable, education, from the model there were significant effects for age [F(1,34)=5.05, p=0.03] and sex [F(1,34)=4.36, p=0.04] on MMSE score. MMSE performance declined with age and females performed better than males. After controlling for the effects of age and sex there was no significant difference between the schizophrenia and control groups on MMSE performance [F(1,34)=1.66, p=0.21], indicating that the MMSE was a poor discriminating measure between the two groups (Table 2).
Table 3 Correlations among ARCS domain scores (n=38)
ARCS, Audio Recorded Cognitive Screen.
*p<0.05; **p<0.01; ***p<0.001.
Correlations between ARCS and RBANS and MMSE scores
The correlations between the ARCS scaled domain and RBANS index scores are detailed in Table 4. As predicted, there were moderate to strong significant correlations between corresponding ARCS and RBANS domains in immediate memory, delayed memory, and overall scores. Unexpectedly, there were no significant correlations between the ARCS and RBANS visuospatial ability, language, and attention domains. MMSE score correlated significantly with ARCS total score (r=0.64, p<0.001) and RBANS total score (r=0.47, p=0.003).
Table 4 Correlations between ARCS scaled domains and RBANS index scores (n=38)
ARCS, Audio Recorded Cognitive Screen; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status
*p<0.05; **p<0.01; ***p<0.001.
Associations between cognitive measures and functional outcomes
Logistic regression was performed to examine the likelihood of being employed including both group membership and ARCS overall score as predictors. Overall 57.9% of the sample were employed (n=22) and 42.1% of the sample were unemployed (n=16). As expected, the effects were statistically significant for both ARCS score [χ2(1)=7.16, p=0.007] and group [χ2(1)=7.74, p=0.005], indicating that ARCS score and schizophrenia are both reliable independent predictors of a participant’s likelihood of being employed. The odds ratio was calculated for a 1 SD drop in overall ARCS score. For every 1 SD drop in overall ARCS score, the participants had a 5.32 greater chance of being unemployed, 95% CI [1.21, 23.4]. Participants with schizophrenia were 13.3 times more likely to be unemployed than healthy controls, OR=13.3, 95% CI [2.08, 138.3]. There was no significant interaction between ARCS overall score and group [χ2(1)=0.04, p=0.83]. Within the schizophrenia group, there were significant positive correlations between employment hours per week and ARCS attention (r=0.51, p=0.03) and QuickARCS (r=0.49, p=0.03) scores. These results suggest that cognitive functioning as measured by the ARCS is a significant factor in the likelihood of an individual with schizophrenia being in paid employment or full time education and is significantly related to how many hours of employment an individual is engaged in. However, there appear to be other factors independent of cognition that are also involved in vocational functioning in schizophrenia.
As the two groups differed in sex and education level, there is a possibility that those variables may have influenced the relationship between ARCS performance and employment status. Also, self-reported anxiety was shown to be differentially associated with ARCS performance in the two groups. Due to the small sample size we could not add further variables to the above regression analysis as it may have over-fitted the model, so we ran separate logistic regression analyses to test for the effects of sex, education level, and DASS21 anxiety on employment status. We assessed the likelihood of being employed with sex and group as predictors. There was a significant effect for group [χ2(1)=17.36, p<0.001], but no significant effect for sex [χ2(1)=1.19, p=0.27] on employment status. We also assessed the likelihood of being employed with education level and group as predictors. Again, there was a significant effect for group [χ2(1)=17.36, p<0.001], but no significant effect for education level on employment status [χ2(2)=0.71, p=0.70]. We assessed the likelihood of being employed with DASS21 anxiety and group as predictors. There was a significant effect for group [χ2(1)=12.52, p<0.001], but no significant effect for DASS21 anxiety on employment status [χ2(1)=1.98, p=0.16]. Therefore, in the current sample it appears that sex, education level, and anxiety were not related to employment status and we can have more confidence that the association between ARCS performance and employment status is a true relationship.
We also assessed the associations between RBANS and MMSE performance and employment status. Logistic regression was conducted on employment status using group and RBANS overall score as predictors. In this analysis, employment status was determined from data collected from the ASRB at time of intake and is therefore contemporaneous with the administration of the RBANS. Overall 57.9% of the sample (89.5% of the control group, 26.3% of the schizophrenia group) was employed. There was a significant effect for group on employment status [χ2(1)=10.49, p=0.001], but there was no significant effect for RBANS total score on employment status [χ2(1)=0.13, p=0.72]. There was no significant interaction between RBANS total score and group [χ2(1)=0.02, p=0.89]. Similarly, we conducted a regression analysis on current employment status with MMSE score and group as predictors. Again, there was a significant effect for group [χ2(1)=11.13, p<0.001], but no significant effect for MMSE score on employment status [χ2(1)=0.92, p=0.34]. There was no significant interaction between MMSE score and group on employment [χ2(1)=0.81, p=0.37]. The RBANS and the MMSE were not significant predictors of the likelihood of being employed in the current sample.
Table 5 displays the correlations between ARCS scores and SFS scores within the entire sample (n=38). As expected, there were several correlations between ARCS and SFS scores. The strongest correlations were between QuickARCS and SFS interaction, employment/occupation, and overall scores. The ARCS attention domain had the most number of significant correlations with SFS domain scores, correlating with independence (competence), recreation, prosocial behaviour, employment/occupation, and overall. ARCS memory had a significant relationship with employment/occupation. ARCS fluency was significantly associated with withdrawal and interaction.
Table 5 Correlations between ARCS and SFS domain scores (n=38)
ARCS, Audio Recorded Cognitive Screen; SFS, Social Functioning Scale.
*p<0.05; **p<0.01; ***p<0.001.
As we were mainly interested in the relationship between ARCS performance and social functioning in individuals with schizophrenia, we then excluded the control group from the analysis. When we analysed the correlations between ARCS scores and SFS scores in the schizophrenia group (n=19) only, the overall pattern of associations remained, but none of the relationships reached significance. ARCS fluency had modest non-significant correlations with withdrawal (r=0.39, p=0.10) and interaction (r=0.41, p=0.08). There were also non-significant relationships between ARCS attention and SFS independence (competence) (r=0.30, p=0.20), recreation (r=0.32, p=0.18), prosocial behaviour (r=0.32, p=0.18), employment/occupation (r=0.33, p=0.17), and overall scores (r=0.35, p=0.14).
Discussion
In a comparison between the performance of relatively stable, community-dwelling individuals with schizophrenia and healthy controls, the ARCS was able to discriminate between the two groups on memory, fluency, and overall score. These results suggest the ARCS is sensitive to the cognitive impairment associated with schizophrenia, with the memory and fluency domains being the main discriminating measures between the two groups. This finding is consistent with the findings of the previous pilot study (Reference Loughland, Allen and Gianacas26), as well as with the demonstrated pattern of impairment reported in the literature (Reference Heinrichs and Zakzanis33). Previous research consistently shows individuals with schizophrenia have significant impairment in memory, attention, and executive functioning, with visuospatial ability and vocabulary the least affected areas of cognition (Reference Heinrichs and Zakzanis33). The lack of significant difference between the two groups on the ARCS attention domain observed in the current study may be a consequence of our sampling strategy. Register-recruited participants such as those in the current study, have been shown to be higher functioning than those recruited from psychiatric treatment settings, which are often convenient recruitment pools for research (Reference Loughland, Carr, Lewin, Barnard, Chapman and Walton35). They are also more likely to be familiar with cognitive testing and testing environments. Register-recruited samples have been shown to have impairment in memory, but near normal levels of performance on measures of attention, visuospatial ability, and language (Reference Loughland, Lewin, Carr, Sheedy and Harris36), as reflected in our study.
The ARCS performed well when compared with other commonly used screening tools. There was a positive relationship between scores on the ARCS and the MMSE, however, the MMSE was shown to be less effective than the ARCS at distinguishing between individuals with schizophrenia and healthy controls. Once demographic factors were accounted for, the differences between the two groups detected by the MMSE disappeared, which is consistent with previous research that shows the MMSE may not be sensitive enough to detect the cognitive impairment associated with schizophrenia (Reference Faustman, Moses and Csernansky23,Reference Ganguli, Brar and Vemulapalli37). On the other hand, ARCS scores were found to be independent of age, sex, and education level, providing further support for the effectiveness of the ARCS scaling process and the appropriateness of this instrument for assessing cognitive impairment in schizophrenia.
The ARCS and the RBANS compared relatively well despite the fact that current ARCS data were compared with RBANS data from up to 6 years previously (M=3.37 years). The ARCS and RBANS correlated on memory and overall scores, suggesting a high degree of overlap between the two tests in detecting deficits in these areas as well as a degree of stability in the impairment in memory and overall cognition in this sample (Reference Censits, Ragland, Gur and Gur9–Reference Addington, Saaedi and Addinton11). The magnitude of the correlations between ARCS and RBANS memory and overall scores were very similar to those demonstrated in the previous pilot study (r=0.49, r=0.51, respectively) (Reference Loughland, Allen and Gianacas26). The lack of correlation between ARCS and RBANS visuospatial scores is also consistent with the pilot study, suggesting that the different tasks used to measure visuospatial ability in the ARCS (clock drawing task) and RBANS (complex figure copy and line orientation task) may involve different cognitive processes. The lack of significant correlations between the corresponding ARCS and RBANS language and attention domain scores is inconsistent with the pilot study, which demonstrated modest correlations between the two tests on these measures (r=0.35, r=0.42, respectively). This inconsistency may be due to the time delay between the administration of the RBANS and the ARCS in the current study. The RBANS data used may not be an accurate representation of the participants’ current cognitive functioning and therefore not an optimal basis for comparison with contemporaneous ARCS performance. In addition, the lack of significant correlations between the corresponding ARCS and RBANS language and attention domain scores in the current study may be a product of the sample size and lack of power in the current study.
Our results provide preliminary support for the ARCS as a predictor of functioning in individuals with schizophrenia. The ARCS was a significant predictor of employment status, with higher overall ARCS scores leading to a higher probability of employment. In addition, better performance on attention and QuickARCS was related to increased hours of employment per week. RBANS and MMSE performance on the other hand, were not found to be associated with employment status in this sample. These findings are consistent with the previous pilot study, which found preliminary evidence that performance on the ARCS may be associated with functional outcomes in individuals with schizophrenia (Reference Loughland, Allen and Gianacas26). These findings are also consistent with the literature demonstrating a consistent association between cognitive impairment and employment status in individuals with schizophrenia (Reference Mcgurk and Meltzer14,Reference Hofer, Baumgartner and Bodber15). The association between ARCS attention scores and employment hours is in line with a previous study, which found that performance on tests of attention and executive functioning can differentiate between individuals with schizophrenia who are employed on a full-time basis versus a part-time basis (Reference Mcgurk and Meltzer14).
In regards to social functioning ARCS scores were significantly correlated with several domains on the SFS in the whole sample, however, when these relationships were assessed in the schizophrenia group only they did not reach significance. These results are not inconsistent with previous research that has found that more objective outcome measures such as employment status may be more valid measures of functional outcome than self-report measures (Reference Hofer, Baumgartner and Bodber15). This may explain why in the current study employment status but not scores on the SFS were significantly associated with ARCS scores in the schizophrenia group. In addition, Hofer et al. (Reference Hofer, Baumgartner and Bodber15) proposed that the nature of work settings may make them particularly intolerant of impaired cognitive functioning. On the other hand, an individual’s subjective satisfaction with life may be relatively independent of cognitive functioning. The literature suggests that other factors are likely to be involved in social functioning in individuals with schizophrenia such as clinical symptoms (Reference Hofer, Baumgartner and Bodber15) and social cognition (Reference Fett, Viechtbauer, Dominguez, Penn, Van Os and Krabbendam38). Our finding that some clinical symptoms were associated with outcomes on the SFS is consistent with this theory.
There are several limitations to the current research. An important limitation relates to the comparison of newly collected ARCS data with data from the ASRB, collected at varying times in the past. We chose to take advantage of RBANS data that were available on the ASRB rather than readminister the RBANS to the participants in the current study because of resource limitations and to reduce participant burden. Cognitive data from the past may not necessarily serve as an accurate proxy for current cognitive functioning. However, a large number of studies suggest that cognitive impairment in schizophrenia is fairly stable from first episode through to late middle age (Reference Censits, Ragland, Gur and Gur9–Reference Addington, Saaedi and Addinton11,Reference Hoff, Sventina, Shields, Stewart and Delisi39–Reference Szoke, Trandafir, Dupont, Meary, Schurhoff and Leboyer41). Most studies have found improvements in performance of individuals with schizophrenia on cognitive tests over time, but this has largely been matched by improvements in control groups (Reference Heaton, Gladsjo, Palmer, Kuck, Marcotte and Jeste10,Reference Addington, Saaedi and Addinton11,Reference Szoke, Trandafir, Dupont, Meary, Schurhoff and Leboyer41). Thus, we felt it would be worthwhile to include the participants RBANS scores obtained from the ASRB register in the current study. It would also have been advantageous to conduct the SANS and the Scale for the Assessment of Positive Symptoms in the current study, rather than relying on clinical data held on the ASRB. This would have enabled us to have a better picture of the current clinical state of the sample.
There are also limitations in regards to our sampling. We hoped to recruit a larger number of participants than that of the previous pilot study, but take-up of the project was low. The small sample size limited the number of variables we could enter into regression analyses and compromised the precision of the statistical tests. In addition, our sample comprised community-dwelling individuals with schizophrenia who had agreed to be contacted regarding participation in research projects. As discussed above, it is likely research-register samples reflect a relatively high functioning subsection of individuals with schizophrenia (Reference Loughland, Carr, Lewin, Barnard, Chapman and Walton35) and our findings may not be generalisable to lower functioning and inpatient populations of individuals with schizophrenia. Also, the inclusion of participants with schizoaffective disorder in the sample means that we cannot generalise the findings of the current study to patients with schizophrenia alone.
In regards to limitations with the ARCS itself, the current study identified a potential issue with the clock drawing task used to measure visuospatial ability. The majority of participants who scored poorly on the ARCS clock drawing task did so because they set the time on the clock, but failed to write the numbers on the clock. This may have been because while the audio recording explicitly instructed them to write the numbers on the clock, the written instructions on the response booklet only instructed them to set the time to 10 past 11. Therefore, these low scores may be more an artefact of not understanding the concept of the task and failing to respond to both the audio and visual stimuli rather than a difficulty with visuospatial functioning. This may help to explain the pattern of results in the current study of most participants in the schizophrenia group scoring within the normal range on the visuospatial domain, but some participants scoring extremely low. The ARCS has since been modified so that the audio and visual instructions are more consistent.
The current study provides support for the ARCS as an acceptable and appropriate assessment option for practitioners working with relatively stable individuals with schizophrenia in the community. Such a population would be likely recipients of such a screening instrument in clinical practice. The ARCS may be particularly attractive to clinicians in these settings due to its efficiency in terms of time and organisational resources. The finding that ARCS scores are associated with functional outcomes, particularly employment, suggests that performance on the ARCS has real-life implications for individuals with schizophrenia and could be useful in guiding treatment planning. Cognitive remediation programs have demonstrated success in improving cognition in individuals with schizophrenia (Reference Wykes, Huddy, Cellard, Mcgurk and Czobor42) and there is a growing body of research identifying psychopharmacological treatments that may improve cognition in this population (Reference Ibrahim and Tamminga43).
Due to the limitations of the sample detailed above our findings can only be viewed as preliminary. Further evaluation of the ARCS in the context of administration to people with schizophrenia could be undertaken by comparing relatively contemporaneous ARCS data with the results from gold standard neuropsychological tests such as the MATRICS battery in a large sample. Such studies would benefit by inclusion of a broader range of functional measures, including measures of occupational engagement with which to compare with the alternate forms of cognitive assessment. Also, while the ARCS has been shown to be feasible and acceptable in community-dwelling individuals with schizophrenia, its acceptability with more acute or inpatient populations remains untested.
Acknowledgements
The authors would like to acknowledge: Anna Sureav and Tammy Moore from Hunter New England Mental Health for their assistance with ethics and research governance; Kim Colyvas from the University of Newcastle Statistical Support Service for his support in study design and data analysis and interpretation; Peter McGeorge and Paula Costa from the O’Brien Centre, St Vincent’s for the use their facilities; and Jason Bridge and Janette Howell from the Schizophrenia Research Institute for their support in accessing volunteers and data. This study was supported by the Australian Schizophrenia Research Bank (ASRB), which is supported by the National Health and Medical Research Council of Australia, the Pratt Foundation, Ramsay Health Care, the Viertel Charitable Foundation and the Schizophrenia Research Institute. Brooke Gelder contributed to the conception and design of the research, collection, analysis, and interpretation of data, drafting of the article, and final approval of the version to be published. Carmel Loughland contributed to the conception and design of the research, interpretation of data, critical revision of the article, and final approval of the version to be published. Vaughan Carr, on behalf of the ASRB, contributed to reviewing and approving the design of the research, the collection of data, and revision and final approval of the version to be published. Peter Schofield contributed to the conception and design of the research, interpretation of data, critical revision of the article, and final approval of the version to be published.
Financial Support
This study was supported by the School of Psychology, University of Newcastle Australia; and the Neuropsychiatry Service, Hunter New England Mental Health.
Conflicts of Interest
Author Peter Schofield was involved in the development of the ARCS, but he is unlikely to receive any personal financial benefit from recommending its ongoing use.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Ethical approval of the study was granted by the Hunter New England Human Research Ethics Committee (HNEHREC ref no: 12/06/20/4.06).
Open access
