Reading tests and cognitive impairment

Scores on both WTAR and NART have been shown to be valid estimators of prior (Crawford et al. Reference Crawford, Deary, Starr and Whalley2001; Deary et al. Reference Deary, Whalley and Crawford2004; McGurn et al. Reference McGurn, Starr, Topfer, Pattie, Whiteman, Lemmon, Whalley and Deary2004; Dykiert & Deary, Reference Dykiert and Deary2013) and premorbid (McGurn et al. Reference McGurn, Starr, Topfer, Pattie, Whiteman, Lemmon, Whalley and Deary2004; Green et al. Reference Green, Melo, Christensen, Ngo, Monette and Bradbury2008) cognitive ability. However, there have been reports of positive moderate correlations between scores on the reading tests and on the Mini-Mental State Examination (MMSE, Folstein et al. Reference Folstein, Folstein and McHugh1975), a widely used screening test for potential cognitive impairment [r = 0.56 in AD patients (Patterson et al. Reference Patterson, Graham and Hodges1994); r = 0.51 in a group with dementia of mixed aetiology (McGurn, et al. Reference McGurn, Starr, Topfer, Pattie, Whiteman, Lemmon, Whalley and Deary2004); and r = 0.49 in healthy older adults (Starr et al. Reference Starr, Whalley, Inch and Shering1992)].

The positive correlation between the reading tests and MMSE scores might mean that performance on the reading tests, which is meant to be stable in cognitive decline, is actually affected in cognitive impairment. Some existing research seems to support this possibility. For example, differences in both WTAR and NART scores were reported for patients with varying dementia severity, who were generally well-matched and expected to have similar premorbid ability levels (Stebbins et al. Reference Stebbins, Wilson, Gilley, Bernard and Fox1990; O'Carroll et al. Reference O'Carroll, Prentice, Murray, Van Beck, Ebmeier and Goodwin1995; McFarlane et al. Reference McFarlane, Welch and Rodgers2006). Further, NART performance appears to improve during recovery from traumatic brain injury (Riley & Simmonds, Reference Riley and Simmonds2003). However, the same was not found for the WTAR (Green et al. Reference Green, Melo, Christensen, Ngo, Monette and Bradbury2008).

One study of healthy older people which had rarely available data on actual premorbid or prior ability (IQ measured in childhood) provided an alternative explanation for the NART–MMSE association (Crawford et al. Reference Crawford, Deary, Starr and Whalley2001). It suggested that the apparent sensitivity of the reading tests to the severity of cognitive impairment (the NART–MMSE correlation) may be caused by the MMSE's being associated with prior cognitive ability level. In that study a significant correlation in older people between NART and MMSE (which might have indicated NART's sensitivity to the degree of cognitive impairment) was abolished after controlling for actual prior (childhood) mental ability. It suggested that the relationship between NART and MMSE largely reflects the variance shared between these measures and prior cognitive ability, rather than suggesting that NART scores depend on the degree of cognitive impairment. To our knowledge, no study has investigated WTAR in this respect.

Aims of the study

The present study aimed to investigate whether the WTAR is related to MMSE score, independently of actual prior cognitive ability. We did this by testing whether the correlation between WTAR and MMSE in a large sample of healthy older people is substantially attenuated when childhood IQ is controlled. We also aimed to replicate the finding of Crawford et al. (Reference Crawford, Deary, Starr and Whalley2001) for the NART–MMSE association in our sample. Moreover, we tested whether education accounts for some of the association between WTAR–NART and MMSE.

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Does prior IQ account for MMSE–WTAR/NART correlations?

To test whether the correlations between the reading tests and MMSE reflect their shared sensitivity to differences in prior cognitive ability, we computed partial correlations between WTAR and MMSE and between NART and MMSE, controlling for age 11 IQ. Partialling out actual prior ability (age 11 IQ from the MHT test) reduced the correlations of WTAR and NART with MMSE. The partial correlation between WTAR and MMSE was pr = 0.23, p < 0.001, reduced from 0.42. This is a 70% reduction in the shared variance (R ²) between WTAR and MMSE, from 17.6% to 5.3%. The partial correlation between NART and MMSE was pr = 0.20, p < 0.001, reduced from 0.41. This is a 76% reduction in the shared variance (R ²) between NART and MMSE, from 16.8% to 4.0%. A formal test for comparing correlations (Steiger, Reference Steiger1980) confirmed that the WTAR–MMSE and NART–MMSE correlations were significantly as well as substantially attenuated by partialling out prior ability (WTAR: z = 9.40, p < 0.001; NART: z = 9.98, p < 0.001).

Does education account for MMSE–WTAR/NART correlations?

All variables of interest (WTAR, NART, MMSE, age 11 IQ) were significantly related to education (Table 1). We investigated the role of education in associations between scores on the reading tests and on the MMSE. An adjustment for highest qualification obtained resulted in an attenuation of WTAR–MMSE and NART–MMSE correlations, but to a lesser degree than was seen with the before-mentioned childhood IQ adjustment. Partial correlation coefficients were pr = 0.36 and pr = 0.34, respectively, down from 0.42 and 0.41. These are 27% and 31% reductions, respectively, in the shared variance (R ²) between WTAR–NART and MMSE (from 18/17% to 13/12%). Both were significant at p < 0.001 and significantly different from their respective zero-order correlation coefficients (WTAR: z = 3.97, p < 0.001; NART: z = 4.33, p < 0.001).

To what extend do education and childhood IQ combined account for MMSE–WTAR/NART correlations?

Partial correlation coefficients, after controlling for both highest qualification and prior IQ simultaneously (WTAR: pr = 0.23; NART: pr = 0.19), were almost identical to those obtained after controlling for prior IQ alone (WTAR: pr = 0.23; NART: pr = 0.20). This suggests that education explained almost no variance in the WTAR–MMSE and NART–MMSE associations beyond that explained by age 11 IQ.

Predicting MMSE from WTAR and NART scores

Because MMSE scores are related to childhood IQ, and because most of the MMSE's variance that is shared with childhood IQ is also shared with scores on reading tests, it might be possible to predict the ‘expected’ MMSE scores for a given level of prior or premorbid cognitive ability from scores obtained on WTAR or NART (see Crawford et al. Reference Crawford, Moore and Cameron1992, for a similar approach in predicting verbal fluency performance). The expected MMSE scores, predicted from regression equations built on our sample, are presented in Table 2. We compared these predicted MMSE scores with the actual MMSE scores obtained. Mean discrepancy from the predicted MMSE was 0.08 (s.d. = 1.30) for the WTAR-predicted MMSE and 0.10 (s.d. = 1.27) for the NART-predicted MMSE. For both discrepancy scores calculated using WTAR and NART, a score of -4 or lower (i.e. an obtained MMSE score that is ⩾4 points below the predicted MMSE score) was obtained rarely, by only 1.5% of our sample. We suggest that this level of discrepancy might be used in screening participants or clients for possible cognitive decline or impairment.

Table 2.

Table for converting obtained WTAR and NART scores into predicted MMSE scores

WTAR, Wechsler Test of Adult Reading; NART, National Adult Reading Test; MMSE, Mini-Mental State Examination.

The ranges of WTAR and NART scores presented here each begin with the lowest score observed in our sample. No extrapolations are made beyond those lowest scores.

In practice, an individualized MMSE cut-off may be obtained by predicting MMSE score from WTAR or NART, using the conversion table (Table 2), and deducting 4 points. For example, for a person with a NART score of 17 (i.e. for a person with an estimated IQ of approximately 100; Nelson & Willison, Reference Nelson and Willison1991) the predicted MMSE would be 27 and the individualized cut-off, 27 – 4 = 23. Thus, for a person with an average IQ, the individualized MMSE cut-off would be ⩽23 (i.e. <24) which corresponds to the one widely used in current practice. However, for a person with a NART score of 10 (corresponding to an IQ of 95), the MMSE cut-off would be lower, ⩽22, and for people with very high IQs (scoring 49 or 50 on the NART), it would be ⩽26.

Sensitivity analysis

Partial correlations are easily understood and have been used to test similar research questions in the past (e.g. Crawford et al. Reference Crawford, Deary, Starr and Whalley2001). However, partial correlations assume that associations between variables are linear and that there are no interactions. This might not be the case, especially since we are dealing with MMSE, whose distribution deviates from normality. Because errors on the MMSE (i.e. reverse of an MMSE score) approximate a Poisson distribution, we performed sensitivity analysis, in which MMSE errors were modelled as a Poisson variate in a generalized linear model. We then calculated the reduction in deviance due to WTAR/NART before and after controlling for age 11 IQ. The results were very similar to those obtained using partial correlations: the reductions in deviance due to WTAR/NART after controlling for age 11 IQ were 75% and 78%, respectively. The reductions in deviance due to WTAR/NART after controlling for highest qualification obtained were 36% and 37%, respectively (highest qualification was not significantly associated with MMSE errors when a reading test score was also included in the model). When both qualification and age 11 IQ were controlled, the reductions in deviance due to WTAR/NART were 79% and 81%, respectively. These were only slightly greater than the reductions after controlling for age 11 IQ alone and highest qualification was not a significant predictor of MMSE in these models. We also tested interactions between predictors in each model of MMSE. None was statistically significant.

We repeated all analyses reported above replacing highest qualification with years in full-time education – an alternative, commonly used, measure of educational attainment. The pattern of results was the same.

We preformed another sensitivity analysis to investigate whether the relationships between MMSE, WTAR/NART and age 11 IQ might be driven by participants with greater degree of cognitive impairment. We excluded all cases with MMSE <24 or a diagnosis of dementia. To do this, we utilized the longitudinal data collected on the sample 3 and 6 years after the initial testing, at mean ages of 72.49 (s.d. = 0.72) and 76.24 (s.d. = 0.68). This way we could identify individuals whose performance at wave 1 might have been negatively affected by a preclinical cognitive impairment or dementia. In total, 25 participants were excluded with low MMSE or dementia diagnosis at baseline or during the 6-year follow-up (seven participants met the criteria already at wave 1, and further 18 at waves 2 and/or 3). The correlations between MMSE and WTAR/NART/age 11 IQ changed very little, indicating that they were not driven by individuals with lowest MMSE scores, and possible undiagnosed pathology, but operate even in what we defined as the normal range. The association is presented graphically in Fig. 1, in which mean scores on WTAR and NART are plotted against MMSE scores. There is a clear pattern of increasing WTAR and NART scores as a function of MMSE, which continues throughout the MMSE range.

Fig. 1.

Average Wechsler Test of Adult Reading (WTAR) and National Adult Reading Test (NART) scores by Mini-Mental State Examination (MMSE) score. Error bars represent 95% confidence intervals.

Implications for research and clinical practice

The remarkable finding from this study might be how much of MMSE–WTAR/NART association can be attributed to individual differences in a 45-min paper-and-pencil test taken at age 11. This requires further investigation, as it may have implications for both research and clinical practice. It suggests that it must be recognized how much MMSE indicates prior ability in addition to cognitive decline. This was previously shown in ‘developmentally delayed’ adults (Myers, Reference Myers1987). Here, we demonstrated that scores from MMSE administered in old age are correlated with childhood cognitive ability estimates, even in a cognitively normal sample (in fact, the LBC1936 sample used in this study is cognitively higher, on average, than the population). Practitioners using MMSE should be aware of the extent to which its scores represent prior ability. What follows is that any given MMSE score does not indicate the same amount of cognitive pathology for people who have different prior cognitive ability. Depending on whether the aim is to determine the level of cognitive functioning, in absolute terms, or to infer a possible negative change from one's pre-existing level, an adjustment to the MMSE score might be advisable. The latter is often preferred in clinical contexts because it is the within-person decline that indicates a potential pathology and it is the subjective change, specific to each individual that will negatively affect their well-being, level of functioning, etc. To that end, our results suggest that MMSE cut-offs should be adapted, based on the individual's initial ability. Because they are rarely available, WTAR or NART scores might be an acceptable, relatively easily obtainable proxy for the actual pre-existing ability level. Our data indicate that an MMSE score of 22 or 23, below the commonly used cut-off (<24) might be expected from cognitively healthy older people with below average IQ. On the other hand, a score as high as 26 among people with high IQ may already be indicative of cognitive deterioration. Our cut-off for high NART scorers is in agreement with a cut-off for dementia screening among highly educated individuals, suggested elsewhere (O'Bryant et al. Reference O'Bryant, Humphreys, Smith, Ivnik, Graff-Radford, Petersen and Lucas2008)

A second finding with a potential clinical relevance and requiring further investigation is the small remaining correlation between the premorbid ability tests and the MMSE after adjustment for prior ability. This might mean that the reading tests are somewhat sensitive to cognitive impairment. However, the more likely explanation is that the degree of attenuation of the relationship between MMSE and the reading tests by controlling for prior ability is an underestimate. For example, whatever the MMSE and WTAR/NART share with respect to prior ability may not be fully captured by the MHT, because the crystallized ability captured by the NART/WTAR peaks in adulthood, a considerable time after the MHT was administered. We expect that if we had a measure of prior ability that was taken in early or middle adulthood and based on a more extensive set of tests, the attenuation might be higher or even near-to-complete. Studies with a premorbid/prior ability estimate from middle age, close to reaching their maximum crystallized ability but before any age-related cognitive decline is expected, might make a valuable contribution.

Strengths and limitations

The principal strength of the study is the availability of a large sample of older adults, from a single year-of-birth cohort, for whom cognitive ability was tested in childhood as well as in older age. The association of WTAR/NART, education and MMSE might vary as a function of age, both because more neurodegenerative problems become apparent with ageing, and because of cohort differences in education. The narrow-age cohort used in our study provides a natural control for these potential effects. The availability of validated IQ-type test results from age 11 enabled us to test which associations found in later life stem from individual differences in cognitive ability in childhood. This was also an ideal time for the IQ test, because it is at the end of universal primary Scottish education, prior to differences in subjects being taken by different individuals in secondary education. Finally, a 6-year follow-up enabled a sensitivity analysis, excluding participants who, after the initial testing occasion, developed cognitive impairment indicative of a possible pathology.

The sample used in this study is of above-average intelligence. Our results might not generalize to people with very little education or lower cognitive ability. Given this limitation, we did not extrapolate predicted MMSE scores beyond the lowest WTAR and NART scores observed in the sample. Another limitation is the lack of prior ability measures from early to mid-adulthood. The cohort was not tested between ages 11 and 70 and so, we could only approximate their maximal prior ability from a short cognitive ability test at age 11 and from the reading tests administered at age 70. It would be useful to investigate the associations addressed in this study on a sample for whom cognitive ability data from mid-life is available.