INTRODUCTION
The diagnosis of amnestic mild cognitive impairment (MCI) requires, in part, objective evidence of memory impairment (Collie & Maruff, 2002; Davis & Rockwood, 2004; Feldman & Jacova, 2005; Petersen, 1995, 2004; Petersen et al., 1994, 1999, 2001; Winblad et al., 2004). Although there is not full consensus on the definition of a low memory score (Bennett, 2003; Dubois et al., 2007; Luis et al., 2003), many clinicians and researchers consider an age- and education-adjusted score that is at least 1.5 standard deviations (SDs) below the mean to be unusually low and sufficient to meet psychometric criteria for amnestic MCI (see two recent consensus papers: Gauthier et al., 2006; Portet et al., 2006). There has been consistent evidence that individuals with amnestic MCI (i.e., both single and multiple domain amnestic MCI) are at a significantly higher risk of progressing to dementia (10–15% per annum) than those with normal cognitive functioning (Bennett, 2003; Bischkopf et al., 2002; Bruscoli & Lovestone, 2004; DeCarli, 2003; Luis et al., 2003; Modrego, 2006; Panza et al., 2005; Shah et al., 2000; Smith et al., 2006; Tuokko & Frerichs, 2000). However, MCI remains a challenging entity to define and provide prognosis for on an individual basis.
As it is currently conceptualized and measured, MCI contains both true positives (i.e., those individuals who will progress to dementia in time) and false positives (i.e., those with long-standing and static relative weaknesses, reversible causes of poor performance on memory measures, situational influences on performance, or measurement error, broadly defined). The presence of both true and false positives in those people labeled with MCI is empirically supported by a large literature (see Table 1). The majority of research on older adults with prodromal dementia tends to focus on those who eventually progress to a diagnosis of dementia, but often fails to identify the characteristics of the large percentage of those who remain stable or are deemed to have normal cognitive abilities at a future assessment.
Studies involving patients with mild cognitive impairment at baseline who present with normal neurocognitive abilities at follow-up
When assessing for the presence of amnestic MCI, clinicians and researchers must be aware of and remain cautious against misdiagnosing a healthy older adult as having cognitive impairment. Without knowledge of base rates of low scores, it is possible to overinterpret isolated low scores and, in turn, make erroneous diagnoses. Although a cutoff of 1.5 SDs below the mean might suggest that only 7% of healthy adults will be false positives on any given memory measure, in clinical practice and research settings, multiple measures are administered and interpreted simultaneously. Blackford and LaRue (1989) noted that “… in a memory battery with many measures, the chances are substantial that at least one score will fall into the impaired range” (p. 303). A small amount of empirical literature supports this concept.
Palmer et al. (1998) presented data on 132 healthy older adults between 50 and 79 years of age (M = 63.8 years; SD = 7.7). This sample was administered a flexible battery of measures, including five memory measures [Wechsler Memory Scale–Revised (WMS-R) Logical Memory, WMS-R Visual Reproduction, Rey Osterrieth Complex Figure, Warrington's Recognition Memory Test-Words, and Warrington's Recognition Memory Test-Faces], that provided 10 age-adjusted scores. When performance on the memory measures was examined simultaneously, nearly 40% had one or more low test scores and nearly 17% had two or more low test scores (i.e., ≤1.3 SDs below the mean). When considering scores in the frankly impaired range (i.e., 2 SDs below the mean), 13% of the healthy older adults had one or more extremely low memory scores.
Brooks et al. (2007) provided the base rates of low memory scores for the Memory Module of the Neuropsychological Assessment Battery (NAB; Stern & White, 2003). Participants were 742 healthy older adults between 55 and 79 years of age (M = 68.1 years; SD = 6.6 years) obtained from the standardization sample. The NAB Memory Module consists of four measures (List Learning, Shape Learning, Story Learning, and Daily Living Memory) that provide 10 demographically-adjusted T scores. When all 10 memory scores were examined simultaneously, 30.8% of healthy older adults had one or more scores 1.5 SDs below the mean and 16.4% had one or more frankly impaired scores (i.e., below the 2nd percentile). Brooks et al. (2007) also reported that the base rates of low memory scores increase substantially as intellectual abilities decrease. For example, 56.5% of older adults with low average intellectual abilities obtained one or more low memory scores (i.e., −1.5 SDs) compared with 18.0% with superior intellectual abilities.
These studies suggest that there is substantial risk of misdiagnosing MCI using the psychometric criterion of an unusually low memory score. For example, in Brooks et al. (2007), 30.8% of healthy older adults would psychometrically meet the Petersen et al. (1994, 1999, 2001) criterion for MCI. de Rotrou et al. (2005) referred to those patients who were diagnosed with MCI at baseline, but who were later found to have returned to normal cognitive abilities, as having “accidental MCI.” It is noteworthy that there are numerous studies with longitudinal data that often contain a subset of patients who no longer meet criteria for MCI at follow-up (see Table 1 for a review of the literature). Of course, it is important to consider that the studies presented in Table 1 represent samples from various sources, that contain differential base rates of actual early dementia (i.e., clinic sample versus community sample), and who were identified using diverse methods (i.e., screening instruments alone versus more elaborate diagnostic procedures). Despite these caveats, these rates of return to normal cognitive abilities, which would be considered the potential rates of false-positive misdiagnosis at time 1, are not trivial.
In a neuropsychological assessment, accuracy in diagnosing MCI (and subsequently predicting later dementia) depends on knowing how often low memory scores are obtained in healthy people. Clinicians administer multiple tests and the results are interpreted in combination. Therefore, it is critical to be informed of the base rates of low scores across a battery of memory tests. Unfortunately, there have been very few studies in this area. The purpose of this study is to demonstrate how often healthy older adults obtain low memory scores on the Wechsler Memory Scale–Third Edition (WMS-III; Wechsler, 1997b) and consider its implications for MCI diagnosis. This descriptive study aims to improve the psychometric accuracy for detecting true memory impairments using the WMS-III, while at the same time minimizing false positives for those older adults who do not have a prodromal neurodegenerative disease.
METHOD
Participants
Participants for the present study included healthy community-dwelling older adults from the United States (n = 550), selected from the WMS-III standardization sample. The sample ranged in age from 55 to 87 years (M = 72.8 years; SD = 9.0 years) and had an average of 11.7 years of education (SD = 3.0 years). Within this study, 42% of our sample was male, 85.3% Caucasian, 8.9% African American, 4.4% Hispanic (non-Caucasian), and 1.5% grouped as “other.” Demographically-adjusted norms were not included for those listed as “other” (n = 8).
The standardization sample was recruited from 28 cities across the United States. The treatment of participants and the collection of data were done in compliance with the Helsinki Declaration. Participants were included if they were medically and psychiatrically healthy, based on a self-report questionnaire. The exclusion criteria included color-blindness, uncorrected hearing loss and/or visual impairment, upper extremity motor problems that might interfere with testing, current treatment for alcohol or drug dependence, consumption of three or more alcoholic beverages on two or more nights per week, having sought attention from a professional for memory or cognitive problems, a history of traumatic brain injury involving loss of consciousness for 5 or more min and/or requiring hospitalization for more than 24 hr, any medical or psychiatric condition that could potentially impact cognitive functioning (e.g., stroke, epilepsy, brain surgery, encephalitis, meningitis, multiple sclerosis, Parkinson's disease, Huntington's chorea, Alzheimer's disease, schizophrenia, or bipolar disorder), or currently receiving any treatment for a medical or psychiatric condition (e.g., electroconvulsive therapy or any antidepressant, anxiolytic, or neuroleptic medication; The Psychological Corporation, 1997).
Measures
The WMS-III (Wechsler, 1997b) is a battery of memory measures designed to evaluate working memory, learning, immediate and delayed recall, and recognition of information presented in verbal and visual modalities. The WMS-III was developed for adults 16 to 89 years of age and was normed using a stratified, U.S. representative sample of 1,250 healthy adults. Age-adjusted and demographically-adjusted (i.e., age, education, sex, and ethnicity) norms are available. The reader is directed to other sources (e.g., Heaton et al., 2003; Taylor & Heaton, 2001) that provide information on the process of adjusting WMS-III scores for the demographic variables.
The WMS-III is composed of four primary tests that evaluate immediate and delayed episodic memory: Logical Memory (recall and recognition for a short story); Faces (recognition of faces); Verbal Paired Associates (recall and recognition for 8 word pairs); and Family Pictures (recall for scenes involving a fictitious family). The Auditory Recognition primary score, the tests from the Working Memory Index (e.g., Letter–Number Sequencing and Spatial Span), and the five optional tests were not included in the present analyses.
Intellectual abilities were estimated using the Wechsler Test of Adult Reading (WTAR; The Psychological Corporation, 2001). The WTAR is a measure of single word reading, which requires the reading and pronunciation of words with irregular grapheme-to-phoneme translation. The WTAR does not rely on comprehension or knowledge of word meaning, but rather relies on previous learning. An advantage of the reading-recognition paradigm is that it is relatively unaffected by mild neurological changes (i.e., Lezak et al., 2004; Strauss et al., 2006).
The WTAR was co-normed with the Wechsler Adult Intelligence Scale–Third Edition (WAIS-III; Wechsler, 1997a). Therefore, the WTAR reading score can be combined with demographic variables to estimate intellectual abilities (i.e., WTAR-demographically-predicted FSIQ). The WTAR-demographically predicted FSIQ was chosen as the measure of predicted overall intellectual abilities because (a) it is brief to administer, (b) it is appropriate for use up to the age of 89 years, (c) the WTAR performance remains relatively stable in the presence of mild cognitive declines, and (d) the WTAR has very strong reliability across the older adult age groups (i.e., internal consistency reliability, r = .92–.95; test–retest reliability, r = .94; The Psychological Corporation, 1997).
Analyses
The prevalence of low WMS-III memory scores was calculated for both the age- and demographically-adjusted normative data by using four cutoff scores that might be routinely used in clinical practice or in research. The cutoffs, along with the corresponding age-adjusted standard score (M = 10; SD = 3) and the demographically-adjusted T score (M = 50; SD = 10), included (1) at or below the 16th percentile or −1 SD (i.e., SS = 7 or T = 40), (2) at or below the 9th percentile (i.e., SS = 6 or T = 36), (3) at or below the 5th percentile (i.e., SS = 5 or T = 34), and (4) at or below the 2nd percentile or −2 SDs (i.e., SS = 4 or T = 30).
The prevalence of low WMS-III age- and demographically-adjusted memory scores was examined for the entire older adult sample (ages 55–87) and across different levels of estimated intellectual abilities (WTAR-Demographics Predicted FSIQ), including unusually low (FSIQ < 80), low average (FSIQ = 80–89), average (FSIQ = 90–109), high average (FSIQ = 110–119), and superior/very superior (FSIQ = 120+). The prevalence of low age-adjusted scores WMS-III is also presented across different levels of education (e.g., less than 8 years, 9–11 years, 12 years, 13–15 years, and 16+ years). The analyses were completed with (1) all eight primary memory subtest scores, (2) the four immediate memory primary subtest scores, and (3) the four delayed memory primary subtest scores. Working memory tests (e.g., Letter–Number Sequencing and Spatial Span), secondary scores, or Index scores were not included in these analyses.
RESULTS
Age-Adjusted WMS-III Scores
The base rates of low WMS-III primary test scores in older adults, when simultaneously considering the eight primary age-adjusted scores, are presented in Table 2. In the total sample (n = 550), approximately 64% had one or more scores and nearly 18% had four or more scores at or below the 16th percentile (i.e., −1 SD). One or more memory test scores at or below the 5th percentile was found in just over 25% of older adults, and 14% had two or more scores below this cutoff. One or more frankly impaired memory scores (at or below the 2nd percentile or −2 SDs) were found in nearly 13% of healthy older adults.
Base rates of low scores (age-adjusted) on the WMS-III primary memory subtests in healthy older adults across levels of estimated intellectual abilities
Substantial differences in the prevalence of low scores were found across the levels of estimated intellectual abilities. For example, in older adults with estimated low average intellectual abilities, 43.3% had one or more scores at or below the 5th percentile compared with 21.4% with estimated high average intellectual abilities [n = 98; χ2(1) = 9.41; p = .002; odds ratio (OR) = 2.8, 95% confidence interval (CI) = 1.4–5.4]. It was uncommon (9.5%) for an older adult with estimated low average intellectual abilities to have three or more scores at or below the 5th percentile. In older adults with estimated high average intellectual abilities, it was uncommon (10.2%) to have two or more scores at or below the 5th percentile.
The base rates of low WMS-III immediate memory and delayed memory subtest scores are presented in Table 3. In the total sample, 18.9% had one or more immediate memory scores at or below the 5th percentile and 4.0% had two or more low immediate memory scores. When the base rates of immediate memory scores were stratified by level of predicted intellectual abilities, 39.2% of older adults with estimated low average intellectual abilities obtained one or more scores at or below the 5th percentile compared with 12.2% of older adults with estimated high average intellectual abilities [χ2(1) = 16.86; p < .001; OR = 4.6; 95% CI = 2.2–9.8]. With the delayed memory tests, 20.9% had one or more delayed memory scores at or below the 5th percentile and 4.0% had two or more low scores. When stratified by level of predicted intellectual abilities, 28.4% of older adults with estimated low average intellectual abilities obtained one or more low delayed memory scores at or below the 5th percentile compared with 19.3% of older adults with estimated high average intellectual abilities.
Base rates of low immediate and delayed memory subtest scores (age-adjusted) on the WMS-III in healthy older adults across levels of estimated intellectual abilities
There were notable differences in the prevalence of low WMS-III age-adjusted subtest scores across levels of education for the eight primary scores (Table 4), the four immediate memory scores (Table 5), and the four delayed memory scores (Table 5). In older adults with 8 or fewer years of education or 9–11 years, 34.9% and 41.8% had one or more primary scores at or below the 5th percentile, respectively (see Table 4). In older adults with 12 years, 13–15 years, and 16+ years of education, one or more scores at or below the 5th percentile was found in 17.7%, 14.6%, and 25.8%, respectively. These differences across levels of education were also present in both the immediate and the delayed memory scores (Table 5).
Base rates of low scores (age-adjusted) on the WMS-III primary memory subtests in healthy older adults across levels of education
Base rates of low immediate and delayed memory subtest scores (age-adjusted) on the WMS-III in healthy older adults across levels of education
Demographically-Adjusted WMS-III Scores
The prevalence of low WMS-III primary test scores in older adults is presented in Table 6. In the total sample (n = 542), over two thirds (i.e., 70%) had one or more scores at or below the 16th percentile (i.e., −1 SD) and nearly 24% had four or more scores below this cutoff. One or more memory scores at or below the 5th percentile were found in 39% of older adults, and one or more extremely low memory scores (i.e., at or below the 2nd percentile) were found in nearly 22% of healthy older adults.
Base rates of low scores (demographically-adjusted) on the WMS-III primary memory subtests in healthy older adults across levels of estimated intellectual abilities
There was a gradual decrease in the prevalence of low scores as estimated intellectual abilities increased. When considering primary scores at or below the 5th percentile, 56.3% with estimated unusually low intellectual abilities had one or more low scores. In older adults with low average, average, or high average estimated intellectual abilities, 41.7%, 38.2%, and 35.1%, respectively, had one or more scores at or below the 5th percentile. In older adults with estimated superior/very superior intellectual abilities, 31.6% had one or more scores at or below the 5th percentile.
The prevalence of low demographically-adjusted WMS-III immediate and delayed memory scores is presented in Table 7. In the total sample, 29.1% had one or more and 8.3% had two or more immediate memory scores at or below the 5th percentile. The prevalence of low immediate memory scores was fairly consistent across the levels of estimated intellectual abilities. On the delayed memory scores, 29.7% of the total sample had one or more and 8.1% had two or more scores at or below the 5th percentile. The prevalence of low delayed memory scores was fairly consistent across estimated intellectual abilities.
Base rates of low scores (demographically-adjusted) on the WMS-III immediate and delayed memory subtests in healthy older adults across levels of estimated intellectual abilities
DISCUSSION
The purpose of this study was to illustrate how often healthy older adults get low memory scores when multiple memory measures from the WMS-III are considered simultaneously. Specifically, the objectives were (a) to evaluate the frequency with which healthy older adults might meet the psychometric criteria for MCI when tested with multiple memory measures and (b) to provide tables to clinicians and researchers with known false-positive rates. This study expands the findings presented by Palmer et al. (1998), Loewenstein et al. (2006), and Brooks et al. (2007), which demonstrate that isolated low memory subtest scores are common in healthy older adults, and presents the base rates of low memory scores for the most commonly used memory battery in North America (Rabin et al., 2005).
In the present study, we examined the prevalence of low memory scores, both age- and demographically-adjusted, in a large sample of healthy older adults (n = 550 for age-adjusted; n = 542 for demographically-adjusted) between 55 and 89 years of age from the WMS-III standardization sample. Regardless of the cutoff scores used, low memory scores are increasingly frequent when multiple memory measures are administered and interpreted simultaneously. When simultaneously examining the eight age-adjusted WMS-III subtest scores, it was uncommon (i.e., <10%) for older adults to have three or more scores that were at or below the 5th percentile (i.e., 1.5 SDs below the mean). Across the eight demographically-adjusted scores, it was also uncommon (i.e., <10%) to have three or more low scores. If we simultaneously consider only the four delayed memory scores, it was uncommon to obtain two or more scores at or below the 5th percentile for both the age- and demographically-adjusted scores. In other words, it was common for healthy older adults to get zero or one low delayed memory subtest score. The fact that it is common for healthy older adults to have one WMS-III delayed memory subtest score 1.5 SDs below the mean calls into question the validity of the current psychometric criteria for MCI (i.e., a memory score more than 1.5 SDs below the mean). Essentially, if the four delayed memory scores are interpreted simultaneously then 20% (age-adjusted normative scores) to 30% (demographically-adjusted normative scores) of healthy older adults will meet psychometric criteria for MCI. If the clinician or researcher requires two or more memory scores (considering all 8 immediate and delayed scores) to fall 1.5 SDs below the mean, then the presumed false-positive rate for MCI would be 14.1% using the age-adjusted norms and 23.1% using the demographic-adjusted norms. Thus, researchers and clinicians should consider these prevalence of low scores data carefully when evaluating individual patients or designing inclusion criteria for prodromal dementia studies.
The inclusion of both age- and demographically-adjusted scores is an important contribution of this study. Having both types of WMS-III normative scores provides clinicians and researchers with a substantial amount of important information when interpreting test performance in older adults. Correcting test scores for demographic variables is generally viewed as an important contribution to neuropsychology for reducing the likelihood of misclassifying cognitive status in persons with lower education or in certain ethnic groups (Collie et al., 1999; Heaton et al., 1996, 2003, 2004; Marcopulos et al., 1999), although there is still work to be done to properly account for differences that might affect test performance (e.g., Manly, 2005, 2006; Manly & Echemendia, 2007). If using the demographically-adjusted scores is not appropriate, then the clinician is still able to account for years of education and estimated intellectual ability when interpreting the prevalence of low WMS-III scores.
There were some notable differences in the prevalence rates of low scores between the age- and demographically- adjusted normative scores. The prevalence of low scores on the WMS-III was generally greater for the demographically- adjusted scores compared with the age-adjusted scores. For example, having at least one score at or below the 5th percentile was found in 25.7% of older adults using the age-adjusted scores but was found in 39.0% with the demographically-adjusted scores. The differences in base rates of low scores between age- and demographically- adjusted scores reflect differences in the normative processes used. Age-adjusted scores were derived using normalized Z-score transformation of the subtest mid-point percentiles. This norming method enables subtests that have skewness, which is common in neuropsychological tests, to have comparable percentile scores across measures. Regression based approaches, which were used when adjusting for demographic factors, do not account for skewness or heteroscedasticity along the regression line, regardless if it is a linear or nonlinear approach. This can result in regressed norms that may over- or underestimate the theoretical relationship between the actual mid-point percentile and the standardized score (Holdnack, 2007). In addition, the change in metric from a scaled score (M = 10; SD = 3) to a T score (M = 50; SD = 10) returns a greater level of precision than the original measurement unit (e.g., 1/3 scaled score units to 1/10 scaled score units). This change in metric may be accounted for by rounding scores up (because percentiles represent the percentage of cases below a specific score) to the nearest third of a standard deviation unit (e.g., 37.5 to 40 or 40.5 to 43 or 43.5 to 47, and so forth). Despite the methodological concerns associated with regression-based norms, these issues do not invalidate the need for using demographic adjustments when interpreting the WMS-III scores. Applying the base rate data provided in this study aids the clinician in a more sophisticated use of these norms for clinical and diagnostic decision making. When using the demographically-adjusted normative data for the WMS-III subtest scores, the clinician should appreciate that healthy adults are likely to get low scores (more low scores than with the age-adjusted norms). Using the tables presented in this study will help the clinician put memory performance in context—and not overinterpret isolated low test scores.
Clinicians and researchers are often concerned with identifying those tests that are most sensitive to memory impairment found in very early dementing illnesses. Unfortunately, this information cannot be determined with the present data for two reasons. First, no clinical subjects with known early dementia were included in this study. Second, based on the normalizing method for creating the age-adjusted scaled scores, there should not be a difference in base rates of low scores from subtest to subtest in the older adults standardization sample. In other words, the test performance was used to create the normative scores and thus, all scores are normally distributed. Knowing which tests are more sensitive to early dementia requires an examination of test performance in a clinical sample of patients. With the demographically-adjusted norms, differences can and do emerge (as seen in the Results section). However, presenting data at a subtest-by-subtest level and evaluating this information in a clinical sample is beyond the scope of this study.
Consistent with the results presented by Brooks et al. (2007), the present study also found that older adults with lesser intellectual abilities have higher base rates of low WMS-III scores and would be more likely to be misclassified as having MCI. For example, nearly 30% of healthy older adults with low average intellectual abilities will meet psychometric criteria for MCI. The difference in the prevalence rates across estimated intellectual abilities was more apparent with the age-adjusted scores than with the demographically-adjusted scores. It is possible that correcting for demographic variables, particularly education, attenuates the relationship between WTAR-demographics predicted intellectual ability and the prevalence of low scores. WTAR performance is positively correlated with both intelligence and education.
The prevalence of low age-adjusted scores was also stratified by level of education. Older adults with lower levels of education are more likely to obtain low scores compared with those with higher levels of education. That is, older adults with fewer years of education are more likely to be misdiagnosed as having cognitive impairment and those with more years of education are more likely to have their cognitive problems missed in a diagnosis. It is unclear why older adults with 16 or more years of education had base rates for the delayed memory scores that were higher than those with 12 years or 13–15 years and more similar to those with 9–11 years of education. However, this is likely a sampling artifact related to differences in the number of cases found in these groups. Heaton et al. (2003) previously demonstrated, using the age-adjusted WAIS-III/WMS-III Index (factor) scores, that having fewer years of education placed a healthy adult at risk for being misdiagnosed as having cognitive impairment. In addition, Heaton et al. (2003) suggested that those with more than a high school level of education would need to demonstrate a more substantial decline in cognitive abilities compared with those with less than high school to be identified as having cognitive impairment. Clearly, clinicians and researchers should be concerned with both misdiagnosis and missed diagnosis of cognitive impairment.
One possible explanation for the percentage of healthy older adults with low memory scores is that human neurocognition is highly variable (Matarazzo & Prifitera, 1989) and individuals can have long-standing relative weaknesses that do not impact their day-to-day functioning. Schretlen et al. (2003) illustrated that nearly two thirds of healthy people across the lifespan had discrepancies of at least 3 SDs between their highest and lowest scores on a battery of neuropsychological measures. Even when test performance outliers were removed from the analyses, the mean discrepancy between their highest and lowest scores was still 2.7 SDs. “Both the complexity of the human central nervous system and individual differences in the organization of neural circuits on which various mental abilities depend argue against the likelihood that any individual will be endowed with identical levels of ability across all domains of cognitive functioning” (p. 864, Schretlen et al., 2003). That is, variability in neurocognitive test performance is the rule, rather than the exception. To more appropriately deal with this variability, clinicians need to be informed and consider base rates information when interpreting test performance.
The literature on older adults who are diagnosed with MCI illustrates that a certain percentage, when followed longitudinally, no longer meet criteria for MCI (see Table 1) and they are referred to as “returning to normal cognition”. de Rotrou et al. (2005) referred to this as “accidental MCI” to denote the likelihood that those people were misdiagnosed at the initial assessment. This kind of backcrossing from MCI to normal would not be consistent with a neurodegenerative disease and likely represents the inherent difficulty with accurately identifying those who will versus those who will not eventually progress to dementia. However, it is unlikely that Petersen et al. (1994) initially conceptualized all persons with MCI as being prodromal to dementia, but rather to represent a clinician's uncertainty regarding the patient's current presentation. It is hopeful that knowing the prevalence of low memory scores will decrease clinician uncertainty.
Of course, it is possible that a small percentage of older adults in the WMS-III standardization sample, the NAB standardization sample (Brooks et al., 2007), and/or the Palmer et al. (1998) study could actually have had MCI. This continues to be a concern with normative samples in neuropsychology (e.g., Ritchie et al., 2007). However, fairly rigorous exclusion criteria were used with this sample with the goal of including only neurologically and psychiatrically healthy older adults. Thus, the base rates of low memory scores might be underestimated given the healthy status of this older adult sample (in contrast to older adults with a variety of medical and minor psychiatric problems). In addition, using the rigorous exclusion criteria when collecting the standardization data suggests that the likelihood of MCI in older adults from a standardization sample would be less than the population prevalence of MCI (i.e., likely 3% to 6%, but with substantial variance depending on sample and methods used; see Feldman & Jacova, 2005, for a review). Unfortunately, a definitive answer to this issue is not knowable with these samples, and further longitudinal research is required.
The concept that healthy adults and older adults get some low scores when a battery of neuropsychological measures are administered and interpreted is not new. For nearly two decades, Heaton and colleagues have presented data for the Halstead-Reitan Battery that illustrates this concept (Heaton et al., 1991, 2004). In addition, this concept has been briefly examined for the Wechsler battery of tests. In a study of six factor scores from the WAIS-III/WMS-III (i.e., verbal comprehension, perceptual organization, processing speed, working memory, auditory memory, and visual memory), Taylor and Heaton (2001) reported that 46% of the standardization sample obtained one or more factor scores below 1 SD. Moreover, Holdnack et al. (2006) illustrated how base rates across multiple test scores were useful in differentiating child clinical populations with mild, moderate, and severe disabilities from healthy controls. Children with moderate to severe disability showed not only a higher rate of multiple low scores but also infrequently had scores above the mean related to their disability. Future clinical studies of MCI and Alzheimer's disease may wish to consider the prevalence of both impaired and higher than the mean score on targeted cognitive functions.
Low memory scores are common in healthy older adults, increase with lower levels of intellectual functioning and/or fewer years of education and might represent normal human variability on testing, a long-standing relative weakness (without a recent change in functioning), or measurement error, broadly defined. Based on the existing research examining the base rates of low scores (Brooks et al., 2007; Crawford et al., 2007; Heaton et al., 1991, 2004; Iverson et al., 2006, 2008a, 2008b; Palmer et al., 1998), it is unlikely that this concept is specific to the WMS-III. Rather, this finding likely represents a poorly understood psychometric phenomenon across all fixed and flexible neuropsychological batteries.
The base rate data presented in this article are ready for clinical and research use. Tables 2, 3, 4, 5, 6, and 7 greatly enhance the routine use of the WMS-III and facilitate the simultaneous interpretation of multiple memory test scores with older adults in clinical and research settings. Clinicians and researchers can use these tables to determine the false-positive rates when interpreting performance across several memory measures or determining inclusion in a study. It is hopeful that this information, in conjunction with corroborative information (e.g., collateral interview and other medical investigations), will improve diagnostic accuracy with older adults. The next step is to apply these psychometric base rate data to archival or prospective samples of clinical patients to identify optimal cutoff scores for identifying memory impairment associated with neurodegenerative disease.
It is important that clinicians and researchers be well informed of the prevalence of low memory scores in healthy older adults and conceptualize performance based on intellectual abilities when interpreting scores. Misclassifying a healthy older adult as having MCI, and informing that person that 10–15% of people with MCI convert to a diagnosis of dementia each year, can potentially have seriously adverse consequences to the patient, family, healthcare system, and society in general. We believe that the base rates information provided in this study are one step toward improving the psychometric accuracy (i.e., positive and negative predictive values) when assessing for the presence of memory impairment using the WMS-III, and in return, minimize the chances of “accidental MCI.”
ACKNOWLEDGMENTS
The authors thank Jennifer Bernardo for assistance with manuscript preparation, Dr. V. Lynn Ashton for her helpful comments on an earlier draft of the manuscript, and The Psychological Corporation for use of the WMS-III standardization data. The data in Tables 2, 3, 4, 5, 6, and 7 are original data produced by special permission of the publisher, The Psychological Corporation, from the standardization data presented in the Wechsler Memory Scale–III. Further reproduction is prohibited without permission from The Psychological Corporation.
