Introduction
Self-ratings and informant-ratings are commonly used to assess cognitive and functional change in individuals with Mild Cognitive Impairment (MCI). While these ratings are assumed to be largely accurate, research shows they are subject to biases. An improved understanding of factors that contribute to biases in self- and informant-ratings of functional abilities in MCI is crucial, not only because these ratings are used to distinguish between the MCI and dementia stages, but also because these ratings are commonly used as outcome measures in clinical trials.
Caregiver ratings are often considered to be more accurate than self-ratings in individuals with MCI/mild dementia given the potential for bias due to personal factors as well as features of their disease course. One such factor is the potential for anosognosia, or poor insight into one’s own functioning due to cognitive impairment (Martyr & Clare, Reference Martyr and Clare2018) which can lead individuals to overestimate their performance. A person’s self-efficacy, or belief in one’s own capacity to execute behaviors necessary to produce specific outcomes (Bandura, Reference Bandura1977), may also impact self-ratings regarding functional abilities. Self-efficacy reflects confidence in the ability to exert control over one’s own motivation, behavior, and social environment, and has not, to our knowledge, been investigated as a potential source of bias. It is possible that individuals with low self-efficacy will rate themselves as having lower functional abilities. When there is no anosognosia, the presence of having cognitive impairment and receiving a diagnosis of MCI or Alzheimer’s diseases (AD) has been shown to be associated with higher stress, greater depression, and worse ratings of functional abilities (Stites et al., Reference Stites, Karlawish, Harkins, Rubright and Wolk2017). Higher depression and stress are associated with worse self-ratings in individuals with dementia (Martyr et al., Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022). This is unsurprising, as negative self-judgements are considered one of the core components of depression (Beck, Reference Beck1987). A last factor that has been associated with bias in self-ratings by individuals with dementia is age. Ratings between caregivers and individuals with dementia are more consistent (suggesting lower levels of bias) for younger individuals with dementia (Martyr et al., Reference Martyr, Clare, Nelis, Marková, Roth, Woods, Whitaker and Morris2012, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022).
The above mentioned studies highlight biases in self-ratings in individuals with dementia, yet informant-ratings are subject to biases as well. A recent study suggests that individuals with mild-to-moderate dementia more accurately appraised their functional abilities than their informants, with informants generally underestimating their overall abilities (Martyr & Clare, Reference Martyr and Clare2018). Caregiver distress and caregiver burden have also been found to contribute to biases, with higher discrepancies between caregiver ratings and self ratings of functional abilities and worse ratings by caregivers who are more burdened or stressed (Martyr et al., Reference Martyr, Nelis and Clare2014, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022). The directionality of this relationship has remained unclear thus far: it is likely that being a caregiver of those with worse functional abilities leads to higher stress and burden, while caregivers who are more stressed might also become more critical in their ratings of the functional abilities in the person for whom they are caring.
The approach of comparing self-ratings in individuals with MCI/dementia to the ratings of informants, without comparison to an objective measure of performance, does not allow clinicians to determine which ratings are subject to bias. Using an independent measure of functional abilities or cognition can help in teasing apart the bias that might be introduced by individuals with MCI/dementia versus the bias that might be introduced by informants. Only one study thus far has compared subjective ratings to more objective methods to assess functioning in MCI (Hackett et al, Reference Hackett, Mis, Drabick and Giovannetti2020). This study also assessed predictors of subjective ratings on functional abilities of individuals with MCI while controlling for objective cognitive performance, and found that functional abilities were rated as more impaired by paid caregivers, spouses, and adult children than by friends/neighbors or other relatives. Informants who cohabitated with the person with MCI or had higher levels of education also rated the person with MCI as more impaired (Hackett et al., Reference Hackett, Mis, Drabick and Giovannetti2020). While this study has the methodological strength of assessing sources of bias in ratings compared to objective cognitive performance, it did not assess other sources of bias such as stress, burden, and depressive symptoms.
Thus, a limitation on investigations of bias in ratings of functional abilities is that most have utilized methods of comparing self-ratings to informant-ratings without a direct comparison to objective performance (Martyr et al., Reference Martyr, Clare, Nelis, Marková, Roth, Woods, Whitaker and Morris2012, Reference Martyr, Nelis and Clare2014, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022). Both types of ratings are subjective, and as a result, it can be difficult to determine whose ratings are more accurate due to the potential sources of bias and their direction. Studies thus far have not simultaneously examined the discrepancy between reported functional status and objective cognitive performance in both individuals with MCI and informants to help in teasing apart the bias that might be introduced by individuals with MCI versus the bias that might be introduced by informants. The overall aim of the current study was to extend earlier findings by comparing subjective self- and informant-ratings to objective cognitive performance to determine (1) the extent of bias in both types of subjective ratings and (2) to evaluate different determinants of bias for individuals with MCI vs. informants, including demographic predictors, stress, burden, and depressive symptoms as potential sources of bias. We further extended the literature by including self-efficacy as a predictor of discrepancy given that the belief in one’s own capacity to execute behaviors necessary to produce specific outcomes in MCI may also impact their functional ability ratings.
We hypothesized that using an objective approach, our study would replicate earlier findings based on a subjective approach that underestimation of the individual with MCI’s functional abilities occur in ratings by individuals with MCI and informants who are younger, more highly educated, experience higher stress and who report higher depressive symptoms (Hackett et al., Reference Hackett, Mis, Drabick and Giovannetti2020; Martyr et al, Reference Martyr, Clare, Nelis, Marková, Roth, Woods, Whitaker and Morris2012, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022). We also hypothesized that underestimation of functional abilities relative to objective cognitive performance would also occur in (1) ratings by individuals with MCI with lower self-efficacy and (2) ratings by informants with higher burden (Martyr et al., Reference Martyr, Nelis and Clare2014, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022).
Method
Study design
This study included cross-sectional data from individuals with MCI and their study partners who served as informants.
Participants and setting
A total of 118 individuals with MCI and their informants were enrolled in the Charlie and Harriet Shaffer Cognitive Empowerment Program (CEP), a collaboration between the School of Interactive Computing and the School of Architecture at Georgia Institute of Technology and the Neurology Department of Emory University School of Medicine. The CEP is a 12-month comprehensive lifestyle program that addresses modifiable risk factors associated with progression of cognitive decline.
CEP participants are referred from Emory’s Cognitive Neurology Clinic after receiving a clinical diagnosis of MCI. Diagnostic criteria used in Emory’s Cognitive Neurology Clinic include a subjective concern by the person with MCI, an informant, or a clinician regarding a change in cognition compared to a previous level of cognitive performance, a clinician-rated judgement of cognitive impairment based on cognitive testing, and relatively preserved independence in instrumental activities of daily living (IADLs; Albert et al., Reference Albert, DeKosky, Dickson, Dubois, Feldman, Fox, Gamst, Holtzman, Jagust, Petersen, Snyder, Carrillo, Thies and Phelps2011). Clinicians generally relied on multiple sources of data including a clinical interview with the person with MCI and one or more family members. Clinicians typically also used the Functional Activity Questionnaire (Pfeffer et al., Reference Pfeffer, Kurosaki, Harrah, Chance and Filos1982) or the Activities of Daily Living questionnaire (Lawton & Brody, Reference Lawton and Brody1969) as one of the sources of information for their diagnoses. Other inclusion criteria for the CEP are (1) stamina for enrollment in a program that includes physical exercise, (2) having a study partner who is able and willing to attend assessment and goal setting sessions, (3) English proficiency, and (4) being medically stable and independent in toileting. Individuals with MCI due to systemic illness, substance abuse or psychiatric disease are excluded. Over 90% of CEP participants have MCI due to AD, confirmed via clinical cerebrospinal fluid (CSF) testing using the Athena CSF ADMARK® test which measures the biomarkers A-beta 42 peptide (A-beta 42), phospho-tau (P-tau) and total tau (T-tau) and provides an algorithmic interpretation of results based on cutoff values (Blennow et al., Reference Blennow, Dubois, Fagan, Lewczuk, De Leon and Hampel2015).
The CEP protocol is approved by the Emory Institutional Review Board, and all participants and study partners provide informed consent allowing researchers to access their medical records. The research is completed in accordance with the Helsinki Declaration.
Procedures
At the time of their enrollment and prior to the start of CEP programing, participants received the measures described below.
Measures
The Montreal Cognitive Assessment
The Montreal Cognitive Assessment (MoCA) was used to assess global cognitive functioning in all participants. The MoCA is a cognitive screening tool that assesses several domains including visuospatial abilities, executive functioning, language, memory, and attention (Nasreddine et al., Reference Nasreddine, Phillips, Bedirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005). The maximum score is 30 points, with higher scores indicating better performance.
Everyday Cognition
Individuals with MCI and informants completed the Everyday Cognition questionnaire (ECog; Farias et al., Reference Farias, Mungas, Reed, Cahn-Weiner, Jagust, Baynes and DeCarli2008). The ECog is a brief questionnaire that assesses cognitively relevant functional abilities in the following areas: divided attention, visuospatial abilities, language, memory, planning and organization. A higher score reflects worse functioning (range = 39–156).
Dependent variables
Three discrepancy variables were generated to evaluate bias in subjective ratings, which allows for interpretation of the magnitude and directionality (i.e., which score is worse) and facilitates the interpretation of the findings. We generated z-score corrections to provide a consistent scale for all measures. Z-scores were calculated by dividing the sample mean by the sample standard deviation for each measure, which was done separately for the sample of individuals with MCI and the sample of informants. The ECogI-ECogS characterized the discrepancy between informant-rated and self-rated functional status. The ECOGI-MoCA variable characterized the discrepancy between informant-rated functional status and objective cognitive performance of the individual with MCI. The ECogS-MoCA variable characterized the discrepancy between self-rated functional status and objective cognitive performance in individuals with MCI. Supplemental Table 1 includes further description of how each discrepancy variable was calculated. For all discrepancy variables, negative values indicate worse informant-ratings/self-ratings than MoCA scores, whereas positive values indicate better informant-ratings/self-ratings than MoCA scores (which will be referred to as relative underestimation or overestimation of performance, respectively). For the ECogI-ECogS negative values indicate worse informant-ratings than self-ratings.
Independent variables
Perceived Stress Scale
Both individuals with MCI and informants were asked to complete the 10-item Perceived Stress Scale (PSS). The PSS is a commonly used questionnaire to assess perceived stress and the degree to which situations in one’s life are appraised as stressful. (Cohen et al., Reference Cohen, Kamarck and Mermelstein1994). It ranges from 10 to 50 points, and higher scores reflect higher levels of stress.
Center for Epidemiologic Studies Depression scale
Both individuals with MCI and informants were asked to complete the Center for Epidemiologic Studies Depression Scale (CES-D). The CES-D is a well validated 20-item self-report measure of depressive symptoms with a four-factor structure consisting of negative affect, lack of positive affect, somatic symptoms, and interpersonal difficulties subscales (Radloff, Reference Radloff1977). It ranges from 20 to 80 points, and higher scores reflect higher levels of depression.
Zarit Burden Interview-12
Informants were asked to complete the Zarit Burden Interview – 12 Item (ZBI), a widely used instrument for assessing the burden experienced by the caregivers of individuals with dementia (Zarit et al., Reference Zarit, Orr and Zarit1985). It ranges from 0 to 48 points, and higher scores reflect higher levels of burden.
Self-efficacy
Individuals with MCI were asked to complete the NIH Self-Efficacy Scale (NIH Self-Eff). The NIH Self-Efficacy scale is a reliable measure of one’s perceived ability to control meaningful life events (Salsman et al., Reference Salsman, Butt, Pilkonis, Cyranowski, Zill, Hendrie, Kupst, Kelly, Bode, Choi, Lai, Griffith, Stoney, Brouwers, Knox and Cella2013). It ranges from 0 to 40 points, and higher scores reflect higher levels of self-efficacy.
Statistical analyses
Statistical analyses were conducted using SPSS 22.0 software (Armonk, 2007). Prior to statistical analysis, all data were screened for outliers, identified as z > ± 3 standard deviations above or below the mean and removed from analyses, which occurred for a maximum of two scores per measure. Dependent variables and the residuals of the analyses were normally distributed as assessed by Kolmogorov–Smirnov analyses (see Supplemental Table 2).
Bivariate Pearson correlations were conducted to evaluate the relationship between the measures, and to identify variables of interest for subsequent analyses. Linear regression models were applied using forward selection to evaluate these variables’ ability to predict the discrepancy variables. Separate models were utilized for different variables of interest, resulting in a total of three models, one for each of the three discrepancy variables that served as dependent variables.
Results
Demographic information on the participants can be found in Table 1.
Table 1. Sample demographics
Correlations
Results of the correlation analyses are available in Tables 2 and 3. No significant correlations were found between age and education of the individual with MCI or the informant and the discrepancy variables. Therefore, age and education of the individual with MCI and the informant were not included the tables.
Table 2. Correlations between discrepancy variables and variables of individuals with MCI
CES-D = Center for Epidemiologic Studies Depression Scale, ECog = Everyday Cognition questionnaire, MoCA = Montreal Cognitive Assessment, PSS = Perceived Stress Scale, Self-Eff = NIH Self-Efficacy Scale.
*indicates significance at p < .01
Table 3. Correlations between discrepancy variables and variables of the informant
CES-D = Center for Epidemiologic Studies Depression Scale, ECog = Everyday Cognition questionnaire, MoCA = Montreal Cognitive, PSS = Perceived Stress Scale, ZBI = Zarit Burden Interview-12.
*indicates significance at p < .01.
Linear regression analyses
A linear regression model was completed for each outcome variable (ECogI-ECogS, ECogI-MoCA, ECogS-MoCA) using the variables that were significantly correlated with each discrepancy variable. In each model, the forward selection method was used, and all predictors were entered simultaneously.
Self- and informant-rating discrepancy
For the ECogI-ECogS discrepancy, the final model included informant stress (PSS; β = .43, t(67) = 4.23, p < 0.001), and self-efficacy in the individual with MCI (Self-Eff; β = 0.35, t(67) = 3.48, p < 0.001). Worse ratings by the informant relative to self-ratings were found for individuals with MCI who had higher self-efficacy and for informants with higher perceived stress. The final model explained a significant proportion of variance in the ECogI-ECogS discrepancy (R 2 = 0.33, F(2, 66) = 16.06, p < 0.001).
Bias in informant-rating compared to objective cognition
The only significant predictor of the ECogI-MoCA discrepancy was informant stress (PSS: β = −.48, t(41) = −3.49, p = .001). Worse informant-ratings of functional status relative to objective cognition were found in informants with higher stress. In this equation, stress also explained a significant proportion of variance in the ECogI-MoCA discrepancy (R 2 = 0.23, F(1, 41) = 12.15, p = 0.001).
Bias in self-rating compared to objective cognition
For the ECogs-MoCA discrepancy, self-efficacy was the only significant predictor (β = .34, t(81) = 0.34, p = 0.002). Worse self-ratings of functional status relative to objective cognition were found in individuals with MCI with worse self-efficacy. Self-efficacy in the individuals with MCI explained a significant proportion of variance in the ECogS MoCA discrepancy (R 2 = .11, F(1, 81) = 10.32, p = 0.002).
Controlling for informant cognition
To ensure that cognition of the informant was not impacting the results, all three models were rerun while controlling for informant MoCA. The informant MoCA was not a significant predictor of the ECogI-ECogS discrepancy (β = 0.066, t(65) = 0.621, p = 0.537), ECogI-MoCA discrepancy (β = −0.03. t(90) = −0.29, p = 0.770), or the ECogS-MoCA discrepancy (β = -0.03, t(67) = −0.29, p = 0.777).
Discussion
Prior studies addressing biases in functional ability ratings by individuals with MCI and informants have done so by assessing the discrepancy between subjective self- and informant-ratings. The current study aimed to replicate and extend earlier findings using a more objective approach and to evaluate determinants of bias for individuals with MCI vs. informants including stress, burden, depressive symptoms, and self-efficacy.
Extending prior findings, an assessment of the discrepancy between self- and informant-ratings of functional ability revealed worse ratings by individuals with MCI who reported lower self-efficacy, and by caregivers with higher perceived stress. These findings were in line with hypotheses, and with prior findings that caregiver distress is associated with a higher discrepancy between self-and informant-ratings (Mangone et al., Reference Mangone, Sanguinetti, Baumann, Gonzalez, Pereyra, Bozzola, Gorelick and Sica1993; Martyr et al., Reference Martyr, Nelis and Clare2014, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022).
Using a novel, more objective approach to assess bias in subjective ratings of functional status, our results confirmed that for informants, higher stress was associated with worse ratings of functional abilities in individuals with MCI relative to objective cognitive performance. These results are consistent with results using the more subjective approach described above and are also in line with prior studies that found that caregiver distress is associated with a higher discrepancy between self- and informant-ratings (Mangone et al., Reference Mangone, Sanguinetti, Baumann, Gonzalez, Pereyra, Bozzola, Gorelick and Sica1993; Martyr et al., Reference Martyr, Nelis and Clare2014, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022). Importantly, prior studies were unable to disentangle whether informants were more stressed because of objectively lower functional abilities in the individuals with dementia, or if informants who were more stressed (for unrelated reasons) tended to rate functioning of the individuals with MCI more negatively, thereby introducing bias. By using a more objective approach that controlled for the overall cognition of the individuals with MCI, the current study confirms that higher stress in informants was associated with informants reporting worse functional abilities relative to objective cognition in individuals with MCI.
Our results also showed that lower self-efficacy in individuals with MCI was associated with worse self-ratings of functional activities relative to objective cognitive functioning, and, vice versa, that higher self-efficacy in individuals with MCI was associated with better self-ratings of functional activities relative to objective cognitive functioning. These findings suggest that having higher self-efficacy (or confidence in the ability to exert control over motivation, behavior, and social environment) predicted the degree of relative overestimation of one’s current level of functioning compared to objective performance. It is possible that a lack of insight or anosognosia impacted these ratings, and future studies should aim to differentiate self-efficacy from a lack of insight.
An important consideration in our study is that it is possible that our measure of objective functioning may have been susceptible to the same factors that impact subjective ratings of function. For instance, prior literature has shown that depression and stress are associated with cognitive deficits (Dotson et al., Reference Dotson, McClintock, Verhaeghen, Kim, Draheim, Syzmkowicz, Gradone, Bogoian and Wit2020; Marin et al., Reference Marin, Lord, Andrews, Juster, Sindi, Arsenault-Lapierre, Fiocco and Lupien2011). The literature on the link between self-efficacy and cognition is less consistent (Beaudoin & Desrichard, Reference Beaudoin and Desrichard2011) but it is possible that low self-efficacy too lowers cognitive performance. While this is an important consideration, a potential bias in objective cognitive performance due to depression, stress, and low self-efficacy would be expected to be in the same direction as the potential bias in subjective ratings that could be introduced in individuals with high depression, stress, and low self-efficacy. As such, a potential bias in MoCA scores due to depression, stress, or low self-efficacy would expect to lower rather than inflate our discrepancy scores and statistical power, suggesting that the actual level of bias introduced by these constructs may be higher.
Another important consideration is that the ECog assesses functional abilities that are cognitively relevant in several different cognitive domains including divided attention, visuospatial abilities, language, memory, planning and organization. The ECog may therefore be more consistent with the MoCA (resulting in smaller discrepancies) than other questionnaires related to functional activities.
Our study did not confirm earlier findings that relative underestimation of the individual with MCI’s functional abilities would occur in ratings by informants who are younger, more highly educated, report higher depressive symptoms, or who report greater caregiver burden (Hackett et al., Reference Hackett, Mis, Drabick and Giovannetti2020; Martyr et al, Reference Martyr, Clare, Nelis, Marková, Roth, Woods, Whitaker and Morris2012, Reference Martyr, Gamble, Nelis, Collins, Alexander, Morris, Quinn, Pentecost, Rusted, Victor, Thom, Matthews and Clare2022). Given that we used a forward selection method in the regression analyses, which selects variables that maximize explained variance, it is possible that stress in informants was selected over caregiver burden and depressive symptoms given that it is a broader, overarching concept, which can encompass aspects of caregiver burden and depressive symptoms. Correlation analyses, which showed that caregiver burden and depressive symptoms both correlated with perceived stress, also support this notion.
Limitations
A limitation of the current study is that no objective measure of functional abilities was obtained. Performance-based assessments of functional abilities, such as by the Independent Living Scales (Loeb, Reference Loeb1996) can provide more direct assessments of functional abilities in MCI and insight into whether individuals with MCI or their informants are more accurate in their ratings. However, even performance-based assessments of functional abilities are imperfect, as individuals’ actual daily activities and tasks vary greatly. Any objective task may fail to capture the specific tasks that an individual with MCI may struggle with most at home. Research has shown that cognitive abilities are strongly associated with functional activities, both when functional activities are measured with a questionnaire (e.g., Still et al., Reference Still, Pajewski, Chelune, Rapp, Sink, Wadley, Williamson and Lerner2019) and when using an objective measure of functional abilities (Duff et al., Reference Duff, Porter, Dixon, Suhrie and Hammers2020). Thus, using our approach of simultaneously examining the discrepancy between reported functional status and objective cognitive performance in both individuals with MCI and informants helps clarify that bias is introduced by both groups.
Another limitation is that it is possible that the clinicians who diagnosed MCI in our participants relied on the same family member for their input regarding functional abilities as those who provided ratings for this study, which ultimately means that the biases found in this manuscript may have also played a role in their diagnosis of MCI for a portion of our sample. In addition, providers in the Cognitive Neurology Clinic made their diagnoses of MCI based on clinical judgment rather than applying uniform research cutoffs for cognitive tests such as standard scores of –1.0 or –1.5 across the measures. As such, this approach likely led to additional variability in our MCI sample. Future studies should attempt to replicate the current findings in a more rigorous research diagnostic sample. As described above, our methods did not allow us to control for lack of insight, and it is therefore possible that a lack of insight contributed to the discrepancies between self-ratings and informant-ratings or objective cognitive functioning. A strength of the current study, however, is the inclusion of an objective measure of cognitive performance, which allows us to begin to parse out the effects of bias in both self- and informant-ratings compared to an objective benchmark. A last limitation is that the current sample consisted primarily of individuals with MCI due to Alzheimer’s disease and primarily included white, non-Hispanic individuals with a relatively high level of education.
Generalizability
The current sample consisted primarily of individuals with MCI due to Alzheimer’s disease. It is unclear if the results generalize to individuals with MCI due to other etiologies and to individuals of lower levels of education or more diverse backgrounds. In addition, the current sample was largely non-Hispanic, White, and relatively well educated. It is unclear if results generalize to individuals from other communities, including individuals with a different race and ethnicity, and individuals with lower levels of education. Future studies should aim to conduct a comparison of subjective ratings and objective performance in an ethnically and demographically diverse sample to elucidate differences which may exist in different sociocultural groups.
Interpretation and conclusions
An improved understanding of sources of bias in self- and informant-ratings of functional abilities in MCI is crucial, not only because these ratings are used to distinguish between the MCI and dementia phase, but also because these ratings are commonly used as outcome measures in clinical trials. In our sample of individuals with MCI primarily due to Alzheimer’s disease, informants with higher stress as well as individuals with MCI endorsing lower self-efficacy were more likely to underestimate functional abilities relative to objective cognitive functioning. These findings highlight the potential for biases in both self- and informant-ratings of functional abilities in MCI.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355617723011463.
Funding statement
This research was made possible by funding from the Charlie and Harriet Shaffer Cognitive Empowerment Program and the James M. Cox Foundation.
Competing interests
None.
Open access
