Introduction
Access to dementia assessments is uneven in North America and across the world (Geddes et al., Reference Geddes2020). This disparity has important clinical repercussions, particularly in regions where specialized resources are limited, and identification is delayed until later manifestations (Kamoga et al., Reference Kamoga, Rukundo, Wakida, Nakidde, Obua and Buss2019). The development of unsupervised platforms that do not require highly trained administrators may resolve this dual impasse and improve clinical outcomes (Bird and Lim, Reference Bird and Lim2021). Moreover, they may create a low-cost recruitment infrastructure for early intervention trials, where no disease-modifying drug in Alzheimer’s disease (AD) has met all primary endpoints (Cummings et al., Reference Cummings, Reiber and Kumar2018; Marsden and Mestre-Ferrandiz, Reference Marsden and Mestre-Ferrandiz2015) in part due to poor recruitment of individuals without overt impairment or who are in the earliest stages of disease (Gauthier et al., Reference Gauthier2016; Mortby et al., Reference Mortby2018). Advances in online services open the possibility of assessment portals in any region with an internet connection, for any individual with access to the internet and a computing device. Moreover, cognitive tests have been computerized and can be delivered without an administrator, and with convergent validity with those administered in tertiary cognitive assessment centers (Brooker et al., Reference Brooker2020; Mackin et al., Reference Mackin2018; Nosheny et al., Reference Nosheny2020; Papp et al., Reference Papp2021; Perin et al., Reference Perin2020).
The addition of behavioral assessments to online platforms may provide additional relevant information. Neuropsychiatric symptoms (NPS) such as agitation, anxiety, apathy, depression, and psychosis are considered core features of dementia and are associated with poorer patient outcomes (Lanctôt et al., Reference Lanctôt2017). However, NPS can often precede cognitive symptoms (Shin, Reference Shin2021), including in 30% of those who develop AD (Wise et al., Reference Wise, Rosenberg, Lyketsos and Leoutsakos2019). Mild behavioral impairment (MBI) is a pre-dementia neurobehavioral syndrome characterized by the de novo emergence and persistence of NPS in older adults representing a change from long-standing patterns of behavior (Ismail et al., Reference Ismail2016). MBI is associated with amyloid, tau, neurodegeneration, and AD risk genes (Andrews et al., Reference Andrews, Ismail, Anstey and Mortby2018; Creese et al., Reference Creese, Brooker, Aarsland, Corbett, Ballard and Ismail2021b; Gill et al., Reference Gill2021; Johansson et al., Reference Johansson2021; Lussier et al., Reference Lussier2020; Matuskova et al., Reference Matuskova2021; Miao et al., Reference Miao, Chen, Gill, Naude, Smith and Ismail2021; Naude et al., Reference Naude2020; Ruthirakuhan et al., Reference Ruthirakuhan, Ismail, Herrmann, Gallagher and Lanctot2022), and a greater risk of incident cognitive decline and dementia (Creese et al., Reference Creese2019; Gill et al., Reference Gill2020; Ismail et al., Reference Ismail, McGirr, Gill, Hu, Forkert and Smith2021; Matsuoka et al., Reference Matsuoka, Ismail and Narumoto2019; Taragano et al., Reference Taragano2018; Tsunoda et al., Reference Tsunoda2021; Wolfova et al., Reference Wolfova2021). Incorporating MBI into screening may provide a complementary approach to early detection (Mortby et al., Reference Mortby2018). However, informant information is often required to validate the syndrome, and structured assessment tools suitable for widespread dissemination through unsupervised platforms have only recently been developed. The Mild Behavioral Impairment Checklist (MBI-C) incorporates informant information and is the validated case ascertainment instrument developed specifically to capture MBI in accordance with the criteria developed by the International Society to Advance Alzheimer’s Research and Treatment-Alzheimer’s Association (ISTAART-AA)(Creese et al., Reference Creese2020; Ismail et al., Reference Ismail2017; Mallo et al., Reference Mallo2019; Saari et al., Reference Saari, Smith and Ismail2021). Translated into over 20 languages, the MBI-C may also allow a broader reach for obtaining online informant reports of behavioral change.
The aim of this study was to investigate informant-based MBI in an online unsupervised platform, the Brain Health Registry (BHR), capable of assessing early dementia risk markers (Weiner et al., Reference Weiner2018). We determined the utility of the BHR for converging assessments of cognitive and behavioral symptoms using neuropsychological testing and informant-reported MBI-C. We hypothesized that participants with MBI + status would have poorer cognitive performance measured by the Lumos test battery. We further hypothesized that individuals with poorer memory, executive function, processing speed, and inhibitory control would have a higher burden of MBI symptoms.
Methods
Brain Health Registry
The BHR (Weiner et al., Reference Weiner2018) is an internet-based public registry and cohort that recruits participants using a variety of methods including a website, social media, brochures, and online advertising. All participants are required to give informed consent with an online consent form. Upon completion of the consent form, participants may complete questionnaires regarding personal and family medical history, early childhood history, sleep quality, diet, quality of life scales, psychiatric symptomatology, as well as online cognitive testing via Lumosity (Morrison et al., Reference Morrison, Simone, Ng and Hardy2015), CogState (Lim et al., Reference Lim2015), or Memtrax (Ashford et al., Reference Ashford, Gere and Bayley2011) tests. Additionally, study partners of BHR participants can register on the BHR Study Partner portal, on which informant-rated measures are completed (Nosheny et al., Reference Nosheny2018).
Study participants
Participants were included if: (1) Lumosity cognitive tests were completed; and (2) their informant completed the MBI-C via the BHR Study Partner portal within a year of the cognitive tests. Participants were excluded if they reported (1) developmental or learning disorders, (2) neurological conditions such as movement disorders, multiple sclerosis, traumatic brain injury, (3) current or past psychiatric diagnoses including schizophrenia, major mood or anxiety disorders, or PTSD.
Study variables
Lumosity online Forward Memory Span
The assessment of Forward Memory Span is based on the Corsi block-tapping tasks (Milner, Reference Milner1971). The participant is asked to recall the sequence of circles in the same order it was presented. The length of the sequence increases by one every two trials. The session comes to an end when the participant records two incorrect answers at the same span level. This task is used as a measure of visual short-term memory and attention.
Lumosity online Reverse Memory Span
The Reverse Memory Span task is a slightly altered version of the original Corsi block-tapping tasks. It is identical to the forward visual memory span assessment, with the exception that the participant is asked to recall the sequence of circles in the reverse order. This reverse task is used as a measure of visual working memory and attention.
Lumosity online Trail Making Test B
In Trail Making Test (TMT) B, blue circles (numbered 1 to 12) and capital letters (A to L) are arranged in six possible layouts with non-overlapping spatial locations. The participant must alternate between numbers and letters for this task, clicking in increasing order. When the blue circle is clicked, it turns orange, and a straight line appears to connect the circles. The timer for the task begins when the participants click the first circle. If the participant records an incorrect click, an X appears on their screen, and they are required to go back to the previous circle. For this study, we included the response time and number of errors as measures of processing speed attention and sequencing ability.
Lumosity online Go/No-Go
In the Go/No-Go assessment, participants are presented with target pictures and distractor stimuli. The target picture is chosen from a set of photos of fruit. Each stimulus appears after a random delay between 1000 and 3000 ms to discourage anticipatory responding. The participant is instructed to respond as quickly as possible within 1500 ms. The assessment ends when a participant responds to 10 “Go” trials. If the participant submits three incorrect responses (responding to “no-go” or failing to respond to “go”), the participant will restart the task. The participant is given feedback on timing and correctness. This assessment is used to measure response inhibition and speed of information processing.
Mild Behavioral Impairment Checklist
The MBI-C is included in the BHR Study Partner portal and is therefore completed by an informant. The MBI-C is explicit that symptoms are de novo in later life, represent a change from long-standing patterns of behavior, and are persistent for at least 6 months. The MBI-C consists of questions in the five MBI domains of apathy, mood and anxiety, agitation and impulsivity, impaired social cognition, and psychosis, with items geared towards capturing NPS in community-dwelling, functionally independent, non-demented older adults. The scale takes ∼7–8 min to complete, consisting of 34 questions; scoring is from 0 to 3, representing absent, mild, moderate, and severe changes, with a total score range of 0–102 (Ismail et al., Reference Ismail2017).
Statistical analysis
Continuous demographic variables (age and years of education) were analyzed using independent sample t-tests to compare MBI + and MBI- groups; sex distribution between the two groups was analyzed using chi-square tests. MBI-C was dichotomized based on a validation in primary care non-demented older adults in which scores of >7 differentiated MBI + from MBI- with a sensitivity of 0.93, specificity of 0.76, and AUC of 0.93 (Mallo et al., Reference Mallo2018b). As exploratory analyses, cutpoints of >5 and >6 were also analyzed. Univariate analysis of covariance (ANCOVA) was used to compare performance on Lumosity cognitive tests between MBI + and MBI- groups, covarying for age, sex, education, and neuropsychological and neurobehavioral assessment interval. Skewed data were log-transformed; however, the TMT response time variable was analyzed with a negative binomial regression due a skewed distribution with an overrepresentation of zeros. For Go/No-Go errors, ordinal logistic regression was performed because the response variable only had three possible values: 0, 1, and 2.
Additionally, negative binomial regressions were fitted to assess Lumosity task prediction of MBI-C total scores. Negative binomial regression is preferred when the data are skewed, as in this sample where the mode on the MBI-C is zero indicating no emergent and persistent NPS. Lumosity task measures as continuous scores were the independent variables in these models. The covariates included were age, sex, education, and neuropsychological and neurobehavioral assessment interval. The p-values for Lumosity task measures were calculated using likelihood ratio tests.
Statistical analyses were performed using SPSS v26 and R 3.6.2.
Results
Participant selection is described in Figure 1. The final sample included 499 participants with a mean age of 67 years (SD 10.4), of which 308/499 were females (61%) (Table 1). The number of MBI + participants was 31 (6.2%) (Figure 1). A significantly greater number of men were classified as MBI+ (64%, p = 0.002). MBI + participants had significantly poorer Forward Memory Span (mean sequence length of 4.68 vs. 5.26, p = 0.005; Figure 2a), poorer Reverse Memory Span (3.81 vs. 4.85, p < 0.0001; Figure 2b), more TMT errors (4.29 vs. 1.85, p = 0.01; Figure 2c), and longer TMT completion time (67.67 vs. 45.08 s, p < 0.0001; Figure 2d). MBI was not associated with the number of errors (p = 0.84) and response time (p = 0.16) on the Go/No-Go task (Figures 2e & f). The effect sizes for these differences were modest. Of the tests that significantly differed between groups, the largest effect size (Cohen’s f) was for Reverse Memory Span (0.20), followed by TMT response time (0.20), memory span (0.13), and TMT accuracy (0.11). See Tables 2a–c for statistical reporting. Analyses using cutpoints of >5 and >6 for MBI-C show very similar results and are included in supplemental tables (Supplemental Tables 1–6).
Flowchart of participants from the BHR included for analysis.
(a) Positive result (>7) on the MBI-C is associated with shorter Memory Span; (b) positive result (>7) on the MBI-C is associated with shorter Reverse Memory Span; (c) positive result (>7) on the MBI-C is associated with more errors in the Trail Making-B task; (d) positive result (>7) on the MBI-C is associated with longer response time in Trail Making-B task; (e) positive result (>7) on the MBI-C is not associated with the number of errors on a GoNoGo task; and (f) positive result (>7) on the MBI-C is not associated with the response time on a GoNoGo task.
Summary statistics for demographics
(a) Summary statistics for Lumosity tasks (Cutpoint of 7) (ANCOVA); (b) Summary statistics for using MBI-C status (Cutpoint of 7) to predict Trail Making Errors (Negative Binomial Regression); (c) Summary statistics for using MBI-C status (Cutpoint of 7) to predict Go/No-Go Errors (Ordinal Logistic Regression)
1 Beta coefficients represent the estimate percent difference in Trail Making errors associated with status
Negative binomial regressions utilizing Lumosity scores to predict MBI score determined that worse Memory Span (χ 2 (1, N = 499) = 6.6, p = 0.01), worse Reverse Memory Span χ 2 (1, N = 498) = 5.4, (p = 0.02), more TMT errors (χ 2 (1, N = 499) = 5.8, p = 0.02), and longer TMT response time (χ 2 (1, N = 499) = 9.6, p = 0.002) were all associated with higher MBI-C total scores. Go/No-Go errors (χ 2 (1, N = 497) = 0.16, p = 0.69) and Go/No-Go response time (χ 2 (1, N = 497) = 0.97, p = 0.33) were not associated with MBI-C score (Table 3).
Summary statistics for Lumosity tasks predicting MBI-C total score
1 Beta-coefficients represent the estimate percent difference in total MBI-C score given one unit change in Lumosity task measure.
Discussion
In a sample of 499 participant dyads in BHR, we demonstrated the feasibility of delivering unsupervised online assessments of behavioral and cognitive markers of dementia risk. Utilizing the validated cutoff score of >7 on the MBI-C, MBI + status was associated with significant differences in memory and executive function, measured using Forward Memory Span, Reverse Memory Span, TMT errors, and TMT speed. Further, significant associations were found between poorer objectively measured cognitive performance, in the domains of memory and executive function, and MBI symptom severity. Effect sizes were small, ranging from 0.11 to 0.20. The findings do suggest that a simple informant reported behavioral measure completed via an online portal might be a relevant addition to neuropsychological testing, warranting further study in BHR. In other work, MBI + status has demonstrated significant and meaningful associations with incident cognitive decline and dementia across several studies, settings, and populations (Creese et al., Reference Creese2019; Gill et al., Reference Gill2020; Ismail et al., Reference Ismail, McGirr, Gill, Hu, Forkert and Smith2021; Matsuoka et al., Reference Matsuoka, Ismail and Narumoto2019; Taragano et al., Reference Taragano2018; Tsunoda et al., Reference Tsunoda2021; Wolfova et al., Reference Wolfova2021). Thus, while convergent with tests of memory and executive function, behavioral and cognitive markers of risk may be distinct, potentially offering complementary measures of risk.
Our data indicate that the BHR is an effective platform to conduct remote assessments of cognitive functioning with convergence of behavioral and cognitive tests. Poorer performance on unsupervised online neuropsychological testing has been associated with self-report mild cognitive impairment (MCI) and AD (Mackin et al., Reference Mackin2018). Online participant testing is an efficient and reliable tool for neuropsychological testing, which can identify performance decrements in executive dysfunction and memory (Morrison et al., Reference Morrison, Simone, Ng and Hardy2015). Similarly, online informant reports such as those collected in the BHR Study Partner portal are valuable and informative. Online study partner-reported cognitive decline is comparable to data collected in clinic, is associated with objectively defined participant cognition (Nosheny et al., Reference Nosheny2018), and is associated with amyloid, clinical diagnosis of dementia due to AD, and in-clinic cognitive screening test scores (Nosheny et al., Reference Nosheny2020). In this study, online informant-reported behavioral symptoms associated with differences in memory and executive function collected via the participant portal. This harmonized utilization of participant and study partner portals is effective and can allow continuation of research activities even during trying times such as the recent pandemic, where consistent in person visits between clinicians and patients were not feasible. Although the COVID-19 pandemic has highlighted the need for alternative infrastructure to allow continued care, the tools that have been developed may permit the assessment of older adults who for physical, social, or cognitive reasons could not previously access care. This approach also allows outreach to areas less accessible to academic centers.
The cognitive domain differences detected with the MBI-C include memory and executive function, which are relevant and important for AD risk (Wilson et al., Reference Wilson, Leurgans, Boyle and Bennett2011). Early decline in memory and executive function has been shown to be associated with the preclinical disease process; thus, early detection of reductions in cognitive functioning may be useful in identifying populations at risk (Almkvist et al., Reference Almkvist1998; Nagata et al., Reference Nagata2011). Both memory and executive function are important endpoints in AD trials (Vellas et al., Reference Vellas, Andrieu, Sampaio, Coley and Wilcock2008). Longitudinal cohorts have demonstrated that an acceleration of decline in memory performance occurs 3–4 years before a diagnosis of MCI and 7 years before a diagnosis of AD, while for executive function an accelerated decline occurs 2-3 years before AD diagnosis (Grober et al., Reference Grober, Hall, Lipton, Zonderman, Resnick and Kawas2008; Mistridis et al., Reference Mistridis, Krumm, Monsch, Berres and Taylor2015). The finding of small but significant associations between MBI and poorer memory and executive function performance is consistent with the evolving description of the cognitive profile of MBI (Rouse et al., Reference Rouse, Small, Schinka, Loewenstein, Duara and Potter2021; Wong et al., Reference Wong, Ng, Yatawara, Low, Ismail and Kandiah2020; Yoon et al., Reference Yoon2019). These findings are consistent with a previous study using the UK online PROTECT study portal in which self-reported MBI, measured with the MBI-C, was associated with faster decline in attention and working memory at 1 year in older adults with normal cognition (Creese et al., Reference Creese2019). A subsequent analysis of cognitively normal PROTECT participants, with a median follow-up time of 3 years, demonstrated an association between baseline informant-reported MBI + status and decline in measures of working memory and fluid intelligence (Wolfova et al., Reference Wolfova2021). In an overlapping sample, AD genetic risk was determined using polygenic risk scores. AD genetic risk was associated with worse cognition in the informant-reported MBI + group but not in the MBI- group. The strongest association was in those with more severe MBI, aged ≥65 years (Creese et al., Reference Creese2021a). These convergent findings support leveraging online cognitive and behavioral measures to explore dementia risk. In a recent study of National Alzheimer Coordinating Center data, the combination of informant-reported MBI and subjective cognitive decline (SCD) had a greater risk of incident cognitive and functional decline at 3 years compared to either MBI or SCD alone (Ismail et al., Reference Ismail, McGirr, Gill, Hu, Forkert and Smith2021). Further, in a subsequent study, those with SCD and MBI together had a shorter median time to incident MCI compared to those with SCD in the absence of MBI (Nathan et al., Reference Nathan, Gill and Ismail2020). Taken together the results suggest that individuals with subtle neuropsychological test score differences and MBI together may be at higher risk for cognitive and functional decline.
As the case ascertainment instrument developed to measure MBI in accordance with the ISTAART-AA MBI criteria, the MBI-C was designed to: (1) operationalize the MBI concept; (2) measure a selected list of NPS which may help identify preclinical or prodromal dementia; and (3) predict the risk of several dementias (Ismail et al., Reference Ismail2017). This instrument has been validated in an online cohort of cognitively normal older adults (Creese et al., Reference Creese2020), a primary care sample with SCD (Mallo et al., Reference Mallo2019) or MCI (Mallo et al., Reference Mallo2018a), and a cognitive neurology clinic population with SCD and MCI (Hu et al., Reference Hu, Patten, Fick, Smith and Ismail2019; Saari et al., Reference Saari, Smith and Ismail2021).
This study is the first of its kind, investigating MBI within the BHR. The findings show associations between behavior and cognition, and further support the utility of the study partner portal. These findings will also serve as a baseline, to assess longitudinal changes in behavior and cognition. Notwithstanding the promising findings of linking behavior and cognition utilizing online study portals, there do exist barriers to this approach. Older adults may be less likely to use technology to begin with, potentially limiting the sample (Lorence and Park, Reference Lorence and Park2006). Furthermore, lack of computer knowledge, and loss of vision and fine motor skills may affect the ability of older adults to successfully access and operate technology, resulting in negative attitudes and frustration (Gatto and Tak, Reference Gatto and Tak2008; Gell et al., Reference Gell, Rosenberg, Demiris, LaCroix and Patel2015; Gitlow, Reference Gitlow2014). Socioeconomic status can also influence the ability to utilize online assessments. Limited access to computers or tablets may preclude some from participating, and those without a stable internet connection may not successfully complete online neuropsychological testing due to disruptions or low bandwidth (Darrat et al., Reference Darrat, Tam, Boulis and Williams2021). These barriers could also limit the diversity in the sample that does participate in online studies.
Further, study limitations may affect interpretation and generalizability. These limitations include high education levels and restriction to participants and study partners who can successfully complete tasks online (Nosheny et al., Reference Nosheny2018). Since the BHR is an online self-report database, the lack of a clinical diagnosis within the sample group is a potential source of error. BHR participants may have undiagnosed and/or unreported neurodegenerative disease or psychiatric disorders, which may be associated with greater MBI score. While online cognitive testing has been validated (Lim et al., Reference Lim2015; Mackin et al., Reference Mackin2018), further research is needed, given the lack of supervision or control for test environment, external factors, distractors, or cues. Further, important disease-related factors such as severity and time since symptom onset were not controlled for. Although MBI was associated with statistically significant differences in Lumosity neuropsychological test scores, effect sizes were small, and the clinical significance of these differences is difficult to interpret in the largely cognitively healthy population enrolled in BHR. These data are promising but not conclusive, and further research is required. Whether these small differences in cognitive test scores represent greater risk for incident cognitive decline and dementia can be addressed in the future using longitudinal data and a cohort that includes participants with cognitive impairment.
In summary, in this BHR study combining self- and informant-rated measures, the convergence of behavioral risk markers for dementia and cognitive differences was observed, reflected by neuropsychological tests incorporating memory and executive function. The findings lend additional support to online unsupervised administration of cognitive and neuropsychiatric measures, as a low-cost approach to improve access to neurocognitive assessments, potentially identifying at-risk older adults.
Conflict of interest
Data used in this study were collected using the BHR, which is funded by the NIH, Alzheimer’s Association, Alzheimer’s Drug Discovery Foundation, California Department of Public Health, Connie and Kevin Shanahan, The Drew Foundation, General Electric, Global Alzheimer’s Platform Foundation, Larry L. Hillbolm Foundation, The Ray and Dagmar Dolby Family Fund, The Rosenberg Alzheimer’s Project and Patient-Centered Outcomes Research Institute.
Source of funding
Z. Ismail is funded by the Canadian Institutes of Health Research, and has received consulting fees/honoraria from Otsuka/Lundbeck, outside the submitted work. His institution has received funds from Acadia, Biogen, Roche, and Sunovion, also outside the submitted work. R. Nosheny is a co-investigator for the BHR. R. S. Mackin has received grant funding from the National Institute of Mental Health and has received research support from Johnson & Johnson. M. Weiner receives support for his work from the following: National Institute of Health, Department of Defense, Patient-Centered Outcomes Research Institute, California Department of Public Health, University of Michigan, Siemens, Biogen, Larry L. Hillbolm Foundation, Alzheimer’s Association, The State of California, Johnson & Johnson, Kevin and Connie Shanahan, GE, Vrije Universiteit Medical Center Amsterdam, Australian Catholic University, The Stroke Foundation and the Veterans Administration. He has served on the Advisory Boards for Eli Lilly, Cerecin/Accera, Roche, Alzheon, Inc., and BHR.
Description of authorsʼ roles
H. Chen, Z. Ismail, A. McGirr, and R. Nosheny designed the study objectives. M. Camacho, R. S. Mackin, R. Nosheny, M. Weiner, and T. Williams supervised the data collection process. H. Chen and Z. Ismail contributed to the statistical design of the study, and H. Chen performed the statistical analysis. H. Chen, Z. Ismail, F. Kassam, and A. McGirr wrote the manuscript. Z. Ismail, A. McGirr, and M. Weiner advised methodology for the data analysis. All authors contributed to the interpretation of data and revising the manuscript.
Supplementary material
For supplementary material accompanying this paper visit https://doi.org/10.1017/S1041610221002878
Open access
