Validated computerized assessments for cognitive functioning are crucial for older individuals and those at risk of cognitive decline. The National Institutes of Health (NIH) Toolbox Cognition Battery (NIHTB-CB) exhibits good construct validity but requires validation in diverse populations and for adults aged 85+. This study uses data from the Assessing Reliable Measurement in Alzheimer’s Disease and cognitive Aging study to explore differences in the factor structure of the NIHTB-CB for adults 85 and older, Black participants versus White participants, and those diagnosed as amnestic Mild Cognitive Impairment (aMCI) vs cognitively normal (CN).

Subtests from the NACC UDS-3 and NIHTB-CB were administered to 503 community-dwelling Black and White adults ages 55–99 (367 CN; 136 aMCI). Confirmatory factor analyses were used to investigate the original factor structure of NIHTB-CB that forms the basis for NIHTB-CD Index factor scores.

Factor analyses for all participants and some participant subsets (aMCI, White, 85+) substantiated the two anticipated factors (Fluid and Crystallized). However, while Black aMCI participants had the expected two-factor structure, for Black CN participants, the List Sorting Working Memory and Picture Sequence tests loaded on the Crystallized factor.

Findings provide psychometric support for the NIHTB-CB. Differences in factor structure between Black CN individuals and Black aMCI individuals suggest potential instability across levels of cognitive impairment. Future research should explore changes in NIHTB-CB across diagnoses in different populations.

The ability to remotely monitor cognitive skills is increasing with the ubiquity of smartphones. The Mobile Toolbox (MTB) is a new measurement system that includes measures assessing Executive Functioning (EF) and Processing Speed (PS): Arrow Matching, Shape-Color Sorting, and Number-Symbol Match. The purpose of this study was to assess their psychometric properties.

MTB measures were developed for smartphone administration based on constructs measured in the NIH Toolbox® (NIHTB). Psychometric properties of the resulting measures were evaluated in three studies with participants ages 18 to 90. In Study 1 (N = 92), participants completed MTB measures in the lab and were administered both equivalent NIH TB measures and other external measures of similar cognitive constructs. In Study 2 (N = 1,021), participants completed the equivalent NIHTB measures in the lab and then took the MTB measures on their own, remotely. In Study 3 (N = 168), participants completed MTB measures twice remotely, two weeks apart.

All three measures exhibited very high internal consistency and strong test-retest reliability, as well as moderately high correlations with comparable NIHTB tests and moderate correlations with external measures of similar constructs. Phone operating system (iOS vs. Android) had a significant impact on performance for Arrow Matching and Shape-Color Sorting, but no impact on either validity or reliability.

Results support the reliability and convergent validity of MTB EF and PS measures for use across the adult lifespan in remote, self-administered designs.

Symptom clustering research provides a unique opportunity for understanding complex medical conditions. The objective of this study was to apply a variable-centered analytic approach to understand how symptoms may cluster together, within and across domains of functioning in mild cognitive impairment (MCI) and dementia, to better understand these conditions and potential etiological, prevention, and intervention considerations.

Cognitive, motor, sensory, emotional, and social measures from the NIH Toolbox were analyzed using exploratory factor analysis (EFA) from a dataset of 165 individuals with a research diagnosis of either amnestic MCI or dementia of the Alzheimer’s type.

The six-factor EFA solution described here primarily replicated the intended structure of the NIH Toolbox with a few deviations, notably sensory and motor scores loading onto factors with measures of cognition, emotional, and social health. These findings suggest the presence of cross-domain symptom clusters in these populations. In particular, negative affect, stress, loneliness, and pain formed one unique symptom cluster that bridged the NIH Toolbox domains of physical, social, and emotional health. Olfaction and dexterity formed a second unique cluster with measures of executive functioning, working memory, episodic memory, and processing speed. A third novel cluster was detected for mobility, strength, and vision, which was considered to reflect a physical functioning factor. Somewhat unexpectedly, the hearing test included did not load strongly onto any factor.

This research presents a preliminary effort to detect symptom clusters in amnestic MCI and dementia using an existing dataset of outcome measures from the NIH Toolbox.

As the older adult population increases in the coming decades, the number of persons that develop dementia of the Alzheimer’s type (DAT) will increase accordingly. Though curative treatment for Alzheimer’s disease remains elusive, early detection of cognitive decline allows for initiation of pharmacological treatment to slow disease progression and non-pharmacological approaches to support quality of life and well-being of affected individuals and their care partners. Streamlined approaches that bridge the gap between brief screenings and comprehensive neuropsychological evaluation are needed. The NIH Toolbox Cognition Battery (NIHTB-CB) is a brief, easily administered, computerized cognitive battery that assesses various aspects of both fluid and crystallized cognitive abilities. ARMADA (Advancing Reliable Measurement in Alzheimer’s Disease and Cognitive Aging) is a multi-site study that aims to validate the NIHTB across the spectrum from normal aging to DAT. The current study utilized longitudinal data from ARMADA to determine whether performance on the NIHTB-CB detects cognitive decline in persons with normal cognition (NC), mild cognitive impairment (MCI), and mild DAT over the span of two years. We predicted that scores would decline for the MCI and DAT groups, but not for the NC group.

Participants were 191 participants drawn from the larger ARMADA cohort aged 65-84 (nNC = 118, nMCI = 47, nDAT = 26) that completed the NIHTB-CB at baseline and 12 months. The clinical groups were significantly older than the NC group at baseline (MNC = 72.72, MMCI = 76.63, MDAT = 75.42; p < .001) and the NC and MCI groups had significantly more years of education than the DAT group (MNC = 17.03, MMCI = 16.83, MDAT = 15.54; p = .008).

Mixed model ANOVAs determined differences in uncorrected NIHTB-CB scores between clinical groups at baseline and 12 months, controlling for age and education. There were significant interactions between time and clinical group for Flanker (p < .001), Pattern Comparison (p < .001), and Picture Vocabulary (p = .001), such that the DAT group demonstrated a more negative slope of change than the NC and MCI groups. For Oral Reading, the MCI group demonstrated a more negative slope of change than the NC and DAT groups (p = .01).

Differential score trajectories were found for the Flanker task, with a more negative pattern of change in scores in the DAT group compared to the NC and MCI groups. Contrary to expectation, scores decreased for the two crystallized subtests across groups, which may reflect regression to the mean given high baseline scores, especially for Picture Vocabulary; however, these results were also moderated by group with less decline in scores in the NC group, which may indicate involvement of non-crystallized abilities in executing a single word comprehension task. Group differences were subtle, which may in part reflect the relatively short period of follow up. The Flanker task appears to be most sensitive to decline in mild DAT compared to MCI and NC. Results provide preliminary support for the utility of NIHTB-CB in detecting cognitive decline along the cognitive aging to DAT spectrum.

To present the Mobile Toolbox (MTB), comprised of an expandable library of cognitive and other tests, including adapted versions of NIH Toolbox® measures. The MTB provides a complete research platform for app creation, study management, data collection, and data management. We will describe the MTB project and MTB research platform and demonstrate examples of assessments.

MTB is the product of an NIH-funded, multi-institutional effort involving Northwestern University, Sage Bionetworks, Penn State, University of California San Francisco, University of California San Diego, Emory University, and Washington University. The MTB assessment library is a dynamic repository built upon Sage Bionetworks mobile health platform. All MTB measures are created or adapted for a mobile interface using iOS and Android smartphones. Guided by the principles of open science, many components are open source to allow researchers and developers to integrate externally developed tests, including supplemental scales (e.g., passively collected contextual factors) assessing variables such as mood and fatigue that might influence cognitive test performance.

The current MTB library includes eight core cognitive tests based on well-established neuropsychological measures: two language tasks (Spelling and Word Meaning), two executive functioning tasks (Arrow Matching and Shape-Color Sorting), an associative memory task (Faces and Names), an episodic memory task (Arranging Pictures), a working memory task (Sequences) and a processing speed task (Numbers and Symbols). Additional cognitive assessments from other popular test libraries including the International Cognitive Ability Resource (ICAR), Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRACS) and Test My Brain are currently being implemented, as are non-cognitive measures from the NIH Toolbox Emotion Battery and the Patient-Reported Outcomes Measurement Information System (PROMIS). The MTB library includes measures suitable for use in research studies incorporating point-in-time and burst designs as well as ecological momentary assessment (EMA).

The MTB was created to address many of the scientific, practical, and technical challenges to cognitive assessment by capitalizing on advances in technology measurement and cognitive research. Initial psychometric evaluation of measures has been performed, and additional clinical validation is underway in studies with persons at risk for cognitive impairment or Alzheimer’s disease (AD), diagnosed with mild cognitive impairment (MCI) or AD, Parkinson’s disease, and HIV-associated Neurocognitive Disorders. Calculation of norms and reliable change indicators is in progress. The MTB is currently available to beta testers with public release planned for Summer, 2023. Clinical researchers will be able to use the MTB system to design smartphone-based test batteries, deploy and manage mobile data collection in their research studies, and aggregate and analyze results in the context of large-scale norming data.

To present validation evidence for the first eight cognitive measures available through Mobile Toolbox (MTB). These measures use a remote self-administered platform to assess language, working memory, episodic memory, executive function, and processing speed.

We used two separate samples, recruited as part of a larger study, to validate MTB measures. Sample I, comprised of 92 English-speaking adults ages 18-85, was used to assess internal consistency and construct validity. Participants were first administered “gold standard” cognitive measures (Wechsler Memory Scale-IV Verbal Paired Associates I and II; Wechsler Adult Intelligence Scale-IV Symbol Search, Digit Span, Coding, and Letter-Number Sequencing; Delis-Kaplan Executive Function System Color-Word Interference Test, Peabody Picture Vocabulary Test, Wechsler Individual Achievement Test-4 Spelling, and the Wisconsin Card Sorting Test), after which they completed MTB (pre-loaded on a study-provided smartphone) on their own. Internal consistency was evaluated using measure-appropriate indices (split-half reliability, Cronbach’s alpha or IRT-based indices). Pearson correlation coefficients between MTB tests and measures of similar constructs were used to evaluate concurrent validity. For two tests with timing-dependent scores, Arrow Matching and Shape-Color Sorting, separate analyses were performed for iOS and Android devices. Sample II, with 1,120 English-speaking participants ages 18-90, was used to evaluate age-related change. Participants completed MTB measures remotely on their own smartphones, in a preset order, within a 14-day period. Spearman correlation coefficients, corrected for education, were calculated to evaluate relationships between age and test scores.

Sample I participants were 67% female, 52% white, 99% non-Hispanic; average age=48 (SD= 17). Education was: < high school (1%); high school (55%); some college (21%); college (15%); graduate degree (8%). Internal consistency estimates ranged from 0.81 to 0.99. Pearson correlations between MTB and external measures ranged from 0.41 to 0.86 (all p < .01). Of the timed tests, only Shape-Color sorting showed significant score differences between Android and iOS devices. Sample II was 57% female, 13% Hispanic, 72% white, mean age = 45 (SD = 21). Education distribution was: < high school (2%); high school (34%); some college (34%), college (20%); graduate degree (11%). Measures of executive function (r = -0.50; r=-0.57) and processing speed (r= -0.61) showed the expected negative correlation with age (all p <0.001). Negative correlations, although weaker, were also seen on measures of working memory (r=-0.2) and episodic memory (r=-0.2, r=-0.37; p.<.001). Vocabulary performance improved with age (r=0.4; p<.001), while spelling scores remained stable (r=0.09).

Initial studies support the validity and reliability of the first eight MTB cognitive measures in two diverse samples. MTB tests showed satisfactory construct validity, as demonstrated by the associations between MTB and well-established tests. Furthermore, most MTB measures correlated with age in the expected directions. Executive function, processing speed and memory typically decrease with age and this decrease was reflected in MTB test performance. In contrast, spelling and vocabulary, typically preserved as we age, did not decrease in our sample. Our results support the use of MTB in cognitive aging research.

A critical need in the neuropsychology field is development and validation of efficient, scalable assessments of cognition. The Mobile Toolbox (MTB), a novel suite of mobile device-compatible, app-based cognitive assessments, was developed to address this need. The goals of this study were (1) To collect longitudinal normative data for the MTB assessments in a large, ethnoculturally and educationally diverse cohort; (2) To assess the feasibility and usability of remote assessment using MTB.

Participants were recruited from the UCSF Brain Health Registry (BHR), an online cohort (N>100,000) that collects longitudinal cognitive, functional, behavioral, and health data using online neuropsychological tests and self- and study-partner report surveys. BHR participants who opted to learning about additional research opportunities were sent automated email invitations to enroll in the MTB study. Those who indicated study interest were provided instructions within the BHR online portal for downloading the MTB app. All participants had the opportunity to complete a single baseline administration of MTB (Word Meaning, Sequences, Spelling, Arranging Pictures, Arrow Matching, Faces and Names, Shape-Color Sorting, Number Match). Those who completed the baseline assessment within three days were invited to continue into the longitudinal study, where they complete MTB assessments at a single, short-term timepoint (day 7, 14, or 21; study arms sequentially assigned), and then at 6-month intervals. Enrollment across demographic groups was monitored, and study invitations were sent to specific demographic groups, with the goal of enrolling a sample of 800 participants in the longitudinal study: equal distribution across eight, 10-year age bands (ages 18-80+); 60% with <16 years of education; 10% non-Latinx Black, 15% Latinx, and 5% non-White other ethnocultural identity.

Between January-June 2022, 48,110 BHR participants were invited to the MTB study. Of those, 8294 (17%) expressed interest, 3401 (7%) completed the baseline assessment, 850 (1.8%) were assigned to the longitudinal study, and 782 (1.6%) completed a short-term longitudinal assessment. Study staff received 797 help tickets submitted by participants asking for email support to complete MTB. The baseline cohort had and average age of 64 years and an average of 16.6 years of education, 76.2% female, 2.1% non-Latinx Black, 7.1% Latinx, 86.8% non-Latinx White, and 4% from other ethnocultural groups. The longitudinal cohort had an average age of 62.3 years and an average of 16.1 years of education, 80% female, 2.8% non-Latinx Black, 8.5% Latinx, 83.5% non-Latinx White; and 5% other ethnocultural group. Compared to those invited to the study, those who enrolled in the longitudinal study were older, had higher educational attainment, and were more likely to be female and self-identify as non-Latinx White (p<0.05 for all).

Efficient enrollment and task completion of a large cohort in a novel, app-based mobile cognitive assessment is feasible in a completely remote setting. Most participants were able to complete MTB without individual support, indicating good usability. This approach can be scaled up to efficiently assess cognition in many research and healthcare settings. A remaining challenge is achieving robust ethnocultural and educational diversity.

To describe advantages and disadvantages of using digital assessments remotely and in-person to inform clinical and research practice.

As part of a larger study,1120 adults completed a battery of remotely administered tests (Mobile Toolbox) and a subset of this sample completed examiner administered in-person testing (NIH Toolbox® Cognition Battery). Attention was given to making the sample reflective of the US 2020 Census during participant recruitment. Of the 1120 participants, the majority of the sample were female (57%) and Caucasian (72%) and had a mean age of 45 (SD = 21). In terms of education, equal percentages had high school (34%) or some college (34%).

NIH Toolbox cognitive tests of processing speed, language, executive function, attention, and episodic memory were administered via a trained examiner and correlates of these tests were self-administered remotely via a smartphone. Using examples, we will show which aspects of cognitive assessment had the best correlations between remote self-administration and face-to-face examination and which had lower correlations.

Digital remote assessments can help overcome barriers by enabling repeated testing in naturalistic conditions, reducing participant burden and expense, and increasing research accessibility for populations currently under-represented. Moreover, the ubiquity of internet-connected devices vastly increases opportunities to remotely monitor other dimensions relevant to cognition using smartphone apps and wearable sensors. In addition to improving access to testing, digitally administered assessments dramatically improve some individual’s tolerance to testing with shorter tests that can be administered via computer adaptive testing (CAT). Despite these benefits, some aspects of the cognitive assessment cannot be adequately replicated remotely and thus yield lower correlations to their examiner-administered alternatives. Clinical and research implications are discussed.

Studying phenotypic and genetic characteristics of age at onset (AAO) and polarity at onset (PAO) in bipolar disorder can provide new insights into disease pathology and facilitate the development of screening tools.

To examine the genetic architecture of AAO and PAO and their association with bipolar disorder disease characteristics.

Genome-wide association studies (GWASs) and polygenic score (PGS) analyses of AAO (n = 12 977) and PAO (n = 6773) were conducted in patients with bipolar disorder from 34 cohorts and a replication sample (n = 2237). The association of onset with disease characteristics was investigated in two of these cohorts.

Earlier AAO was associated with a higher probability of psychotic symptoms, suicidality, lower educational attainment, not living together and fewer episodes. Depressive onset correlated with suicidality and manic onset correlated with delusions and manic episodes. Systematic differences in AAO between cohorts and continents of origin were observed. This was also reflected in single-nucleotide variant-based heritability estimates, with higher heritabilities for stricter onset definitions. Increased PGS for autism spectrum disorder (β = −0.34 years, s.e. = 0.08), major depression (β = −0.34 years, s.e. = 0.08), schizophrenia (β = −0.39 years, s.e. = 0.08), and educational attainment (β = −0.31 years, s.e. = 0.08) were associated with an earlier AAO. The AAO GWAS identified one significant locus, but this finding did not replicate. Neither GWAS nor PGS analyses yielded significant associations with PAO.

AAO and PAO are associated with indicators of bipolar disorder severity. Individuals with an earlier onset show an increased polygenic liability for a broad spectrum of psychiatric traits. Systematic differences in AAO across cohorts, continents and phenotype definitions introduce significant heterogeneity, affecting analyses.

Hispanics/Latinos are the largest and fastest-growing minority population in the United States. To facilitate appropriate outcome assessment of this expanding population, the NIH Toolbox for Assessment of Neurological and Behavioral Function® (NIH Toolbox®) was developed with particular attention paid to the cultural and linguistic needs of English- and Spanish-speaking Hispanics/Latinos.

A Cultural Working Group ensured that all included measures were appropriate for use with Hispanics/Latinos in both English and Spanish. In addition, a Spanish Language Working Group assessed all English-language NIH Toolbox measures for translatability.

Measures were translated following the Functional Assessment of Chronic Illness Therapy (FACIT) translation methodology for instances where language interpretation could impact scores, or a modified version thereof for more simplified translations. The Spanish versions of the NIH Toolbox Cognition Battery language measures (i.e., Picture Vocabulary Test, Oral Reading Recognition Test) were developed independently of their English counterparts.

The Spanish-language version of the NIH Toolbox provides a much-needed set of tools that can be selected as appropriate to complement existing protocols being conducted with the growing Hispanic/Latino population in the United States.