With the increased use of computer-based tests in clinical and research settings, assessing retest reliability and reliable change of NIH Toolbox-Cognition Battery (NIHTB-CB) and Cogstate Brief Battery (Cogstate) is essential. Previous studies used mostly White samples, but Black/African Americans (B/AAs) must be included in this research to ensure reliability.

Participants were B/AA consensus-confirmed healthy controls (HCs) (n = 49) or mild cognitive impairment (MCI) (n = 34) adults 60–85 years that completed NIHTB-CB and Cogstate for laptop at two timepoints within 4 months. Intraclass correlations, the Bland-Altman method, t-tests, and the Pearson correlation coefficient were used. Cut scores indicating reliable change provided.

NIHTB-CB composite reliability ranged from .81 to .93 (95% CIs [.37–.96]). The Fluid Composite demonstrated a significant difference between timepoints and was less consistent than the Crystallized Composite. Subtests were less consistent for MCIs (ICCs = .01–.89, CIs [−1.00–.95]) than for HCs (ICCs = .69–.93, CIs [.46–.92]). A moderate correlation was found for MCIs between timepoints and performance on the Total Composite (r = -.40, p = .03), Fluid Composite (r = -.38, p = .03), and Pattern Comparison Processing Speed (r = -.47, p = .006).

On Cogstate, HCs had lower reliability (ICCs = .47–.76, CIs [.05–.86]) than MCIs (ICCs = .65–.89, CIs [.29–.95]). Identification reaction time significantly improved between testing timepoints across samples.

The NIHTB-CB and Cogstate for laptop show promise for use in research with B/AAs and were reasonably stable up to 4 months. Still, differences were found between those with MCI and HCs. It is recommended that race and cognitive status be considered when using these measures.

Validated computerized assessments for cognitive functioning are crucial for older individuals and those at risk of cognitive decline. The National Institutes of Health (NIH) Toolbox Cognition Battery (NIHTB-CB) exhibits good construct validity but requires validation in diverse populations and for adults aged 85+. This study uses data from the Assessing Reliable Measurement in Alzheimer’s Disease and cognitive Aging study to explore differences in the factor structure of the NIHTB-CB for adults 85 and older, Black participants versus White participants, and those diagnosed as amnestic Mild Cognitive Impairment (aMCI) vs cognitively normal (CN).

Subtests from the NACC UDS-3 and NIHTB-CB were administered to 503 community-dwelling Black and White adults ages 55–99 (367 CN; 136 aMCI). Confirmatory factor analyses were used to investigate the original factor structure of NIHTB-CB that forms the basis for NIHTB-CD Index factor scores.

Factor analyses for all participants and some participant subsets (aMCI, White, 85+) substantiated the two anticipated factors (Fluid and Crystallized). However, while Black aMCI participants had the expected two-factor structure, for Black CN participants, the List Sorting Working Memory and Picture Sequence tests loaded on the Crystallized factor.

Findings provide psychometric support for the NIHTB-CB. Differences in factor structure between Black CN individuals and Black aMCI individuals suggest potential instability across levels of cognitive impairment. Future research should explore changes in NIHTB-CB across diagnoses in different populations.