237 results
Measuring Dispositional Empathy in South African Children
- Susan Malcolm-Smith, Lea-Ann Pileggi, Raphaella Lewis
-
- Journal:
- Acta Neuropsychiatrica / Accepted manuscript
- Published online by Cambridge University Press:
- 25 April 2024, pp. 1-22
-
- Article
- Export citation
-
Objective:
Empathy is a key factor to examine in development, because of its predictive associations with both aggression and successful prosocial behavior. However, established measures of empathy for Low-to-Middle Income Countries, including South Africa, are lacking. In children, parent-report measures are key. However, a local study examining empathy and aggression (Malcolm-Smith et al., 2015) found poor psychometric performance for a widely used parent-report measure of dispositional empathy, the Griffith Empathy Measure (GEM). We thus investigated which of two questionnaires measuring dispositional cognitive and affective empathy perform better in this context.
Method:We contrasted internal consistency reliability of a simplified version of the GEM (SGEM; n = 160) and a parent-report version of the Questionnaire of Cognitive and Affective Empathy (QCAE; n = 440) in a low-mid socio-economic status sample. Convergence between the measures and factor structure were also assessed.
Results:The parent-report version of the QCAE performed well as a measure of child dispositional cognitive and affective empathy, with good reliability (overall α= .90 vs. SGEM α= .63), and confirmatory factor analysis supporting the two-factor structure. The SGEM’s reliability and failure to correlate with QCAE indicated poor psychometric performance.
Conclusion:This is the first psychometric evaluation of the QCAE as a parent-report measure, and our results indicate that it should prove useful for future assessments of dispositional empathy in children across a variety of contexts.
Item Response Theory for Creativity Measurement
- Nils Myszkowski
-
- Published online:
- 16 February 2024
- Print publication:
- 14 March 2024
-
- Element
- Export citation
-
Item-response theory (IRT) represents a key advance in measurement theory. Yet, it is largely absent from curricula, textbooks and popular statistical software, and often introduced through a subset of models. This Element, intended for creativity and innovation researchers, researchers-in-training, and anyone interested in how individual creativity might be measured, aims to provide 1) an overview of classical test theory (CTT) and its shortcomings in creativity measurement situations (e.g., fluency scores, consensual assessment technique, etc.); 2) an introduction to IRT and its core concepts, using a broad view of IRT that notably sees CTT models as particular cases of IRT; 3) a practical strategic approach to IRT modeling; 4) example applications of this strategy from creativity research and the associated advantages; and 5) ideas for future work that could advance how IRT could better benefit creativity research, as well as connections with other popular frameworks.
Psychometric reliability, validity, and generalizability of 3MSE scores among American Indian adults: the Strong Heart Study
- Astrid M. Suchy-Dicey, Thao T. Vo, Kyra Oziel, Dedra S. Buchwald, Lonnie A. Nelson, Steven P. Verney, Brian F. French
-
- Journal:
- Journal of the International Neuropsychological Society , First View
- Published online by Cambridge University Press:
- 24 January 2024, pp. 1-10
-
- Article
- Export citation
-
Objective:
Modified Mini-Mental State Examination (3MSE) is often used to screen for dementia, but little is known about psychometric validity in American Indians.
Methods:We recruited 818 American Indians aged 65–95 for 3MSE examinations in 2010–2013; 403 returned for a repeat examination in 2017–2019. Analyses included standard psychometrics inferences for interpretation, generalizability, and extrapolation: factor analysis; internal consistency-reliability; test-retest score stability; multiple indicator multiple cause structural equation models.
Results:This cohort was mean age 73, majority female, mean 12 years education, and majority bilingual. The 4-factor and 2nd-order models fit best, with subfactors for orientation and visuo-construction (OVC), language and executive functioning (LEF), psychomotor and working memory (PMWM), verbal and episodic memory (VEM). Factor structure was supported for both research and clinical interpretation, and factor loadings were moderate to high. Scores were generally consistent over mean 7 years. Younger participants performed better in overall scores, but not in individual factors. Males performed better on OVC and LEF, females better on PMWM. Those with more education performed better on LEF and worse on OVC; the converse was true for bilinguals. All differences were significant, but small.
Conclusion:These findings support use of 3MSE for individual interpretation in clinic and research among American Indians, with moderate consistency, stability, reliability over time. Observed extrapolations across age, sex, education, and bilingual groups suggest some important contextual differences may exist.
89 The Neuropsychology of Complex Homelessness
- Jamie Berry, Arthur Shores, Karen Wallace, Peter Valpiani, Grace Rullis, Nicola Earls, Talia Nardo
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 80-81
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Although the cognitive profiles of people experiencing homelessness have been described in the literature, the neuropsychological profile of people experiencing complex homelessness has not been delineated. Complex homelessness is homelessness that continues despite the provision of bricks and mortar solutions. People experiencing complex homelessness often have an array of physical health, mental health, substance use, neurodevelopmental and neurocognitive disorders. The present study aimed to delineate the neuropsychological profile of people experiencing complex homelessness and explore the utility of neuropsychological assessment in supporting this population.
Participants and Methods:19 people experiencing complex homelessness in Sydney, Australia, were consecutively referred by specialist homelessness services for neuropsychological assessment. They underwent comprehensive assessment of intelligence, memory and executive functioning and completed questionnaires to screen for the presence of ADHD, PTSD, depression, anxiety and stress. A range of performance validity measures were included. Referrers were asked to complete questionnaires on history of childhood trauma, psychological functioning, drug and alcohol use, functional cognitive abilities, homelessness factors, personality, risk of cognitive impairment and adaptive functioning and to note existing or suspected mental health, neurodevelopmental and neurocognitive disorders. Referrers also completed a post-assessment pathways questionnaires to identify whether the neuropsychological assessment facilitated referral pathways (e.g., for government housing or financial assistance). Clinicians completed a post-assessment diagnosis survey, which was compared to the pre-assessment known or suspected diagnoses. Finally, referrers were asked to complete a satisfaction questionnaire regarding the neuropsychological assessment.
Results:Mean (SD) WAIS-IV indexes were VCI = 81.1 (14.5), PRI = 86.1 (10.9), WMI = 80.5 (13.0), PSI = 81.6 (10.2). Mean WMS-IV Flexible (LMVR) indexes were AMI = 68.3 (19.6), VMI = 77.1 (19.3), IMI = 72.7 (17.2), and DMI = 70.5 (17.6). The majority of participants showed unusual differences between WAIS-IV and TOPF-predicted WAIS-IV scores and between WAIS-IV General Ability and WMS-IV Flexible (LMVR) scores. Demographically corrected scores on tests of executive functioning were mostly one or more standard deviations below the mean. The majority of participants screened positive on screening measures of executive dysfunction, PTSD and ADHD and had elevated self-reported psychological distress scores. At least one new diagnosis was made for nine (47%) participants, established diagnoses were confirmed for two (11%) participants, diagnoses were supported for 15 (79%) participants, tentative diagnoses were made for 16 (84%) participants, and five (26%) participants had at least one diagnosis disconfirmed/unsupported. Referrers indicated that the majority of post-assessment pathways were more accessible following the neuropsychological assessment and that they were very satisfied with the neuropsychological assessments overall.
Conclusions:This is one of the first studies to delineate the neuropsychological profile of people experiencing complex homelessness using robust psychometric approaches, including performance validity tests. This population experiences a high burden of cognitive impairment and associated substance use, neurodevelopmental and mental health comorbidities. Neuropsychological assessment makes referral pathways more accessible and is valued by referrers of people experiencing complex homelessness.
43 Comparison of Latent Structures for the Neuropsychiatric Inventory Questionnaire (NPI-Q)
- Nicholas R Amitrano, Maximillian A Obolsky, Zachary J Resch, Jason R Soble, David A Gonzälez
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 723-724
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Existing research has demonstrated that neuropsychiatric/behavioral-psychological symptoms of dementia (BPSD) frequently contribute to worse prognosis in patients with neurodegenerative conditions (e.g., increased functional dependence, worse quality of life, greater caregiver burden, faster disease progression). BPSD are most commonly measured via the Neuropsychiatric Inventory (NPI), or its briefer, informant-rated questionnaire (NPI-Q). Despite the NPI-Q’s common use in research and practice, there is disarray in the literature concerning the NPI-Q’s latent structure and reliability, possibly related to differences in methods between studies. Also, hierarchical factor models have not been considered, even though such models are gaining favor in the psychopathology literature. Therefore, we aimed to compare different factor structures from the current literature using confirmatory factor analyses (CFAs) to help determine the best latent model of the NPI-Q.
Participants and Methods:This sample included 20,500 individuals (57% female; 80% White, 12% Black, 8% Hispanic), with a mean age of 71 (SD = 10.41) and 15 average years of education (SD = 3.43). Individuals were included if they had completed an NPI-Q during their first visit at one of 33 Alzheimer Disease Research Centers reporting to the National Alzheimer Coordinating Center (NACC). All CFA and reliability analyses were performed with lavaan and semTools R packages, using a diagonally weighted least squares (DWLS) estimator. Eight single-level models using full or modified versions of the NPI-Q were compared, and the top three were later tested in bifactor form.
Results:CFAs revealed all factor models of the full NPI-Q demonstrated goodness of fit across multiple indices (SRMR = 0.039-0.052, RMSEA = 0.025-0.029, CFI = 0.973-0.983, TLI = 0.9670.977). Modified forms of the NPI-Q also demonstrated goodness of fit across multiple indices (SRMR = 0.025-0.052, RMSEA = 0.0180.031, CFI = 0.976-0.993, TLI = 0.968-0.989). Top factor models later tested in bifactor form all demonstrated consistently stronger goodness of fit regardless of whether they were a full form (SRMR = 0.023-0.035, RMSEA = 0.015-0.02, CFI = 0.992-0.995, TLI = 0.985-0.991) or a modified form (SRMR = 0.023-0.042, RMSEA = 0.015-0.024, CFI = 0.985-0.995, TLI = 0.9770.992). Siafarikas and colleagues’ (2018) 3-factor model demonstrated the best fit among the full-form models, whereas Sayegh and Knight’s (2014) 4-factor model had the best fit among all single-level models, as well as among the bifactor models.
Conclusions:Although all factor models had adequate goodness of fit, the Sayegh & Knight 4-factor model had the strongest fit among both single-level and bifactor models. Furthermore, all bifactor models had consistently stronger fit than single-level models, suggesting that BPSD are best theoretically explained by a hierarchical, non-nested framework of general and specific contributors to symptoms. These findings also inform consistent use of NPI-Q subscales.
35 Preliminary reliability of the Coma Recovery Scale, Revised (CRS-R) in children with a history of disorders of consciousness after acquired brain injury
- Natasha N Ludwig, Stacy Suskauer, Beth Slomine
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, p. 143
-
- Article
-
- You have access Access
- Export citation
-
Objective:
The Coma Recovery Scale-Revised (CRS-R) is the gold standard assessment of adults with disorders of consciousness (DoC); however few studies have examined the psychometric properties of the CRS-R in pediatric populations. This study aimed to demonstrate preliminary intra-rater and inter-rater reliability of the CRS-R in children with acquired brain injury (ABI).
Participants and Methods:Participants included 3 individuals (ages 10, 15, and 17 years) previously admitted to an inpatient pediatric neurorehabilitation unit with DoC after ABI who were followed in an outpatient brain injury clinic due to ongoing severe disability. ABI etiology included traumatic brain injury (TBI; n=2) and encephalitis (n=1). Study participation took place on average 4.6 years after injury (range 2-9). The Glasgow Outcome Scale-Extended, Pediatric Version (GOS-E Peds), a measure of outcome after pediatric brain injury, was administered as part of screening. Two participants were placed in the GOS-E Peds “lower severe disability” category (i.e., score of 6) and one was placed in the “upper severe disability” category (i.e., score of 5). The CRS-R includes 6 subscales measuring responsivity including Auditory (range 0-4), Visual (range 05), Motor (range 0-6), Oromotor/Verbal (range 03), Communication (range 0-2), and Arousal (range 0-3) with higher scores indicating higherlevel function. Subscales are totaled for a CRS-R Total score. Behaviors shown during the CRS-R are used to determine state of DoC [Vegetative State (VS), Minimally Conscious State (MCS) or emergence from a minimally conscious state (eMCS)] based on 2002 Aspen Guidelines. Participants were administered the CRS-R three consecutive times on the same day. Administrations were completed by two raters in this order: Rater 1 (1A), Rater 1 (1B) and Rater 2. Intra-rater reliability was deemed by percent agreement across the 6 subscales between Rater 1A and 1B. Inter-rater reliability was deemed by percent agreement across the 6 subscales between 1A and 2.
Results:Mean CRS-R Total score for Rater 1A was 22 (SD=1.73, range 20-23), Rater 1B was 22 (SD=1.73, range 20-23), and Rater 2 was 21.33 (SD=2.08, range 19-23). Intra-rater reliability was 100% and inter-rater reliability was 94% across all subscales. All participants were deemed eMCS at all 3 ratings.
Conclusions:Data from this very small sample of children suggests that the CRS-R demonstrates both intra-rater and inter-rater reliability in patients with a history of DoC after ABI. Given that all children were at the high end of the scale (eMCS), further research is needed with a larger sample of children with a range of states of DoC.
42 Cognitive Impairment Stage and Dementia Syndromes Explain Latent Structure Variability on the Neuropsychiatric Inventory Questionnaire (NPI-Q)
- Nicholas R Amitrano, Maximillian A Obolsky, Zachary J Resch, Jason R Soble, David A Gonzälez
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 722-723
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Neuropsychiatric/behavioral-psychological symptoms of dementia (BPSD) frequently contribute to worse prognosis of patients with neurodegenerative conditions. BPSD are commonly measured via a brief, informant-rated version of the Neuropsychiatric Inventory (NPI), the NPI-Q. Previously (see our other submission to this conference), we established optimal latent structures by comparing different factor models in the literature using confirmatory factor analyses (CFAs). However, questions remain as to why so many different models were found in the literature. One possibility is sampling differences, including different proportions of individuals across cognitive stages (e.g., mild cognitive impairment, moderate dementia) or syndromes (e.g., Alzheimer’s amnestic syndrome, Dementia with Lewy Bodies). We tested this hypothesis by subjecting candidate models to measurement invariance (MI) analyses stratified by cognitive stage and syndrome.
Participants and Methods:Individuals were included if they had completed an NPI-Q during their first visit at an Alzheimer Disease Research Center reporting to the National Alzheimer Coordinating Center (NACC). This resulted in 20,500 individuals (57% female; 80% White, 13% Black, 8% Hispanic), with a mean age of 71 (SD = 10.41) and 15 average years of education (SD = 3.43). Regarding staging, 75.9% of individuals did not meet criteria for all-cause dementia, whereas 24.1% individuals had all-cause dementia. Regarding syndromes, 35.6% had an Alzheimer’s presentation (“AD-type”) and 5.6% had either a behavioral variant frontotemporal dementia or Lewy-Body dementia presentation (“behavioral-type”). A 3-factor and 4-factor model were subject to MI across these groupings. We conducted MI analyses for equal forms, equal loadings, and equal intercepts using the lavaan R package with a diagonally weighted least squares (DWLS) estimator.
Results:The 3-factor model demonstrated good fit among individuals experiencing (CFI = 0.965, TLI = 0.955) and not experiencing (CFI = 0.984, TLI = 0.979) dementia, as well as among AD-type (CFI = 0.983, TLI = 0.978) presentations, but had borderline poor fit for behavioral-type (CFI = 0.932, TLI = 0.912) presentations. The 4-factor model had better fit among those experiencing (CFI = 0.985, TLI = 0.977) and not experiencing (CFI = 0.995, TLI = 0.992) dementia. Additionally, the 4-factor model demonstrated good of fit for AD-type (CFI = 0.993, TLI = 0.989) and poorer fit for behavioral-type (CFI = 0.949, TLI = 0.922) syndromes. Chi-square differences suggested that equal loading and equal intercept hypotheses should be rejected for both 3- and 4-factor models, for both staging and syndromal groupings. However, relative fit indices suggested that the equal form, equal loading, and equal intercept hypotheses could be adequate for only the 4-factor model.
Conclusions:The variability of factor structures in the BPSD literature appears, at least partially, explained by sampling variability among cognitive stages and dementia syndromes. The best models in the literature appear to have good fit in non-demented individuals and, among those who have dementia, in those with an AD syndrome. Only Sayegh & Knight’s 4-factor model had adequate (albeit, not optimal) fit among those with all-cause dementia and, more specifically, among those with a behavioral-type dementia syndrome. These findings inform BPSD theory and practical implementation of NPI-Q subscales.
3 Ethnoracial Differences in Anchor Agreement and MCID Estimation in Alzheimer’s Disease
- Samantha E John, Stacey Moeller, Denise Tanner
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 506-507
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Alzheimer’s disease (AD) clinical trials lack diverse representation, limiting their generalizability. In addition, the clinical meaningfulness of observed changes during treatment may vary as a function of participant characteristics. Defining meaningful change in AD within diverse ethnoracial groups is therefore greatly needed. Meaningful change in AD trials can be assessed by three different anchors: participants, informants, or clinicians. Previous research has suggested that estimations of the minimal clinically important difference (MCID) vary by disease severity, choice of anchor, and anchor agreement. These relationships have been studied primarily within non-Hispanic white (NHW) samples. This project evaluates anchor-based MCID within and across the three most prevalent ethnoracial groups in the United States, non-Hispanic White (NHW), Hispanic/Latino (H/L), and Black/African-American (B/AA).
Participants and Methods:Data from the National Alzheimer’s Coordinating Center Uniform Dataset (NACC UDS) were used to investigate MCID within older adults (ages 50+) diagnosed as cognitively normal or cognitively impaired due to suspected AD. Data were taken from all versions of the UDS and consisted of all available participants with two consecutive annual visits. The identified sample (N=22,043) is approximately 83.6% NHW, 4.7% H/L, and 11.7% B/AA. Participant, informant, and clinician anchor variables were utilized to compare proportions of anchor agreement within and across ethnoracial groups. MCID on the Mini-Mental State Exam (MMSE) was estimated within each ethnoracial group and compared across the independent variables of anchor agreement and disease severity (cognitively normal (CN), mild cognitive impairment (MCI), and dementia) in 2x3 ANOVAs.
Results:Participant age (M = 71.56, SD = 9.03) did not significantly differ across ethnoracial groups; years of education significantly differed across groups, p < .001, with NHW (M=15.83 SD=3.05), H/L (M=12.49, SD=5.01), and B/AA (M=14.42, SD=3.22). Across all three anchors (participant, informant, clinician), unanimous agreement about the presence or absence of a decline in functioning was present in about 75.1% of the full sample. To further explore agreement differences across groups, anchor agreement was classified into a 3-level variable: 1) agreement that the participant remained stable over time, 2) agreement that the participant declined, and 3) disagreement. The proportion of each level within each ethnoracial group was significantly different, (x2(4, n = 22,043) = 179.16, p < .001, phi = .09, NHW (34.5% agreement-stable, 41.4% agreement-declined, 24.1% disagreement), H/L (30.5%, 42.6%, 26.9%, respectively), and B/AA (42.2%, 28.1%, 29.7%, respectively). MCID estimates on the MMSE followed similar trends within each ethnoracial group. There was a significant main effect of disease severity, such that MCID estimates increased in magnitude with increasing disease severity. There were no significant main effects of anchor agreement for any ethnoracial group. Within the NHW sample only, an interaction effect between diagnostic severity and anchor agreement was significant (p = .007).
Conclusions:Across ethnoracial groups, MCID estimates on the MMSE are reliably influenced by the severity of disease. However, the benefit of anchor-based MCID estimates may vary by ethnoracial group with respect to both anchor choice and use of anchor agreement. The origins of anchor disagreement and perceived stability in functioning warrant further exploration.
11 The Psychometric Characteristics of a Novel Metamemory Questionnaire for Children
- Mary Godfrey, Samantha van Terheyden, Gabriel Loud, Jack Wiese, Maxine Reger, Christopher Vaughan, Gerard Gioia
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 529-530
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Metamemory is a component of metacognition that includes both the knowledge of factors that affect memory (i.e., declarative metamemory) and knowledge and application of factors in one's own learning and recall performance (i.e., procedural metamemory; Kreutzer et al., 1975). Previous researchers have examined children's metamemory through interviews and found that metamemory abilities are positively associated with age and performance on memory measures (see Godfrey et al., 2022 for review). However, there is not yet a standardized measure to evaluate children's metamemory. The current study aimed to examine the psychometric characteristics of a declarative metamemory questionnaire, the Measure of Metamemory (MoM-10), for children ages 6-12 years old. Based on previous research, we hypothesized that performance on the MoM-10 would not be associated with sex but would be positively associated with age and learning and memory performance.
Participants and Methods:A total of 75 English-speaking typically developing children between the ages of 6 to 12 years old were recruited for the current study (M age=9.1+1.92; females 49%). Participants completed the MoM-10 which assessed declarative metamemory via 10 multiple choice questions (accuracy score of 0 or 1 points per question) and required participants to provide an explanation for their multiple-choice answer (explanation score of 0, 1, or 2 points per question). The metamemory questionnaire provided two outcome variables: an Accuracy score of 10 possible points, and an Explanation score of 20 possible points. Additionally, participants completed a 3-trial pictorial learning/memory task which provided an Immediate Recall score and Delayed Recall score.
Results:As hypothesized, there were no sex differences on the MoM-10 Accuracy scores (t(73)=0.71, p=0.48) or Explanation scores (t(73)=-73, p=0.47). Consistent with our hypothesis, age was significantly associated with Accuracy (r=0.31, p<0.01) and Explanation scores (r=0.79, p<0.001). Internal consistency of the MOM-10 was moderate for the Explanation score (Cronbach's alpha=0.68) and low for the Accuracy score (KR-20=0.54). Lastly, after controlling for age, participants' MoM-10 Accuracy score was significantly associated with Immediate Recall (r=0.32, p<0.01) on the learning/memory task and the Explanation scores were significantly associated with the Immediate Recall (r=0.36, p<0.01) and Delayed Recall scores (r=0.32, p<0.01) on the learning/memory task.
Conclusions:The current study presents an initial review of psychometric properties of a metamemory questionnaire for children ages 6 to 12 years old. Additionally, as hypothesized, these results suggest the MOM-10 performance is significantly positively associated with participants' age and immediate and delayed recall performance on a pictorial learning/memory task. These associations provide lines of evidence for convergent validity given the expected maturation of metamemory with both age and with improvements in actual memory performance. However, based on the low internal consistency of the accuracy scores, further refinement will be explored including possibly rephrasing questions from the current item set or perhaps excluding current items in future use of the scale.
Symposium 08: Neuropsychological Considerations for Alzheimer's Disease Clinical Trials
- Andrew Kiselica, Kevin Duff
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 504-505
-
- Article
-
- You have access Access
- Export citation
-
The 2011 National Institute on Aging and Alzheimer's Association (NIA-AA) criteria for the diagnosis of Alzheimer's disease (AD) focused on clinical signs and symptoms to make a diagnosis of probable or possible AD. Under these criteria, emphasis was placed on gathering objective evidence of cognitive decline, which gave neuropsychologists a central role as diagnosticians in AD clinical trials. The release of the 2018 NIA-AA research framework put greater emphasis on the use of biomarkers, especially measures of amyloid, tau, and neurodegeneration, to define AD. Once AD is defined based on these biomarkers, it is staged via clinical signs and symptoms. Thus, the role of neuropsychologists has shifted from being central to diagnosis to a possibly more ancillary role of staging the disease once it is determined to be present. The move away from clinical signs towards biomarkers only became more prominent with the recent, controversial Food and Drug Administration approval of Aducanumab as an AD treatment based on evidence of change in biomarkers without clear evidence of clinical benefit. In this landscape, the fit of neuropsychologists in AD clinical trial research has become less clear.
This symposium will address the role of neuropsychologists in modern AD clinical trial research. The presenters will highlight varied ways in which neuropsychologists can enrich and improve AD clinical trials. First, Dr. Dustin Hammers from Indiana University will discuss how neuropsychological methods can help us to understand which participants do, and perhaps more importantly, do not get enrolled in clinical trials. Second, Dr. Mirella Diaz-Santos from the University of California Los Angeles will summarize her work to enroll Hispanic individuals in the Human Connectome Project, improving inclusivity. Third, Dr. Tamar Gollan from the University of California San Diego will summarize her work on novel behavioral markers of AD risk discovered from the study of Spanish-English bilingual patients. Fourth, Dr. Andrew Kiselica from the University of Missouri will highlight psychometric considerations in interpreting clinically meaningfully change in AD clinical trials using data from the National Alzheimer's Coordinating Center. Fifth, Dr. Samantha John from the University of Nevada at Las Vegas will discuss the influence of race/ethnicity on how clinically meaningful change is defined using data from a diverse cohort.
Dr. Kevin Duff will serve as discussant for this series of studies. He will highlight the important roles that neuropsychologists can play in improving AD clinical trial screening processes, expanding inclusion of diverse patients into trials, and enhancing interpretation of the clinical meaningfulness of trial results. He will also reflect on the future of neuropsychology's role in the AD clinical trial landscape and encourage audience questions and responses to the research presented.
1 Clinically Meaningful Change in Alzheimer’s Disease Depends on Anchor Agreement and Disease Severity
- Andrew M Kiselica, Cynthia M Mikula, Samantha John, Marta Stojanovic
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 505-506
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Measures of clinical significance are critical for meaningful interpretation of treatment outcome research on Alzheimer’s disease. A common method of quantifying clinical significance is to calculate a minimal clinically important difference (MCID), which represents the smallest numerical change on an outcome measure that corresponds to an added benefit in a patient’s life. Often the MCID is calculated based on an anchor response. Individuals who report a meaningful change serve as the “anchors”, and the mean level of change for this group serves as the MCID. In research on Alzheimer’s disease, there are several possible raters to provide anchors, including patients, family observers, and clinicians, who may or may not agree on whether there has been a meaningful change in outcome. The goal of this study was to examine the extent to which agreement among anchors impacts MCID estimation and whether this relationship is moderated by cognitive severity status.
Participants and Methods:Analyses were completed on a longitudinal sample of 2,247 adults, age 50-103, from the Uniform Data Set 3.0. Outcome measures included the Clinical Dementia Rating - Sum of Boxes (CDR-SB), Functional Activities Questionnaire, and Montreal Cognitive Assessment.
Results:For all of the outcomes, the MCID estimate was significantly higher when meaningful decline was endorsed by all of the raters compared to situations in which there was disagreement among the raters. For example, on the CDR-SB, agreement significantly impacted MCID estimates (F(1, 2241)=168.80, p<0.001; partial h2 = 0.07), such that the agreement group had greater CDR-SB change score (mean=1.29, SD1.98) than the no agreement group (mean=0.37, SD=1.38; Tukey HSD: p<0.001). In addition, the MCID estimate increased with increasing levels of cognitive impairment. For instance, on the CDR-SB, MCID estimates were significantly different across the severity groups (F(2, 2241)=138.27, p<0.001; partial h2 = 0.11), such that increase in CDR-SB was highest for the mild dementia group (mean=1.84, SD=2.42), moderate in the MCI group (mean=0.71, SD=1.30), and lowest for the cognitively normal group (mean=0.07, SD=0.55; Tukey HSD; all p’s < 0.001). Finally, cognitive severity status moderated the influence of agreement among raters on MCID estimation for the CDR-SB and FAQ, such that rater agreement demonstrated less influence on the MCID as disease severity increased. For example, on the CDR-SB, post-hoc tests revealed that there was a significant difference across agreement groups in the cognitively normal (p<0.001; Cohen’s d = 0.96) and MCI groups (p<0.001; Cohen’s d = 0.49), but agreement did not impact MCID estimates for the mild dementia group (p=0.065).
Conclusions:MCID estimates based on one anchor may underestimate meaningful change, and researchers should consider the viewpoints of multiple raters in constructing MCIDs. Consideration of agreement appears most important in the early stages of cognitive decline, which are the focus of most modern clinical trials.
12 The Development of a Pediatric Metamemory Questionnaire and Scoring Procedure
- Jack Wiese, Mary Godfrey, Samantha van Terheyden, Gabriel Loud, Maxine Reger, Gerard Gioia, Christopher Vaughan
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 530-531
-
- Article
-
- You have access Access
- Export citation
-
Objective:
To create a standardized scoring procedure to evaluate open-ended responses as part of a novel questionnaire (Measure of Metamemory; MoM) designed to assess declarative metamemory in youth. Metamemory is an aspect of metacognition that is one’s knowledge of the factors related to storage and retrieval of information (Flavell 1971; Kreutzer et al., 1975), and includes both declarative metamemory (i.e., one’s knowledge about factors influencing memory) and procedural metamemory (i.e., one’s understanding of their own memory performance).
Participants and Methods:Fourteen short vignettes related to memory were administered to 100 participants (age 6-12) with questions such as, “Two children hear a story they must remember. The first person is 5 years old. The second person is 12 years old. Who is most likely to remember it best?” After answering, they were then prompted to explain their answer (i.e., “Why?”) and their responses were recorded verbatim. To develop standardized and objective criteria for each of the 14 open-ended responses, responses from a subsample of 20 youth were collectively examined by the study team and a scoring structure similar to open-ended items on common intelligence tests (e.g., WISC-V/WAIS-IV) was created. Two points (full credit) were awarded for complete and thorough understanding of memory processes related to the question; 1 point was given for partially accurate or incomplete understanding of the related memory process; and 0 points for an inability to correctly express an understanding of relevant memory concepts. This scoring guide was then applied independently by each of the six raters to an additional 25 participants (ages 6-12 mean age (SD)). To assess the interrater reliability of this 3-point ordinal scoring system, we examined both Fleiss’ kappa and 2-way random-effects, single-rater, absolute agreement Intra-Class Correlations (ICC).
Results:Across the six independent raters, reliability coefficients for each of the 14 items ranged from (Fleiss') k =. 277 to .792 (ICC ranged from .481 to .880). Of these 14 items, the kappa value was classified (using interpretation rules for Cohen’s kappa) as “substantial” for 10 items, “moderate” for 3 items, and “fair” for 1 item. Based on these lower inter-rater reliabilities, two items were subsequently removed from the measure to create the 12-item open-ended measure of metamemory, the MoM-12, with reliable scoring for youth as young as 6 years old.
Conclusions:A consensus process established a quantifiable scoring procedure to assess open-ended responses related to youth’s knowledge of memory (e.g., metamemory). Reliability metrics identified acceptable interrater reliabilities in 12 of the 14 original items. Further examination of psychometric properties, including internal consistency and lines of evidence for validity is needed. The successful crafting of a scoring procedure is a first step towards developing a psychometrically sound measure (the MoM) to evaluate and study metamemory concepts objectively and reliably in youth.
52 Bayesian Logistic Regression Bias Adjustment for Data Observed without a Gold Standard: A Simulation Study of Clinical Alzheimer’s Disease
- William F Goette, Hudaisa Fatima, Jeff Schaffert, Anne R Carlew, Heidi Rossetti, Laura H Lacritz, C. Munro Cullum
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 259-260
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Definitive diagnosis of Alzheimer’s disease (AD) is often unavailable, so clinical diagnoses with some degree of inaccuracy are often used in research instead. When researchers test methods that may improve clinical accuracy, the error in initial diagnosis can penalize predictions that are more accurate to true diagnoses but differ from clinical diagnoses. To address this challenge, the current study investigated the use of a simple bias adjustment for use in logistic regression that accounts for known inaccuracy in initial diagnoses.
Participants and Methods:A Bayesian logistic regression model was developed to predict unobserved/true diagnostic status given the sensitivity and specificity of an imperfect reference. This model considers cases as a mixture of true (with rate = sensitivity) and false positives (rate = 1 - specificity) while controls are mixtures of true (rate = specificity) and false negatives (rate = 1 - sensitivity). This bias adjustment was tested using Monte Carlo simulations over four conditions that varied the accuracy of clinical diagnoses. Conditions utilized 1000 iterations each generating a random dataset of n = 1000 based on a true logistic model with an intercept and three arbitrary predictors. Coefficients for parameters were randomly selected in each iteration and used to produce a set of two diagnoses: true diagnoses and observed diagnoses with imperfect accuracy. Sensitivity and specificity of the simulated clinical diagnosis varied with each of the four conditions (C): C1 = (0.77, 0.60), C2 = (0.87, 0.44), C3 = (0.71, 0.71), and C4 = (0.83, 0.55), which are derived from published values for clinical AD diagnoses against autopsy-confirmed pathology. Unadjusted and bias-adjusted logistic regressions were then fit to the simulated data to determine the models’ accuracy in estimating regression parameters and prediction of true diagnosis.
Results:Under all conditions, the bias-adjusted logistic regression model outperformed its unadjusted counterpart. Root mean square error (the variability of estimated coefficients around their true parameter values) ranged from 0.23 to 0.79 for the unadjusted model versus 0.24 to 0.29 for the bias-adjusted model. The empirical coverage rate (the proportion of 95% credible intervals that include their true parameter) ranged from 0.00 to 0.47 for the unadjusted model versus 0.95 to 0.96 for the bias-adjusted model. Finally, the bias-adjusted model produced the best overall diagnostic accuracy with correct classification of true diagnostic values about 78% of the time versus 62-72% without adjustment.
Conclusions:Results of this simulation study, which used published AD sensitivity and specificity statistics, provide evidence that bias-adjustments to logistic regression models are needed when research involves diagnoses from an imperfect standard. Results showed that unadjusted methods rarely identified true effects with credible intervals for coefficients including the true value anywhere from never to less than half of the time. Additional simulations are needed to examine the bias-adjusted model’s performance under additional conditions. Future research is needed to extend the bias adjustment to multinomial logistic regressions and to scenarios where the rate of misdiagnosis is unknown. Such methods may be valuable for improving detection of other neurological disorders with greater diagnostic error as well.
84 Preliminary Psychometric Examination of a Short Questionnaire of Executive Functions
- Justin E. Karr
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 285-286
-
- Article
-
- You have access Access
- Export citation
-
Objective:
The behavioral assessment of executive functions has become increasingly common in clinical practice, with a self-report measure of executive functions becoming one of the most commonly administered assessment instruments of the construct in clinical practice. These subjective measurements serve as an alternative to objective tests of executive functions, which have been criticized for poor ecological validity. Many behavioral measures of executive functions are now available, but there are some issues with those currently in use, in that many are lengthy, proprietary, and/or do not measure executive functions that align with a theoretical framework of the multidimensional construct. This study aimed to examine the psychometric properties of a new short questionnaire of executive functions designed to be concise, theoretically based, and ultimately freely available for use in research and clinical practice.
Participants and Methods:Participants included 575 college undergraduate students who completed an online questionnaire to earn credit in psychology courses. They were, on average, 18.9 years-old (SD=1.0, range: 18-22), 82.4% female, and 78.8% White. All participants completed 20 self-report items on a four-point ordinal scale measuring five theorized executive function constructs of Planning, Inhibition, Working Memory, Shifting, and Emotional Control. The 20 items were analyzed using confirmatory factor analysis and factor reliabilities were estimated using omega. As a validity analysis, correlations between the total score with measures of subjective cognition and ADHD symptoms were compared to correlations between the total score with measures of anxiety and depression, hypothesizing stronger correlations of executive functions with cognition and ADHD than negative affect.
Results:The initial 20-item model did not fit well, x2=1560.10, df=160, p<.0001, CFI=0.822, TLI=0.788, RMSEA=0.130 (90% CI: 0.1240.136). The polychoric inter-item correlations were examined for high cross-factor correlations and low intra-factor correlations. This process resulted in the removal of one item from each factor, The modified model, inclusive of 15 items, presented with adequate fit to the data, X2=470.56, df=80, p<.0001, CFI=0.936, TLI=0.916, RMSEA=0.097 (90% CI: 0.0890.106). The total score has good reliability (Q=.82), whereas estimates for each factor ranged from .56 to .79. The total score showed a stronger correlation with ADHD symptoms (r=-.59) and subjective cognition (r=.59) than depression (r=.46, z=4.05, p<.001) and anxiety symptoms (r=.38, z=6.29, p<.001).
Conclusions:These preliminary findings provided modest psychometric support for this short 15-item self-report questionnaire of executive functions. The questionnaire had acceptable fit and evidence for validity, in that the total executive function score had a stronger correlation with subjective cognitive complaints and ADHD symptoms than negative affect. The reliability of some individual factors fell below conventional cutoffs for acceptable reliability, indicating a need for further refinement of this new questionnaire.
2 Validity and Reliability of Mobile Toolbox Cognitive Assessments
- Cindy J Nowinski, Aaron Kaat, Jerry Slotkin, Erika La Forte, Yusuke Shono, Miriam Novack, Sarah Pila, Elizabeth Dworak, Stephanie R Young, Zahra Hosseinian, Hubert Adam, Richard Gershon
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 780-781
-
- Article
-
- You have access Access
- Export citation
-
Objective:
To present validation evidence for the first eight cognitive measures available through Mobile Toolbox (MTB). These measures use a remote self-administered platform to assess language, working memory, episodic memory, executive function, and processing speed.
Participants and Methods:We used two separate samples, recruited as part of a larger study, to validate MTB measures. Sample I, comprised of 92 English-speaking adults ages 18-85, was used to assess internal consistency and construct validity. Participants were first administered “gold standard” cognitive measures (Wechsler Memory Scale-IV Verbal Paired Associates I and II; Wechsler Adult Intelligence Scale-IV Symbol Search, Digit Span, Coding, and Letter-Number Sequencing; Delis-Kaplan Executive Function System Color-Word Interference Test, Peabody Picture Vocabulary Test, Wechsler Individual Achievement Test-4 Spelling, and the Wisconsin Card Sorting Test), after which they completed MTB (pre-loaded on a study-provided smartphone) on their own. Internal consistency was evaluated using measure-appropriate indices (split-half reliability, Cronbach’s alpha or IRT-based indices). Pearson correlation coefficients between MTB tests and measures of similar constructs were used to evaluate concurrent validity. For two tests with timing-dependent scores, Arrow Matching and Shape-Color Sorting, separate analyses were performed for iOS and Android devices. Sample II, with 1,120 English-speaking participants ages 18-90, was used to evaluate age-related change. Participants completed MTB measures remotely on their own smartphones, in a preset order, within a 14-day period. Spearman correlation coefficients, corrected for education, were calculated to evaluate relationships between age and test scores.
Results:Sample I participants were 67% female, 52% white, 99% non-Hispanic; average age=48 (SD= 17). Education was: < high school (1%); high school (55%); some college (21%); college (15%); graduate degree (8%). Internal consistency estimates ranged from 0.81 to 0.99. Pearson correlations between MTB and external measures ranged from 0.41 to 0.86 (all p < .01). Of the timed tests, only Shape-Color sorting showed significant score differences between Android and iOS devices. Sample II was 57% female, 13% Hispanic, 72% white, mean age = 45 (SD = 21). Education distribution was: < high school (2%); high school (34%); some college (34%), college (20%); graduate degree (11%). Measures of executive function (r = -0.50; r=-0.57) and processing speed (r= -0.61) showed the expected negative correlation with age (all p <0.001). Negative correlations, although weaker, were also seen on measures of working memory (r=-0.2) and episodic memory (r=-0.2, r=-0.37; p.<.001). Vocabulary performance improved with age (r=0.4; p<.001), while spelling scores remained stable (r=0.09).
Conclusions:Initial studies support the validity and reliability of the first eight MTB cognitive measures in two diverse samples. MTB tests showed satisfactory construct validity, as demonstrated by the associations between MTB and well-established tests. Furthermore, most MTB measures correlated with age in the expected directions. Executive function, processing speed and memory typically decrease with age and this decrease was reflected in MTB test performance. In contrast, spelling and vocabulary, typically preserved as we age, did not decrease in our sample. Our results support the use of MTB in cognitive aging research.
85 Performance Consistency on a Measure of Sustained and Selective Attention
- Lauren M. Baumann, Keith P. Johnson, Lee Ashendorf
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, p. 286
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Attention concerns, particularly difficulties with focusing and regulating attention, are reported in diverse clinical contexts. The Ruff 2&7 Selective Attention Test (Ruff 2&7; Ruff & Allen, 1996) is a measure of sustained and selective attention that assesses automatic detection and effortful processing. The goal of this study was to create an internal consistency metric within this test and to determine cognitive predictors by evaluating associations with executive control of attention and other cognitive skills. It was hypothesized that those who are more consistent across Ruff 2&7 performance would have more robust executive functioning skills, particularly those related to regulating and directing attention and the planning and utilization of cognitive resources.
Participants and Methods:The current study examined a clinical sample of 98 United States veterans with a history of mild traumatic brain injury. After excluding invalid cases (n=24), the final sample consisted of 74 veterans (Age=38.5 (8.9) years old; 13.9 (2.2) years of education; 78% male; 82% white, 7% Black, 8% Hispanic, 2% Asian). A consistency score was defined as the absolute value of the intertrial change in target hits plus errors across each pair of trials of the same stimulus type (Automatic Detection, AD, and Controlled Search, CS). Hierarchical linear regression modeling was used to evaluate the relative contributions of memory and executive functions (Rey Auditory Verbal Learning Test, Delis-Kaplan Executive Function System Tower Test, phonemic fluency, Trail Making Test B) and subjective symptom report (PTSD Checklist for DSM-5, Barkley Adult ADHD Rating Scale for DSM-IV).
Results:The mean deviation scores for the two trial types were similar (AD mean=13.6, SD=5.9; CS mean=13.6, SD=5.3). In predicting consistency across AD trials, delayed recall contributed 11% unique variance (p=.013), while no other block was statistically significant. For CS trials, self-reported PTSD and inattention symptoms contributed a combined 20% of unique variance to the model (p=.007), while there were no statistically significant cognitive predictors in this model.
Conclusions:Contrary to expectation, executive function measures did not explain statistically significant variance in performance across either trial type. Less consistent performance on AD trials was associated with weaker verbal memory. Less consistent performance on CS trials, which theoretically require greater executive control, was not associated with any cognitive scores, but was associated with more severe self-reported psychological and inattention symptoms. These findings buttress the conceptual distinction between AD and CS trial types, and they point to both cognitive and non-cognitive underpinnings of performance consistency.
30 Item response theory and differential item functioning of the AD8: The High School & Beyond Study
- Mark Lee, Justina F Avila-Rieger, Rob Warren, Eric Grodsky, Chandra Muller, Adam M Brickman, Jennifer J Manly
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, p. 240
-
- Article
-
- You have access Access
- Export citation
-
Objective:
The AD8 is a validated screening instrument for functional changes that may be caused by cognitive decline and dementia. It is frequently used in clinics and research studies because it is short and easy to administer, with a cut off score of 2 out of 8 items recommended to maximize sensitivity and specificity. This cutoff assumes that all 8 items provide equivalent “information” about everyday functioning. In this study, we used item response theory (IRT) to test this assumption. To determine the relevance of this measure of everyday functioning in men and women, and across race, ethnicity, and education, we conducted differential item functioning (DIF) analysis to test for item bias.
Participants and Methods:Data came from the 2021 follow up of the High School & Beyond cohort (N=8,690; mean age 57.5 ± 1.2; 55% women), a nationally representative, longitudinal study of Americans who were first surveyed in 1980 when they were in the 10th or 12th grade. Participants were asked AD8 questions about their own functioning via phone or internet survey. First, we estimated a one-parameter (i.e., differing difficulty, equal discrimination across items) and two-parameter IRT model (i.e., differing difficulty and differing discrimination across items). We compared model fit using a likelihood-ratio test. Second, we tested for uniform and non-uniform DIF on AD8 items by sex, race and ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic), education level (high school or less, some college, BA degree or more), and survey mode (phone or internet). We examined DIF salience by comparing the difference between original and DIF-adjusted AD8 scores to the standard error of measurement of the original score.
Results:The two-parameter IRT model fit the data significantly better than the one-parameter model, indicating that some items were more strongly related to underlying everyday functional ability than others. For example, the “problems with judgment” item had higher discrimination (more information) than the “less interest in hobbies/activities” item. There were significant differences in item endorsement by race/ethnicity, education, and survey mode. We found significant uniform and non-uniform DIF on several items across each of these groups. For example, for a given level of functional decline (theta) White participants were more likely to endorse “Daily problems with thinking/memory” than Black and Hispanic participants. The DIF was salient (i.e., caused AD8 scores to change by greater than the standard error of measurement for a large portion of respondents) for those with a college degree and phone respondents.
Conclusions:In a population representative sample of Americans ∼age 57, the items on the AD8 contributed differing levels of discrimination along the range of everyday functioning that is impacted by later life cognitive impairment. This suggests that a simple cut-off or summed score may not be appropriate since some items yield more information about the underlying construct than others. Furthermore, we observed significant and salient DIF on several items by education and survey mode, AD8 scores should not be compared across education groups and assessment modes without adjustment for this measurement bias.
51 Feasibility of Remote Administration of a Modified UDSv3 Cognitive Battery
- Bonnie C Sachs, Stephen R Rapp, Sarah A Gaussoin, Iris Leng, Heather A. Shappell, Mark A Espeland, Eric Fischer, Lauren A. Latham, Benjamin J Williams, James Bateman, Maryjo Cleveland, Mia Yang, Samantha Rogers, Suzanne Craft
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 563-564
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Face-to-face administration is the “gold standard” for both research and clinical cognitive assessments. However, many factors may impede or prevent face-to-face assessments, including distance to clinic, limited mobility, eyesight, or transportation. The COVID19 pandemic further widened gaps in access to care and clinical research participation. Alternatives to face-to-face assessments may provide an opportunity to alleviate the burden caused by both the COVID-19 pandemic and longer standing social inequities. The objectives of this study were to develop and assess the feasibility of a telephone- and video-administered version of the Uniform Data Set (UDS) v3 cognitive batteries for use by NIH-funded Alzheimer’s Disease Research Centers (ADRCs) and other research programs.
Participants and Methods:Ninety-three individuals (M age: 72.8 years; education: 15.6 years; 72% female; 84% White) enrolled in our ADRC were included. Their most recent adjudicated cognitive status was normal cognition (N=44), MCI (N=35), mild dementia (N=11) or other (N=3). They completed portions of the UDSv3 cognitive battery, plus the RAVLT, either by telephone or video-format within approximately 6 months (M:151 days) of their annual in-person visit, where they completed the same in-person cognitive assessments. Some measures were substituted (Oral Trails for TMT; Blind MoCA for MoCA) to allow for phone administration. Participants also answered questions about the pleasantness, difficulty level, and preference for administration mode. Cognitive testers provided ratings of perceived validity of the assessment. Participants’ cognitive status was adjudicated by a group of cognitive experts blinded to most recent inperson cognitive status.
Results:When results from video and phone modalities were combined, the remote assessments were rated as pleasant as the inperson assessment by 74% of participants. 75% rated the level of difficulty completing the remote cognitive assessment the same as the in-person testing. Overall perceived validity of the testing session, determined by cognitive assessors (video = 92%; phone = 87.5%), was good. There was generally good concordance between test scores obtained remotely and in-person (r = .3 -.8; p < .05), regardless of whether they were administered by phone or video, though individual test correlations differed slightly by mode. Substituted measures also generally correlated well, with the exception of TMT-A and OTMT-A (p > .05). Agreement between adjudicated cognitive status obtained remotely and cognitive status based on in-person data was generally high (78%), with slightly better concordance between video/in-person (82%) vs phone/in-person (76%).
Conclusions:This pilot study provided support for the use of telephone- and video-administered cognitive assessments using the UDSv3 among individuals with normal cognitive function and some degree of cognitive impairment. Participants found the experience similarly pleasant and no more difficult than inperson assessment. Test scores obtained remotely correlated well with those obtained in person, with some variability across individual tests. Adjudication of cognitive status did not differ significantly whether it was based on data obtained remotely or in-person. The study was limited by its’ small sample size, large test-retest window, and lack of randomization to test-modality order. Current efforts are underway to more fully validate this battery of tests for remote assessment. Funded by: P30 AG072947 & P30 AG049638-05S1
18 Measurement Invariance of ImPACT in Bilingual and Monolingual High School Athletes
- Hana Kuwabara, Grace Goodwin, Christine Salva, Jessica Woodyatt, Julia Maietta, Staci Ross, Thomas Kinsora, Daniel Allen
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 432-433
-
- Article
-
- You have access Access
- Export citation
-
Objective:
Previous studies have found differences between monolingual and bilingual athletes on ImPACT, the most widely used sport-related concussion (SRC) assessment measure. Most recently, results suggest that monolingual English-Speaking athletes outperformed bilingual English- and Spanish-speaking athletes on Visual Motor Speed and Reaction Time composites. Before further investigation of these differences can occur, measurement invariance of ImPACT must be established to ensure that differences are not attributable to measurement error. The current study aimed to 1) replicate a recently identified four-factor model using cognitive subtest scores of ImPACT on baseline assessments in monolingual English-Speaking athletes and bilingual English- and Spanish-speaking athletes and 2) to establish measurement invariance across groups.
Participants and Methods:Participants included high school athletes who were administered the ImPACT as part of their standard pre-season athletic training protocol in English. Participants were excluded if they had a self-reported history of concussion, Autism, ADHD, learning disability or treatment history of epilepsy/seizures, brain surgery, meningitis, psychiatric disorders, or substance/alcohol use. The final sample included 7,948 monolingual English-speaking athletes and 7,938 bilingual English- and Spanish-speaking athletes with valid baseline assessments. Language variables were based on self-report. As the number of monolingual athletes was substantially larger than the number of bilingual athletes, monolingual athletes were randomly selected from a larger sample to match the bilingual athletes on age, sex, and sport. Confirmatory factor analysis (CFA) was used to test competing models, including one-factor, two-factor, and three-factor models to determine if a recently identified four-factor model (Visual Memory, Visual Reaction Time, Verbal Memory, Working Memory) provided the best fit of the data. Eighteen subtest scores from ImPACT were used in the CFAs. Through increasingly restrictive multigroup CFAs (MGCFA), configural, metric, scalar, and residual levels of invariance were assessed by language group.
Results:CFA indicated that the four-factor model provided the best fit in the monolingual and bilingual samples compared to competing models. However, some goodness-of-fit-statistics were below recommended cutoffs, and thus, post-hoc model modifications were made on a theoretical basis and by examination of modification indices. The modified four-factor model had adequate to superior fit and met criteria for all goodness-of-fit indices and was retained as the configural model to test measurement invariance across language groups. MGCFA revealed that residual invariance, the strictest level of invariance, was achieved across groups.
Conclusions:This study provides support for a modified four-factor model as estimating the latent structure of ImPACT cognitive scores in monolingual English-speaking and bilingual English- and Spanish-speaking high school athletes at baseline assessment. Results further suggest that differences between monolingual English-speaking and bilingual English- and Spanish-speaking athletes reported in prior ImPACT studies are not caused by measurement error. The reason for these differences remains unclear but are consistent with other studies suggesting monolingual advantages. Given the increase in bilingual individuals in the United States, and among high school athletics, future research should investigate other sources of error such as item bias and predictive validity to further understand if group differences reflect real differences between these athletes.
100 Hybrid Neuropsychology in practice: Preliminary data and future directions
- Shifali Singh, Laura Germine
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 773-774
-
- Article
-
- You have access Access
- Export citation
-
Objective:
The dynamic transitions towards and away from digitizing mental health care services have put immense pressure on neuropsychology to develop a more technologically advanced approach to patient care. Consistent with this, neuropsychologists are faced with an increasing burden to rapidly learn novel techniques of administering digital and/or remote cognitive measures, which may not be sufficiently standardized and/or psychometrically validated.
Participants and Methods:The Hybrid Neuropsychology model, which aims to provide a framework for the integration of digital tools in neuropsychological assessment, has been incorporated routinely in our clinical practice, and among diverse clinical populations in inpatient and outpatient settings.
Results:This talk aims to 1) provide a brief background on Hybrid Neuropsychology; 2) discuss preliminary findings from our ongoing studies implementing this approach in clinical populations; and 3) offer key insights gleaned from methodological, analytical, and practical perspectives.
Conclusions:Future work towards building a more integrative method of data collection and storage based on extant preliminary data will also be discussed.