Impact statement
-
- The MoCA appears to be a useful tool to screen for cognitive difficulties in this South African population.
-
- That said, some cultural adaptation is needed, and demographics factors such as age, sex, language and education should be considered, to improve the ability of the MoCA to identify MCI.
-
- Until a culturally adapted version of the MoCA has been developed for this population, we suggest that a cut-off score of 25/30 is used for the English version and 23 or 24/30 is used for the Afrikaans version, to reduce incorrectly identifying cognitive difficulties.
Background
Routine screening and monitoring of cognitive function is critical to optimal clinical management of patients across disciplines (Dolansky et al., Reference Dolansky, Hawkins, Schaefer, Sattar, Gunstad, Redle, Josephson, Moore and Hughes2016; Cho et al., Reference Cho, Shin, Chang, Lee, Jeong, Kim, Yun and Son2018; Hagi et al., Reference Hagi, Nosaka, Dickinson, Lindenmayer, Lee, Friedman, Boyer, Han, Abdul-Rashid and Correll2021; Zhou et al., Reference Zhou, Yu, Luo, Xie, Wang and Wan2021). Undetected cognitive impairment can impair treatment outcomes such as therapeutic receptiveness (e.g., ability to engage in psychotherapeutic processes or pharmacological treatment adherence) (Knight et al., Reference Knight, Mills and Baune2019; Sachs et al., Reference Sachs, Berg, Jagsch, Lenz and Erfurth2020; Wu et al., Reference Wu, Yu, Li, Chen and Wang2023). Additionally, if left undetected and therefore untreated, individuals presenting with cognitive decline can progressively deteriorate, significantly impacting their activities of daily living (ADLs) and resulting in an overall poorer quality of life (Hill et al., Reference Hill, McDermott, Mogle, Munoz, Depasquale, Wion and Whitaker2017).
Depending on the severity of cognitive impairment and the degree of functional impairment present, a diagnosis of a mild or major neurocognitive disorder (NCD) may be warranted (American Psychiatric Association [APA], 2013). While both mild and major NCD indicate a decline from premorbid cognitive functioning, major NCD, also known as dementia, requires significant impairment in one or more of the principal cognitive domains (complex attention, executive function, learning and memory, language, visuospatial and social cognition). These impairments represent a decline from a previous level of functioning of sufficient severity to interfere with ADLs (APA, 2013). Conditions such as traumatic brain injury (TBI) or advanced human immunodeficiency virus (HIV) can result in NCDs (APA, 2013). Other etiologies of NCDs include Alzheimer’s disease, vascular pathology, Lewy body dementia and frontotemporal lobar degeneration. Ageing remains the most significant risk factor for NCDs, and considering the growing elderly population globally and in South Africa, the prevalence of age-related diseases, such as NCDs, is likely to rise (Martin Prince et al., Reference Martin Prince, Wimo, Guerchet, Gemma-Claire Ali, Y-T, Prina, Yee Chan and Xia2015; World Health Organization (WHO), 2024). Once diagnosed, most major NCDs are usually irreversible, relatively treatment-resistant, and greater psychosocial and healthcare demands need to be met in order for management to be effective (Rasmussen and Langerman, Reference Rasmussen and Langerman2019). Due to resource constraints faced by many low to middle-income countries (LMICs), such as South Africa, effective management is challenging (Docrat et al., Reference Docrat, Besada, Cleary, Daviaud and Lund2019; Shisana et al., Reference Shisana, Stein, Zungu and Wolvaardt2024).
Compared to major NCD, mild NCD, also known as mild cognitive impairment (MCI), entails more subtle and modest concerns about cognitive decline. This decline typically presents without significant interference with ADLs, such that the person can still function independently (APA, 2013). Therefore, mild NCD represents an intermediate state between normal age-related cognitive decline and major NCDs, representing a critical intervention period (Salzman et al., Reference Salzman, Sarquis-Adamson, Son, Montero-Odasso and Fraser2022) While mild NCD is a risk factor itself for developing a major NCD, such as Alzheimer’s dementia there are other modifiable factors which increase the risk of the progression to a major NCD (Sabbagh et al., Reference Sabbagh, Boada, Borson, Chilukuri, Dubois, Ingram, Iwata, Porsteinsson, Possin, Rabinovici, Vellas, Chao, Vergallo and Hampel2020; Nezhadmoghadam et al., Reference Nezhadmoghadam, Martinez-Torteya, Treviño, Martínez, Santos and Tamez-Peña2021; Wolfova et al., Reference Wolfova, Kucera and Cermakova2021). Some of these modifiable factors include cardiometabolic (Lu et al., Reference Lu, Fülöp, Gwee, Lee, Lim, Chong, Yap, Yap, Pan and Ng2022), treatment adverse effects (Breijyeh and Karaman, Reference Breijyeh and Karaman2020; Franzoi et al., Reference Franzoi, Agostinetto, Perachino, Del Mastro, de Azambuja, Vaz-Luis, Partridge and Lambertini2021), vitamin deficiencies such as vitamin B12 or folate (Zhang et al., Reference Zhang, Luo, Yuan and Ding2020), psychiatric conditions such as severe depression (Varghese et al., Reference Varghese, Frey, Schneider, Kapczinski and de Azevedo Cardoso2022) and suboptimally treated infections such as HIV and syphilis (Hernandez-Ruiz et al., Reference Hernandez-Ruiz, Letenneur, Fülöp, Helmer, Roubaud-Baudron, Avila-Funes and Amieva2022). These may be addressed by dietary and lifestyle changes or appropriate pharmacological treatment, resulting in improved cognition, greater quality of life and notably decreased psychosocial and healthcare resource needs (Zhang et al., Reference Zhang, Xu, Zhang, Wang, Ou, Qu, Shen, Chen, Wu, Zhao, Zhang, Sun, Dong, Tan, Feng, Zhang, Evangelou, Smith and Yu2022). This is particularly relevant to South Africa, given the high prevalence of these potentially modifiable risk factors in South Africa accompanied by significant resource constraints (Docrat et al., Reference Docrat, Besada, Cleary, Daviaud and Lund2019; Alkhatib et al., Reference Alkhatib, Nnyanzi, Mujuni, Amanya and Ibingira2021; Greene et al., Reference Greene, Yangchen, Lehner, Sullivan, Pato, McIntosh, Walters, Gouveia, Msefula, Fumo, Sheikh, Stockton, Wainberg and Weissman2021; Monyeki et al., Reference Monyeki, Mkhatshwa, Thulare, Kemper, Kengne and Moselakgomo2023 ; Cassambai et al., Reference Cassambai, Tetteh, Highton, Kunutsor, Darko, Jeffers, Ikhile, Agot, Olenja, Njoroge, Jessen, Abdala, Senior, Coleman, Khunti, Godia, Alfred, Lamptey, Buabeng, Damasceno and Seidu2024 ; Malan et al., Reference Malan, Zandberg, Visser, Wicks, Kruger and Faber2024). Considering this high prevalence of modifiable risk factors in South Africa, early detection of cognitive decline is a fundamental first step in initiating early intervention, specifically at the community-based level within the public healthcare (PHC) system (Sabbagh et al., Reference Sabbagh, Boada, Borson, Chilukuri, Dubois, Ingram, Iwata, Porsteinsson, Possin, Rabinovici, Vellas, Chao, Vergallo and Hampel2020). However, owing to the subtle changes in mild NCD and resource constraints, it can remain undetected. This necessitates identifying available, suitable, efficient and reliable cognitive screening tools that are sensitive, appropriate, and easily administered (de Villiers, Reference de Villiers2021; Shisana et al., Reference Shisana, Stein, Zungu and Wolvaardt2024).
Several cognitive bedside administered screening tools are available, each with advantages and limitations (Zhuang et al., Reference Zhuang, Yang and Gao2021). The Montreal Cognitive Assessment (MoCA), developed and validated by Nasreddine et al. (Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005), is a freely available one-page, 30-item test typically administered within 10–15 min (www.mocatest.org). While freely available, the MoCA developers encourage virtual training and certification to ensure correct administration. The MoCA evaluates eight cognitive domains: executive functions, visuospatial abilities, short-term and delayed verbal memory, language, attention, concentration, working memory and temporal and spatial orientation. This original MoCA paper version, the MoCA Full, was validated in an English/French-speaking older Canadian sample of adults with normal cognition, mild NCD and Alzheimer’s disease. Using a cut-off score of ≥ 26/30, with an education correction of one point for individuals with ≤ 12 years of education, the sensitivity of the MoCA for identifying mild NCD was 90% and the specificity was 87% (Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005). The original face-to-face administered MoCA Full is now available in over 100 languages, including Afrikaans, isiXhosa, and Zulu, and has advanced in terms of administration modes (e.g. electronic or audio-visual version); sensory adaptation (e.g. MoCA-Blind), and multiple versions to monitor cognitive changes.
Since its release, several countries have evaluated the reliability and validity of the MoCA Full (Freitas et al., Reference Freitas, Simões, Alves and Santana2011; Narazaki et al., Reference Narazaki, Nofuji, Honda, Matsuo, Yonemoto and Kumagai2012; Yu et al., Reference Yu, Li and Huang2012; Memõria et al., Reference Memõria, Yassuda, Nakano and Forlenza2013; Kirkbride et al., Reference Kirkbride, Ferreira-Correia and Sibandze2022; Geller and Slicer, Reference Geller and Slicer2024; Lau et al., Reference Lau, Lin, Lin, Li, Yao, Lin and Wu2024). While the MoCA has shown good reliability and validity in screening for cognitive impairment in some settings limitations and item level and cut-off score modifications have been suggested (Freitas et al., Reference Freitas, Simões, Alves and Santana2011; Narazaki et al., Reference Narazaki, Nofuji, Honda, Matsuo, Yonemoto and Kumagai2012; Yu et al., Reference Yu, Li and Huang2012; Memõria et al., Reference Memõria, Yassuda, Nakano and Forlenza2013). Item-level changes were motivated by cultural sensitivity, recommending replacing foreign animals with more familiar indigenous animals (language domain), and replacing words in the delayed recall test with words from participants’ cultural background (e.g., velvet with silk in the case of the Chinese population) (Freitas et al., Reference Freitas, Simões, Alves and Santana2011; Narazaki et al., Reference Narazaki, Nofuji, Honda, Matsuo, Yonemoto and Kumagai2012; Yu et al., Reference Yu, Li and Huang2012; Memõria et al., Reference Memõria, Yassuda, Nakano and Forlenza2013). Accompanied by cut-off score modification, the reliability and validity of the MoCA in detecting mild NCD is supported in some African countries (Daniel et al., Reference Daniel, Agenagnew, Workicho and Abera2022), including South Africa (Rademeyer and Joubert, Reference Rademeyer and Joubert2016; Thungana, Reference Thungana2022; Van Wijk et al., Reference Van Wijk, Meintjes, Muller, Van Wijk, Wijk, Meintjes and Muller2024). At the same time, other South African-based researchers have questioned the reliability and validity of the test in our setting? (Robbins et al., Reference Robbins, Joska, Thomas, Stein, Linda, Mellins and Remien2013 ; Hakkers et al., Reference Hakkers, Beunders, Ensing, Barth, Boelema, Devillé, Tempelman, Coutinho, Hoepelman, Arends and van Zandvoort2018 ; Kirkbride et al., Reference Kirkbride, Ferreira-Correia and Sibandze2022).
Another cognitive screening tool is the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), which is increasingly used and has been extensively researched (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998; Pandya, Reference Pandya2020; Suliman et al., Reference Suliman, van den Heuvel, Kilian, Bröcker, Asmal, Emsley and Seedat2021 ; Ikanga et al., Reference Ikanga, Patrick, Schwinne, Patel, Epenge, Gikelekele, Tshengele, Kavugho, Mampunza, Yarasheski, Teunissen, Stringer, Levey, Rojas, Chan, Lario Lago, Kramer, Boxer, Jeromin, Alonso and Spencer2024). Originally developed as a brief neuropsychological screening battery for NCDs in older adults, the RBANS has normative data for ages 20–89 and requires approximately 20–30 min administration time (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998). Similar to the MoCA, the RBANS has multiple versions (Form A–D), making it helpful in monitoring cognitive changes over time (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998). However, relative to the MoCA, the RBANS offers a more in-depth assessment which may improve reliability and validly in detecting cognitive impairment (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998; Paul et al., Reference Paul, Lane, Tate, Heaps, Romo, Akbudak, Niehoff and Conturo2011; Shaughnessy et al., Reference Shaughnessy, Rucker and Sanchez2019). The RBANS comprises twelve sub-tests combined to form five index scores and a total score. These five indexes are immediate verbal memory, visuospatial/constructional, language, attention and delayed memory (including verbal and visuospatial). The RBANS index scores are converted to descriptive performance classifications: exceptionally high, above average, high average, average, low average, below average and extremely low (Guilmette et al., Reference Guilmette, Sweet, Hebben, Koltai, Mahone, Spiegler, Stucky and Westerveld2020). Since the publication of the RBANS in 1998, multiple studies have evaluated the reliability, validity, and clinical utility of the tool (De La Torre et al., Reference De La Torre, Suárez-Llorens, Caballero, Ramallo, Randolph, Lleó, Sala and Sánchez2014; Thaler et al., Reference Thaler, Hill, Duff, Mold and Scott2015). Initially available in English and Spanish, the RBANS has been translated into over 40 languages and demonstrated good sensitivity and specificity in identifying patients with Alzheimer’s disease and mild NCD (Karantzoulis et al., Reference Karantzoulis, Novitski, Gold and Randolph2013). Although initially developed for the evaluation of NCD due to Alzheimer’s disease, the RBANS has been used to assess a wide variety of clinical populations, including HIV-associated NCD (Bucher et al., Reference Bucher, Leonhard and Michael Bradley2022), TBI (Arch and Ferraro, Reference Arch and Ferraro2021), depression (Faust et al., Reference Faust, Nelson, Sarapas and Pliskin2017), and Parkinson’s disease (Yang et al., Reference Yang, Garrett-Mayer, Schneider, Gollomp and Tilley2009) among others (Shaughnessy et al., Reference Shaughnessy, Rucker and Sanchez2019). A study that assessed for cross-cultural systematic differences on the RBANS reported no systematic cultural/linguistic bias that would require adjustments to the translations, which, given some of the raised concerns about the MoCA, supports the RBANS as an appealing option (Weber et al., Reference Weber, Randolph and Negash2019).
Considering significant resource constraints and conflicting findings about the reliability and validity of the MoCA in our setting, the purpose of the current study was to evaluate the concordance between the MoCA (Version 7.1) and the RBANS (Version A) as cognitive screening tools to detect mild NCD in a South African adult community sample. Results of this study can inform whether a bedside test, such as the MoCA, can be used in place of longer neurocognitive batteries in our setting. Second, we aimed to contribute to the growing body of literature on the MoCA in South Africa by generating data to further inform the optimal cut-off score to detect cognitive impairment in our sample and setting. We also aimed to evaluate whether demographic factors such as sex and age influence performance and should be considered when interpreting MoCA performance in our setting. This can inform ‘a best approach’ when using and interpreting these available tools in our setting and whether further linguistic and cultural adaptation is needed. To the best of our knowledge, this is the first study to directly evaluate and compare the MoCA and RBANS in a South African setting.
Methods
Study design and setting
The current study was a cross-sectional observational study nested in the ’Understanding the Shared Roots of Neuropsychiatric Disorders (NPD) and Modifiable Risk Factors for Cardiovascular Disease’ (Shared Roots) project. The Shared Roots project was conducted in Cape Town, Western Cape, South Africa (Health Research Ethics Committee [HREC] at Stellenbosch University: N13/08/115). Participants were recruited through purposive sampling using community newspaper advertisements and flyers and were drawn from the dominant ethnic group (mixed ancestry) in this geographical region. The project aimed to investigate contributing factors to comorbidity in neuropsychiatric disorders (NPDs) and metabolic syndrome (MetS). Its aim was investigated in three NPD cohorts: posttraumatic stress disorder (PTSD), schizophrenia, and Parkinson’s disease (PD). The Shared Roots project was a cross-sectional matched case–control study (van den Heuvel et al., Reference van den Heuvel, Stalder, du Plessis, Suliman, Kirschbaum and Seedat2020), and the current study was conducted in the control cohort.
Participants
Shared Roots’ participants were adults (≥ 18 years old) who could read and understand the written informed consent forms in English or Afrikaans, which are the main languages spoken within the Western Cape as well as within the mixed ancestry population (Savedra et al., Reference Savedra, Rosenberg, Macedo and Macedo2021). Based on a clinical history and diagnostic interview, participants were excluded from the control cohort if they (i) had a neurological disorder, (ii) had major current psychiatric disorders including current psychiatric medication use, (iii) had a major medical illness (e.g. epilepsy, stroke, cancer or chronic infections such HIV), (iv) were known with intellectual disability or significant head injury resulting in loss of consciousness, or (v) had a diagnosis of a major neurocognitive disorder. As a result, the sample consisted of generally healthy adults without serious psychiatric or medical morbidity, which might affect cognitive performance.
Procedures
Shared Roots’ participants were assessed for the presence of any psychiatric disorder with a clinician-administered diagnostic interview— the MINI International Neuropsychiatric Interview (Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavas, Weiller, Hergueta and Dunbar1998). Participants also underwent metabolic syndrome screening, neurocognitive tests, blood sampling (e.g. for genomic analyses) and neuroimaging assessments. Study procedures, including the MoCA and RBANS administration, were conducted in English/Afrikaans based on participant preference. The English and Afrikaans MoCA versions were obtained from the website, https://mocacognition.com/, and the English and Afrikaans RBANS source documents, adapted for the South African population with minor changes to the List Recall and Story Memory subtest wording, from the developer. The study team included a psychiatrist, a physician experienced in psychiatry, two psychologists and research nurses. Participants were reimbursed for their travel costs.
Statistical analysis
Statistical analyses were undertaken using Statistica (version 13) and SPSS (version 29). All tests were 2-sided, and statistical significance was set at p < 0.05. Descriptive statistics were computed for MoCA total and domain scores. Criterion validity for global cognition and domain scores between the MoCA and RBANS was assessed using Pearson’s correlation tests. Pearson’s correlation coefficients were also used to determine associations between age and education, and MoCA scores. ANOVAs were used to establish the association between MoCA scores and categorical variables (e.g., sex). The internal consistency of the MoCA was derived using Cronbach’s alpha. Regression analysis was used to create age and education-adjusted z-scores for both the MoCA and RBANS, which were converted to standard scores.
The MoCA consists of 28 items, 27 of which are scored out of 1 and one item (serial 7), which is scored out of 3. Firstly, the serial 7 s item scores were transformed in SPSS to a score out of 1 by dividing the existing scores by 3, thus allowing us to compare all the scores out of 1. Second, a composite mean score was created for all the items (scored out of 1) combined; the mean for this composite score was 0.805 (SD 0.098) for the English and 0.769 (SD 0.109) for the Afrikaans samples, respectively. Items that participants performed poorly in were considered items with a score of more than 2 SD below the mean for the composite score (i.e. less than 0.61 for the English sample and less than 0.55 for the Afrikaans sample).
A Bland–Altman plot was derived to compare the agreement between the MoCA and RBANS and to assess for bias. The Bland Altman plot used raw scores and is a plot on the X axis of the mean of the 2 measures (RBANS/MoCA) taken for each participant, with the Y axis representing the arithmetical difference between the two measures.
Receiver operator characteristics (ROC) analysis was undertaken to assess whether MoCA total scores predicted mild NCD according to the RBANS. Mild NCD was defined as one standard deviation (1SD) below the mean of the standardized score (a score of 85 or less) (Duff et al., Reference Duff, Hobson, Beglinger and O’Bryant2010). We assessed the sensitivity and specificity of the two scales by using recommended cut-off scores of ≤ 26/30 for the MoCA) (Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005) and ≤ 85 (1SD below the mean) for the RBANS (Duff et al., Reference Duff, Hobson, Beglinger and O’Bryant2010).
We report cut-off scores for optimal sensitivity and specificity based on the data. Area under the curve (AUC) was used to compare the diagnostic performance between the MoCA and the RBANS. Finally, multiple regression analysis was performed with the MoCA score as the dependent variable and age, sex, and years of education as the independent variables.
Results
Sample characteristics
The final sample (N = 370) was primarily female (70.8%), with a mean age of 45.96 years (SD:15.07; range = 18–81 years). The majority were Afrikaans first language speaking (73.8%) and had completed secondary school (79.5%) with a mean of 11.06 ± 2.72 years of education (education range = 4–25). When stratified by language, Afrikaans participants were significantly older (p ≤ 0.001) and had lower education levels (p ≤ 0.001) (see Table 1).
Participant characteristics stratified by language
Note: In view of missing data, percentages may not add up to 100%.
MoCA performance
The MoCA showed acceptable internal consistency in both the English and Afrikaans versions (Cronbach alpha = 0.582 and 0.694, respectively). The variables ’orientation to place’ and ’orientation to city’, however, showed zero variance and, as a result, were removed from the analysis.
Age was significantly correlated with the MoCA total score, with older participants performing worse (r = −0.203, p ≤ 0.001). Female participants also performed worse (F = 18.37, p ≤ 0.001), as well as participants with fewer years of education (r = 0.326, p ≤ 0.001). Gender was associated with education with female participants having lower education levels (F(1) = 16.6, p ≤ 0.001. Age was moderately correlated with total years of education (r = −0.434; p ≤ 0.001).
Below-average (2SD below the mean) scores were observed on a number of MoCA items: ‘Alternate Trail Making’, ’Cube copying’, Language: Sentence 2 Repetition ’Verbal fluency’, ’Abstraction: watch-ruler’, ’Recall: face, church, daisy’. These can be seen in Table 2.
MoCA item scores
Note: For above items minimum was 0 and maximum was 1.
All of the 28 MoCA items are scored out of 1 except for Serial 7 which is scored out of 3. Serial 7 results were divided by 3 to produce a score out of 1 that other items could be compared to.
A composite score for all the items was then created; the mean for this composite score was 0.805 (SD 0.098) for the English and 0.769 (SD 0.109) for the Afrikaans samples respectively. Items that participants performed poorly in were considered items with a score of more than 2SD below the mean for the composite score (i.e., less than 0.61 for the English sample and 0.55 for the Afrikaans sample).
Concordance between MoCA and RBANS in evaluating mild NCD
There was a moderate correlation between MoCA and RBANS total scores (r = 0.615; p ≤ 0.001; Eng: r = 0.510, p ≤ 0.001; Afr: r = 0.639, p ≤ 0.001), indicating acceptable criterion-related validity. Correlations were also run on MoCA and RBANS domain scores with MoCA visuo-executive and RBANS visuospatial showing a moderate correlation (r = 0.511, p ≤ 0.001; Eng: r = 0.408; p ≤ 0.001; Afr: r = 0.559, p ≤ 0.001), RBANS delayed memory and MoCA delayed recall a moderate correlation (r = 0.431, p ≤ 0.001; Eng: r = 0.416, p ≤ 0.001; Afr: r = 0.421, p ≤ 0.001) and MoCA and RBANS attention (r = 0.281, p = p ≤ 0.001; Eng: r = 0.312, p ≤ 0.001; Afr: r = 0.269, p ≤ 0.001) and MoCA and RBANS language (r = 0.235, p ≤ 0.001; Eng: r = 0.282; p = 0.04; Afr: r = 0.223, p ≤ 0.001) showing weak correlations.
The Bland–Altman Plot (see Figures 1 and 2 ) indicated good agreement between the MoCA and RBANS and no proportional bias between the two tests. Random scatter around the zero-difference lines and the correlation coefficient between the differences and the averages were not statistically significant (Eng: t = −0.902, p = 0.369; Afr: t = −0.056, p = 0.956).
(a) Bland–Altman Plot – English. (b) Bland–Altman Plot – Afrikaans.
(a) ROC Curve: MoCA compared to RBANS – English. (b) ROC Curve: MoCA compared to RBANS – Afrikaans.
The mean score on the MoCA was 23.41 (SD = 3.23, range 8–30) and the mean raw score on the RBANS 200.37 (SD = 28.17, range 73–289). Previous studies have shown good sensitivity and specificity for RBANS scaled scores in predicting mild NCD at 1SD below a mean of 100 (i.e., at a score of below 85) (Duff et al., Reference Duff, Hobson, Beglinger and O’Bryant2010). As such, we transformed the raw scores to scaled scores and used this as our cut-off score for mild NCD. One hundred-and-twenty participants (32.4%) scored both ≥ 85 on the RBANS and ≥ 26 on the MoCA, and 45 (12.2%) participants scored below both the MoCA and RBANS cut-off points. One hundred and ninety-six participants (53%) scored ≥ 85 on the RBANS and < 26 on the MoCA. Seven participants (2%) scored < 85 on the RBANS and ≥ 26 on the MoCA.
The ROC curves (see Figures 2a,b) demonstrated that the performance of the MoCA for predicting cognitive impairment compared to the RBANS was fair. The AUC for the English sample was 0.711 (95%CI: 0.547, 0.876; p = 0.022). For the Afrikaans sample, the AUC was 0.782 (95%CI: 0.703, 0.861; p = 0.001).
Using the recommended cut-off score of 26/30, the MoCA showed high sensitivity (Eng: 81.8%; Afr: 87.8%) but low specificity (Eng: 43.6%; Afr: 35.0%). In the present study, the cut-off score for optimal sensitivity and specificity to detect mild NCD was 25 for the English sample. At this cut-off, the sensitivity remained at 81.1% and the specificity increased to 57.4%. The optimal cut-off for the Afrikaans sample was between 23 and 24. The sensitivity and specificity were 68.3% and 75.9%%, respectively, for a MoCA cut-off of 23 and 78.0% and 64.1%, for a MoCA cut-off of 24 (see Table 3).
Summary of different MoCA cut-off scores predicting MCI on the RBANS
Discussion
We set out to determine the concordance between two cognitive screening tools – the MoCA and the RBANS – and to identify the optimal cut-off score for the MoCA to detect mild NCD in our sample. Good concordance would suggest that the briefer MoCA can be substituted for longer neurocognitive assessments such as the RBANS, saving time and resources. We also hoped to evaluate whether demographic factors influenced performance in our sample. This will assist in identifying whether adapted norms are required. Additionally, we hoped to contribute to the growing body of literature on the use of the MoCA in South Africa to further inform a ‘best approach’ when using and interpreting available tools in the linguistically, culturally, educationally and economically diverse South African setting.
We first evaluated psychometric properties of the MoCA in our sample and found acceptable internal consistency and good criterion-related validity. This aligns with a South African study, and the few African studies that have measured internal consistency in the MoCA (Masika et al., Reference Masika, Yu and Li2021 ; Kirkbride et al., Reference Kirkbride, Ferreira-Correia and Sibandze2022 ; Daniel et al., Reference Daniel, Agenagnew, Workicho and Abera2022). It differs, however, from the only other published South African study, which reported low reliability (Van Wijk et al., Reference Van Wijk, Meintjes, Muller, Van Wijk, Wijk, Meintjes and Muller2024). This may be since the Van Wijk study utilized a different measure of internal consistency to the other studies. Given the dearth of reliability and validity data on the MoCA for the South African population, these are important findings.
Overall, there was concordance between the MoCA and the RBANS, suggesting fair reliability of the MoCA in identifying mild NCD in our sample, albeit using lower cut-off scores for detection (≤ 25/30 when using the English version and ≤ 23 or 24/30 when using the Afrikaans version). Thus, our findings support previous South African-based research of cut-off score modification to improve MoCA reliability and validity in our context (Rademeyer and Joubert, Reference Rademeyer and Joubert2016; Thungana, Reference Thungana2022; Van Wijk et al., Reference Van Wijk, Meintjes, Muller, Van Wijk, Wijk, Meintjes and Muller2024). Similar to previous research, analysis of MoCA domain and item-level scores suggests that some items may need modification (Freitas et al., Reference Freitas, Simões, Alves and Santana2011; Narazaki et al., Reference Narazaki, Nofuji, Honda, Matsuo, Yonemoto and Kumagai2012; Yu et al., Reference Yu, Li and Huang2012 ; Memõria et al., Reference Memõria, Yassuda, Nakano and Forlenza2013). MoCA language and recall domains showed a weak correlation with corresponding RBANS domains. These findings could potentially be accounted for by cultural variations in the use of language between our sample and the original cultural/language groups which the MoCA was validated. Since the sample was largely Afrikaans-speaking, our findings suggest that further adaptations may be warranted to the currently available Afrikaans translation of the MoCA as well as the original English version.
Regarding individual items, participants scored significantly below average on abstraction (watch-ruler), cube copying and alternate trail making, as well as verbal fluency and recall. The relatively lower level of education (mean education years = 11.05 ± 2.72) of our sample compared with the original validation sample, and the ‘outlier’ items above (which may reflect cultural differences), may, to some extent, account for the inconsistent findings. A study that investigated the discriminant validity of the MoCA in South African samples with HIV and psychiatric/neurocognitive disorders compared to controls found similarly low scores in the entire sample, and a mean of 22 in the healthy control group (Kirkbride et al., Reference Kirkbride, Ferreira-Correia and Sibandze2022). It is possible that linguistic tests that are more culturally appropriate and visuospatial and abstraction items that are simpler and more in line with the average level of education of the South African population would fare better at detecting cognitive difficulties.
The MoCA had good sensitivity in detecting mild NCD at the recommended cut-off score of ≥ 26/30. However, at this cut-off score, the specificity was very low, making it likely that many participants would receive a false positive diagnosis of mild NCD. The optimal MoCA cut-off scores for identifying mild NCD were ≤ 25/30 for the English sample and ≤ 23 or 24/30 for the Afrikaans sample. At these scores, sensitivity decreased while specificity increased. Considering these findings, we would suggest lowering the MoCA cut-off score in this population to ≤ 25/30 and ≤ 23 or 24/30 for the English and Afrikaans samples, respectively. A number of other researchers, including those in other African settings, have similarly suggested that the original cut-off score of 26/30 be reduced, in order to reduce misclassification of individuals from different cultures and contexts (Conti et al., Reference Conti, Bonazzi, Laiacona, Masina and Coralli2015; Wong et al., Reference Wong, Law, Liu, Wang, Lo, Lau, Wong and Mok2015 ; Pinto et al., Reference Pinto, Machado, Bulgacov, Rodrigues-Júnior, Costa, Ximenes and Sougey2019 ; Masika et al., Reference Masika, Yu and Li2021 ; Daniel et al., Reference Daniel, Agenagnew, Workicho and Abera2022).
Demographic variables that demonstrated a significant negative correlation with MoCA scores included age, while years of education showed a positive correlation. This is consistent with other literature on the MoCA (Malek-Ahmadi et al., Reference Malek-Ahmadi, Powell, Belden, Oconnor, Evans, Coon and Nieri2015; Pinto et al., Reference Pinto, Machado, Bulgacov, Rodrigues-Júnior, Costa, Ximenes and Sougey2019 ; Elkana et al., Reference Elkana, Tal, Oren, Soffer and Ash2020 ; Kirkbride et al., Reference Kirkbride, Ferreira-Correia and Sibandze2022 ; Daniel et al., Reference Daniel, Agenagnew, Workicho and Abera2022). Although female sex showed a negative association with MoCA scores, this needs to be interpreted with caution as the sample comprised a disproportionate number of females. Females also had significantly lower levels of education, which may explain the lower MoCA scores. The effect of sex on the MoCA is not as clear in the literature, with some studies reporting significant differences in performance between males and females, whereas others do not (Lu et al., Reference Lu, Li, Li, Zhou, Wang, Zuo, Jia, Song and Jia2011; Robbins et al., Reference Robbins, Joska, Thomas, Stein, Linda, Mellins and Remien2013; Kaya et al., Reference Kaya, Aki, Can, Derle, Kibarolu and Barak2014; Santangelo et al., Reference Santangelo, Siciliano, Pedone, Vitale, Falco, Bisogno, Siano, Barone, Grossi, Santangelo and Trojano2015). There was also a significant negative correlation between age and total years of education, which may, at least to some degree, account for the correlation between age and lower MoCA total scores.
Limitations
This study had several limitations. First, while we used the RBANS, with adaptations, as our comparator screening tool for mild NCD, it has not been validated in the South African population. The importance of validating both the MoCA and RBANS in the population in which they are being administered cannot be overemphasized and is demonstrated by the poor specificity of the English version of the MoCA in this sample. Validation is the optimal way in which appropriate cut-off scores for positive and negative classification of cognitive impairment can be established. That said, to our knowledge, no screening tool for mild NCD has to date been validated in this population. Additionally, neither the MoCA nor the RBANS was culturally adapted for our population. While this does allow us to compare the psychometric performance with populations in other countries, it may also be a source of bias in the sample (Robbins et al., Reference Robbins, Joska, Thomas, Stein, Linda, Mellins and Remien2013). Second, the Afrikaans version of the MoCA was used for patients with Afrikaans as their first language; however, the Afrikaans version has not been standardized. Third, a few factors limit generalizability. Our population comprised individuals of mixed race (Colored ethnicity) and a disproportionate number of females (70.8%). Additionally, 79.5% of the sample completed high school, a number more than double that found nationally (37.3% in 2022) (Statistics South Africa, 2024). The mean age of participants was below 50 years, and older participants generally had lower educational levels compared to younger individuals. Consequently, the findings may not be generalizable to typical patients with cognitive impairment, which predominantly manifests after the age of 65.
Fourth, although we grouped participants according to years of education, we were unable to control for the quality of education. Given that many of the older participants were likely to have been educated during a period of South African history of structural inequality where there was a high degree of variation in the quality of education on the basis of race, this may have influenced performance on these measures. The effects of education on cognitive screening have been widely reported in the literature, to the extent that studies on the MoCA have, in the past, excluded illiterate individuals (Freitas et al., Reference Freitas, Simões, Alves and Santana2011 ) due to concerns that this would impact global scoring. Other studies have recommended changes to items (Yu et al., Reference Yu, Li and Huang2012 ; Hu et al., Reference Hu, Zhou, Hu, Huang, Wei, Qi, Wen and Xu2013 ) to accommodate for the lower level of education in their country compared to the level of education in the original Canadian MoCA validation sample (Nasreddine et al., Reference Nasreddine, Phillips, Bédirian, Charbonneau, Whitehead, Collin, Cummings and Chertkow2005 ). The wide range of education in our sample is likely a representation of the South African population (OECD, 2019), and as such, we followed the latter approach and identified items that participants performed poorly at. If these are also identified in other local studies, it may be prudent to remove or modify them.
Conclusion
To the best of our knowledge, this is the first study to directly evaluate and compare the MoCA and RBANS in a South African setting. While the MoCA, because of its brevity, may be a useful and time-saving screener for mild NCD in this population, our findings suggest that some modification is required for certain domains and items to improve the identification of mild NCD. Until such time that a culturally adapted version of the MoCA has been developed and validated for a population matching our sample, we suggest lowering the cut-off score of the MoCA from 26 to 25 in the English sample, and to 23 or 24 in the Afrikaans sample, in an effort to reduce the false positive detection rate of mild NCD. In the interim, hopefully our findings can contribute to using an informed approach when using and interpreting the MoCA in our varied and resource-constrained settings.
We recommend replication studies comparing the performance of the MoCA compared to the RBANS in other South African ethnic groups and languages to determine if this cut-off can be reproduced. Screening for mild NCD with the MoCA does not, however, replace a more comprehensive neurocognitive assessment. Nonetheless, this study paves the way for future studies of psychometric analysis of the MoCA that focus on scale validity (i.e., exploratory, and confirmatory factor analysis) and test–retest reliability. In addition, validating the MoCA against a gold standard comprehensive cognitive battery will be an important next step in our context.
Open peer review
To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2025.10050.
Author contribution
SSu: Conceptualisation, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing - review & editing; EB: Conceptualisation, Data curation, Investigation, Methodolgy, Validation, Visualisation, Writing - original draft, Writing - review & editing; NB: Formal analysis, Writing - original draft; LvdH: Conceptualization, Data curation, Investigation, Methodology, Project administration, Validation, Visualization, Writing - review & editing; LA: Investigation, review & editing; SK: Investigation, review & editing; RE: Investigation, review & editing; JC: Conceptualization, Funding acquisition, Methodology, Supervision, Writing - review & editing; SSe: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing - review & editing; All authors have read and approved the final version.
Financial support
Research reported in this publication was supported by the South African Medical Research Council (SAMRC) for the “Shared Roots” Flagship Project, Grant no. MRC-RFA-IFSP-01-2013/SHARED ROOTS” through funding received from the South African National Treasury under its Economic Competitiveness and Support Package.
SSu received post-doctoral support from the South African Research Chairs Initiative in PTSD funded by the Department of Science and Technology and the South African National Research Foundation (NRF), funding from the SAMRC through a Self-Initiated Research Grant, and funding from the NRF through the Competitive Programme for Rated Researchers (Grant Number SRUG2204123207).
LvdH was supported in part by the NRF (Grant Number 138430) and by the SAMRC under a Self-Initiated Research Grant.
EB was supported by the South African Medical Research Council Unit on Genomics of Brain Disorders.
SSe was supported by the South African Medical Research Council and was the recipient of the Shared Roots grant (MRC-RFA-IFSP-01-2013/SHARED ROOTS).
RE has received honoraria from AstraZeneca, Bristol-Myers Squibb, Janssen, Lilly, Lundbeck, Organon, Pfizer, Servier, Otsuka and Wyeth. He has also received research funding from Janssen, Lundbeck and AstraZeneca.
Funders have played no role in the study design, data collection, analysis, and interpretation and in writing the manuscript. Its’ contents are solely the responsibility of the authors and do not necessarily represent the official views of the South African MRC or NRF.
Competing interest
The authors declare none.
Ethical approval
This study was performed in line with the principles of the Declaration of Helsinki. Ethical approval for the study was obtained from the Health Research Ethics Committee of the Faculty of Medicine and Health Sciences (FHMS), Stellenbosch University (SU) (N13/08/115), and all participants provided informed consent.
Open access
