Hostname: page-component-8448b6f56d-cfpbc Total loading time: 0 Render date: 2024-04-23T13:11:53.069Z Has data issue: false hasContentIssue false
  • English
  • Français

The Value of Bayesian Methods for Accurate and Efficient Neuropsychological Assessment

Published online by Cambridge University Press:  19 October 2021

Hanne Huygelier Open the ORCID record for Hanne Huygelier [Opens in a new window] ,
Céline R. Gillebert  and
Pieter Moors
Hanne Huygelier*
Affiliation:
Department of Brain & Cognition, KU Leuven, Leuven, Belgium
Céline R. Gillebert
Affiliation:
Department of Brain & Cognition, KU Leuven, Leuven, Belgium
Pieter Moors
Affiliation:
Department of Brain & Cognition, KU Leuven, Leuven, Belgium
*
*Correspondence and reprint requests to: Hanne Huygelier, Tiensestraat 102 box 3711, 3000 Leuven, Belgium. Email: hanne.huygelier@kuleuven.be
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

Clinical neuropsychology has been slow in adopting novelties in psychometrics, statistics, and technology. Researchers have indicated that the stationary nature of clinical neuropsychology endangers its evidence-based character. In addition to a technological crisis, there may be a statistical crisis affecting clinical neuropsychology. That is, the frequentist null hypothesis significance testing framework remains the dominant approach in clinical practice, despite a recent surge in critique on this framework. While the Bayesian framework has been put forward as a viable alternative in psychology in general, the possibilities it offers to clinical neuropsychology have not received much attention.

Method:

In the current position paper, we discuss and reflect on the value of Bayesian methods for the advancement of evidence-based clinical neuropsychology.

Results:

We aim to familiarize clinical neuropsychologists and neuropsychological researchers to Bayesian methods of inference and provide a clear rationale for why these methods are valuable for clinical neuropsychology.

Conclusion:

We argue that Bayesian methods allow for a more intuitive answer to our diagnostic questions and form a more solid foundation for sequential and adaptive diagnostic testing, representing uncertainty about patients’ observed test scores and cognitive modeling of test results.

Keywords

PsychometricsEvidence-basedAssessmentBayesianBase rateDiagnostic Accuracy
Type
Critical Review
Information
Journal of the International Neuropsychological Society , Volume 28 , Issue 9 , October 2022 , pp. 984 - 995
Copyright
Copyright © INS. Published by Cambridge University Press, 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Albers, C., & Lakens, D. (2018). When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias. Journal of Experimental Social Psychology, 74, 187–195. https://doi.org/10.1016/j.jesp.2017.09.004 CrossRefGoogle Scholar
Alexander, G. E., Satalich, T. A., Shankle, W. R., & Batchelder, W. H. (2016). A cognitive psychometric model for the psychodiagnostic assessment of memory-related deficits. Psychological Assessment, 28(3), 279–293. https://doi.org/10.1037/pas0000163 CrossRefGoogle ScholarPubMed
Bergh, D. van den, Doorn, J. van, Marsman, M., Draws, T., Kesteren, E.-J. van, Derks, K., … Wagenmakers, E.-J. (2020). A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP. Lannee Psychologique, 120(1), 73–96.CrossRefGoogle Scholar
Bilder, R. M. (2011). Neuropsychology 3.0: Evidence-Based Science and Practice. Journal of the International Neuropsychological Society, 17(1), 7–13. https://doi.org/10.1017/S1355617710001396 CrossRefGoogle ScholarPubMed
Bilder, R. M., & Reise, S. P. (2019). Neuropsychological tests of the future: How do we get there from here? The Clinical Neuropsychologist, 33(2), 220–245. https://doi.org/10.1080/13854046.2018.1521993 CrossRefGoogle Scholar
Bowen, A., McKenna, K., & Tallis, R. C. (1999). Reasons for Variability in the Reported Rate of Occurrence of Unilateral Spatial Neglect After Stroke. Stroke, 30(6), 1196–1202. https://doi.org/10.1161/01.STR.30.6.1196 CrossRefGoogle ScholarPubMed
Brooks, B. L., Sherman, E. M. S., Iverson, G. L., Slick, D. J., & Strauss, E. (2011). Psychometric Foundations for the Interpretation of Neuropsychological Test Results. In Schoenberg, M. R. & Scott, J. G. (Eds.), The Little Black Book of Neuropsychology: A Syndrome-Based Approach (pp. 893–922). Springer US. https://doi.org/10.1007/978-0-387-76978-3_31 CrossRefGoogle Scholar
Bürkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01 CrossRefGoogle Scholar
Crawford, J. R. (2003). Psychometric Foundations of Neuropsychological Assessment. In Goldstein, L. H. & McNeil, J. (Eds.), Clinical Neuropsychology: A practical guide to assessment and management for clinicians (p. 34). Wiley.Google Scholar
Crawford, J. R., & Garthwaite, P. H. (2007). Comparison of a single case to a control or normative sample in neuropsychology: Development of a Bayesian approach. Cognitive Neuropsychology, 24(4), 343–372. https://doi.org/10.1080/02643290701290146 CrossRefGoogle ScholarPubMed
Crawford, P. J. R., Garthwaite, P. H., & Betkowska, K. (2009). Bayes’ theorem and diagnostic tests in neuropsychology: Interval estimates for post-test probabilities. The Clinical Neuropsychologist, 23(4), 624–644. https://doi.org/10.1080/13854040802524229 CrossRefGoogle ScholarPubMed
Crawford, P. J. R., Garthwaite, P. H., & Slick, D. J. (2009). On percentile norms in neuropsychology: Proposed reporting standards and methods for quantifying the uncertainty over the percentile ranks of test scores. The Clinical Neuropsychologist, 23(7), 1173–1195. https://doi.org/10.1080/13854040902795018 CrossRefGoogle ScholarPubMed
Cumming, G. (2014). The New Statistics: Why and How. Psychological Science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966 CrossRefGoogle ScholarPubMed
de Finetti, B. (1974). Bayesianism: Its Unifying Role for Both the Foundations and Applications of Statistics. International Statistical Review/Revue Internationale de Statistique, 42(2), 117–130. https://doi.org/10.2307/1403075 Google Scholar
Elwood, R. W. (1993). Clinical discriminations and neuropsychological tests: An appeal to bayes’ theorem. Clinical Neuropsychologist, 7(2), 224–233. https://doi.org/10.1080/13854049308401527 CrossRefGoogle ScholarPubMed
Eskes, G. A., Lanctôt, K. L., Herrmann, N., Lindsay, P., Bayley, M., Bouvier, L., … Heart Stroke Foundation Canada Canadian Stroke Best Practices Committees. (2015). Canadian Stroke Best Practice Recommendations: Mood, Cognition and Fatigue Following Stroke practice guidelines, update 2015. International Journal of Stroke: Official Journal of the International Stroke Society, 10(7), 1130–1140. https://doi.org/10.1111/ijs.12557 CrossRefGoogle ScholarPubMed
Etz, A., Gronau, Q. F., Dablander, F., Edelsbrunner, P. A., & Baribault, B. (2018). How to become a Bayesian in eight easy steps: An annotated reading list. Psychonomic Bulletin & Review, 25(1), 219–234. https://doi.org/10.3758/s13423-017-1317-5 CrossRefGoogle ScholarPubMed
Frederick, R. I., & Bowden, S. C. (2009). The Test Validation Summary. Assessment, 16(3), 215–236. https://doi.org/10.1177/1073191108325005 CrossRefGoogle ScholarPubMed
Gavett, B. E. (2015). The value of Bayes’ theorem for interpreting abnormal test scores in cognitively healthy and clinical samples. Journal of the International Neuropsychological Society: JINS, 21(3), 249–257. https://doi.org/10.1017/S1355617715000168 CrossRefGoogle ScholarPubMed
Gelman, A., & Geurts, H. M. (2017). The statistical crisis in science: How is it relevant to clinical neuropsychology? The Clinical Neuropsychologist, 31(6–7), 1000–1014. https://doi.org/10.1080/13854046.2016.1277557 CrossRefGoogle Scholar
Gordon, N. G. (1977). Base Rates and the Decision Making Model in Clinical Neuropsychology. Cortex, 13(1), 3–10. https://doi.org/10.1016/S0010-9452(77)80048-8 CrossRefGoogle ScholarPubMed
Gouvier, W. D. (2001). Are you sure you’re really telling the truth? NeuroRehabilitation, 16(4), 215–219. https://doi.org/10.3233/NRE-2001-16404 CrossRefGoogle ScholarPubMed
Grant, W. M., Hickock, G., & Fridriksson, J. (2018). A cognitive psychometric model for assessment of picture naming abilities in aphasia. Psychological Assessment, 30(6), 809–826. https://doi.org/10.1037/pas0000529 Google Scholar
Habekost, T. (2015). Clinical TVA-based studies: A general review. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.00290 CrossRefGoogle ScholarPubMed
Haines, N., Vassileva, J., & Ahn, W.-Y. (2018). The Outcome-Representation Learning Model: A Novel Reinforcement Learning Model of the Iowa Gambling Task. Cognitive Science, 42(8), 2534–2561. https://doi.org/10.1111/cogs.12688 CrossRefGoogle ScholarPubMed
Hepworth, L. R., Rowe, F. J., Walker, M. F., Rockliffe, J., Noonan, C., Howard, C., & Currie, J. (2016). Post-stroke Visual Impairment: A Systematic Literature Review of Types and Recovery of Visual Conditions. Ophthalmology Research: An International Journal, 1–43. https://doi.org/10.9734/OR/2016/21767 Google Scholar
Hilsabeck, R. C. (2017). Psychometrics and statistics: Two pillars of neuropsychological practice. The Clinical Neuropsychologist, 31(6–7), 995–999. https://doi.org/10.1080/13854046.2017.1350752 CrossRefGoogle ScholarPubMed
Huygelier, H., Moore, M. J., Demeyere, N., & Gillebert, C. R. (2020). Non-spatial impairments affect false positive neglect diagnosis based on cancellation tasks. Journal of the International Neuropsychological Society, 26, 668–678. https://doi.org/10.1017/S1355617720000041 CrossRefGoogle ScholarPubMed
Intercollegiate Stroke Working Party. (2016). National clinical guideline for stroke (No. 5; pp. 1–232). Royal College of Physicians. https://www.bgs.org.uk/sites/default/files/content/resources/files/2018-06-05/national_guidelines_2016.pdf Google Scholar
Ivnik, R. J., Smith, G. E., Cerhan, J. H., Boeve, B. F., Tangalos, E. G., & Petersen, R. C. (2001). Understanding the Diagnostic Capabilities of Cognitive Tests. The Clinical Neuropsychologist, 15(1), 114–124. https://doi.org/10.1076/clin.15.1.114.1904 CrossRefGoogle ScholarPubMed
Jeffrey, R. C. (1956). Valuation and Acceptance of Scientific Hypotheses. Philosophy of Science, 23(3), 237–246.CrossRefGoogle Scholar
Jeffreys, H. (1961). Theory of probability. Oxford University Press.Google Scholar
Jones, W. P. (1989). A proposal for the use of Bayesian probabilities in neuropsychological assessment. Neuropsychology, 3(1), 17–22. https://doi.org/10.1037/h0091742 CrossRefGoogle Scholar
Jones, W. P. (1991). Bayesian Interpretation of Test Reliability. Educational and Psychological Measurement, 51(3), 627–635. https://doi.org/10.1177/0013164491513009 CrossRefGoogle Scholar
Kass, R. E., & Raftery, A. E. (1995). Bayes Factors. Journal of the Americal Statistical Association, 90(430), 773–795.CrossRefGoogle Scholar
Kruschke, J. (2014). Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press.Google Scholar
Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14(7), 293–300. https://doi.org/10.1016/j.tics.2010.05.001 CrossRefGoogle ScholarPubMed
Kruschke, J. K. (2011). Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison. Perspectives on Psychological Science, 6(3), 299–312. https://doi.org/10.1177/1745691611406925 CrossRefGoogle ScholarPubMed
Kruschke, J. K., & Liddell, T. M. (2018a). Bayesian data analysis for newcomers. Psychonomic Bulletin & Review, 25(1), 155–177. https://doi.org/10.3758/s13423-017-1272-1 CrossRefGoogle ScholarPubMed
Kruschke, J. K., & Liddell, T. M. (2018b). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25(1), 178–206. https://doi.org/10.3758/s13423-016-1221-4 CrossRefGoogle ScholarPubMed
Labarge, A. S., McCaffrey, R. J., & Brown, T. A. (2003). Neuropsychologists’ abilities to determine the predictive value of diagnostic tests. Archives of Clinical Neuropsychology, 18(2), 165–175. https://doi.org/10.1093/arclin/18.2.165 CrossRefGoogle ScholarPubMed
Lakens, D. (2014). Performing high-powered studies efficiently with sequential analyses. European Journal of Social Psychology, 44(7), 701–710. https://doi.org/10.1002/ejsp.2023 CrossRefGoogle Scholar
Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence Testing for Psychological Research: A Tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269. https://doi.org/10.1177/2515245918770963 CrossRefGoogle Scholar
Lee, M. D., Abramyan, M., & Shankle, W. R. (2016). New methods, measures, and models for analyzing memory impairment using triadic comparisons. Behavior Research Methods, 48(4), 1492–1507. https://doi.org/10.3758/s13428-015-0662-4 CrossRefGoogle ScholarPubMed
Lee, M. D., Lodewyckx, T., & Wagenmakers, J. (2015). Three Bayesian Analyses of Memory Deficits in Patients with Dissociative Identity Disorder. In Raaijmakers, J. G. W., Goldstone, R., Nosofsky, R., & Steyvers, M. (Eds.), Cognitive modeling in perception and memory: A Festschrift for Richard M. Shiffrin (p. 189). New York: Psychology Press.Google Scholar
Lezak, P. of , N. P. and , N. M. D., Lezak, M. D., Howieson, A. P. of , N. and , P. D. B., Howieson, D. B., Loring, P. of , N. D. W., Loring, D. W., & Fischer, J. S. (2004). Neuropsychological Assessment. Oxford University Press.Google Scholar
Marsman, M., & Wagenmakers, E.-J. (2017). Bayesian benefits with JASP. European Journal of Developmental Psychology, 14(5), 545–555. https://doi.org/10.1080/17405629.2016.1259614 CrossRefGoogle Scholar
Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52(3), 194–216. https://doi.org/10.1037/h0048070 CrossRefGoogle ScholarPubMed
Meredith, M., & Kruschke, J. (2020). Bayesian Estimation Supersedes the t-Test.Google Scholar
Miller, J. B., & Barr, W. B. (2017). The Technology Crisis in Neuropsychology. Archives of Clinical Neuropsychology, 32(5), 541–554. https://doi.org/10.1093/arclin/acx050 CrossRefGoogle ScholarPubMed
Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E.-J. (2016). The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin & Review, 23(1), 103–123. https://doi.org/10.3758/s13423-015-0947-8 CrossRefGoogle ScholarPubMed
Myung, J. I., Cavagnaro, D. R., & Pitt, M. A. (2013). A Tutorial on Adaptive Design Optimization. Journal of Mathematical Psychology, 57(3–4), 53–67. https://doi.org/10.1016/j.jmp.2013.05.005 CrossRefGoogle ScholarPubMed
National Stroke Foundation. (2010). Clinical Guidelines for Stroke Management 2010. Royal Australasian College of Physicians and its Australasian Faculty of Rehabilitation Medicine. https://www.pedro.org.au/wp-content/uploads/CPG_stroke.pdf Google Scholar
O’Bryant, S. E., & Lucas, J. A. (2006). Estimating the Predictive Value of the Test of Memory Malingering: An Illustrative Example for Clinicians. The Clinical Neuropsychologist, 20(3), 533–540. https://doi.org/10.1080/13854040590967568 CrossRefGoogle ScholarPubMed
Pendergrass, J. C., Targum, S. D., & Harrison, J. E. (2018). Cognitive impairment associated with cancer: A brief review. Innovations in Clinical Neuroscience, 15(1–2), 36–44.Google ScholarPubMed
Polanczyk, G., de Lima, M. S., Horta, B. L., Biederman, J., & Rohde, L. A. (2007). The Worldwide Prevalence of ADHD: A Systematic Review and Metaregression Analysis. American Journal of Psychiatry, 164(6), 942–948. https://doi.org/10.1176/ajp.2007.164.6.942 CrossRefGoogle ScholarPubMed
Pooley, J. P., Lee, M. D., & Shankle, W. R. (2011). Understanding memory impairment with memory models and hierarchical Bayesian analysis. Journal of Mathematical Psychology, 55(1), 47–56. https://doi.org/10.1016/j.jmp.2010.08.003 CrossRefGoogle Scholar
Rasquin, S. M. C., Lodder, J., & Verhey, F. R. J. (2005). The Effect of Different Diagnostic Criteria on the Prevalence and Incidence of Post-Stroke Dementia. Neuroepidemiology, 24(4), 189–195. https://doi.org/10.1159/000084711 CrossRefGoogle ScholarPubMed
Rouder, J. N., & Haaf, J. M. (2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin & Review, 26(2), 452–467. https://doi.org/10.3758/s13423-018-1558-y CrossRefGoogle ScholarPubMed
Satz, P., Fennell, E., & Reilly, C. (1970). Predictive validity of six neurodiagnostic tests: A decision theory analysis. Journal of Consulting and Clinical Psychology, 34(3), 375–381. https://doi.org/10.1037/h0029301 CrossRefGoogle ScholarPubMed
Scandola, M., & Romano, D. (2021). Bayesian Multilevel Single Case Models using ‘Stan’. A new tool to study single cases in Neuropsychology. Neuropsychologia. https://doi.org/10.1016/j.neuropsychologia.2021.107834 CrossRefGoogle ScholarPubMed
Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2017). Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences. Psychological Methods, 22(2), 322–339. https://doi.org/10.1037/met0000061 CrossRefGoogle ScholarPubMed
Shankle, W. R., Pooley, J. P., Steyvers, M., Hara, J., Mangrola, T., Reisberg, B., & Lee, M. D. (2013). Relating memory to functional performance in normal aging to dementia using hierarchical Bayesian cognitive processing models. Alzheimer Disease and Associated Disorders, 27(1), 16–22. https://doi.org/10.1097/WAD.0b013e31824d5668 CrossRefGoogle ScholarPubMed
Slick, D. J. (2006). Psychometrics in neuropsychological assessment. In Strauss, E., Sherman, E. M., & Spreen, O. (Eds.), A compendium of neuropsychological tests: Administration, norms and commentary (Vol. 1, pp. 3–43). Oxford University Press.Google Scholar
Stadskleiv, K. (2020). Cognitive functioning in children with cerebral palsy. Developmental Medicine & Child Neurology, 62(3), 283–289. https://doi.org/10.1111/dmcn.14463 CrossRefGoogle ScholarPubMed
van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M., & Depaoli, S. (2017). A systematic review of Bayesian articles in psychology: The last 25 years. Psychological Methods, 22(2), 217. https://doi.org/10.1037/met0000100 CrossRefGoogle ScholarPubMed
Voorspoels, W., Rutten, I., Bartlema, A., Tuerlinckx, F., & Vanpaemel, W. (2018). Sensitivity to the prototype in children with high-functioning autism spectrum disorder: An example of Bayesian cognitive psychometrics. Psychonomic Bulletin & Review, 25(1), 271–285. https://doi.org/10.3758/s13423-017-1245-4 CrossRefGoogle Scholar
Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14(5), 779–804. https://doi.org/10.3758/BF03194105 CrossRefGoogle Scholar
Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., … Morey, R. D. (2018). Bayesian inference for psychology. Part II: Example applications with JASP. Psychonomic Bulletin & Review, 25(1), 58–76. https://doi.org/10.3758/s13423-017-1323-7 CrossRefGoogle ScholarPubMed
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100(3), 426–432. http://dx.doi.org/10.1037/a0022790 CrossRefGoogle Scholar
Westlund, E., & Stuart, E. A. (2016). The Nonuse, Misuse, and Proper Use of Pilot Studies in Experimental Evaluation Research: American Journal of Evaluation. https://doi.org/10.1177/1098214016651489 Google Scholar
Woods, S. P., Weinborn, M., & Lovejoy, D. W. (2003). Are Classification Accuracy Statistics Underused in Neuropsychological Research? Journal of Clinical and Experimental Neuropsychology, 25(3), 431–439. https://doi.org/10.1076/jcen.25.3.431.13800 CrossRefGoogle ScholarPubMed
Supplementary material: File

Huygelier et al. supplementary material

Appendices A-C

Download Huygelier et al. supplementary material(File)
File 872.3 KB