Skip to main content

Prospective, Head-to-Head Study of Three Computerized Neurocognitive Assessment Tools Part 2: Utility for Assessment of Mild Traumatic Brain Injury in Emergency Department Patients

  • Lindsay D. Nelson (a1), Robyn E. Furger (a1), Peter Gikas (a1), E. Brooke Lerner (a1), William B. Barr (a2), Thomas A. Hammeke (a1), Christopher Randolph (a3), Kevin Guskiewicz (a4) and Michael A. McCrea (a1)...

Objectives: The aim of this study was to evaluate the reliability and validity of three computerized neurocognitive assessment tools (CNTs; i.e., ANAM, DANA, and ImPACT) for assessing mild traumatic brain injury (mTBI) in patients recruited through a level I trauma center emergency department (ED). Methods: mTBI (n=94) and matched trauma control (n=80) subjects recruited from a level I trauma center emergency department completed symptom and neurocognitive assessments within 72 hr of injury and at 15 and 45 days post-injury. Concussion symptoms were also assessed via phone at 8 days post-injury. Results: CNTs did not differentiate between groups at any time point (e.g., M 72-hr Cohen’s d=−.16, .02, and .00 for ANAM, DANA, and ImPACT, respectively; negative values reflect greater impairment in the mTBI group). Roughly a quarter of stability coefficients were over .70 across measures and test–retest intervals in controls. In contrast, concussion symptom score differentiated mTBI vs. control groups acutely), with this effect size diminished over time (72-hr and day 8, 15, and 45 Cohen’s d=−.78, −.60, −.49, and −.35, respectively). Conclusions: The CNTs evaluated, developed and widely used to assess sport-related concussion, did not yield significant differences between patients with mTBI versus other injuries. Symptom scores better differentiated groups than CNTs, with effect sizes weaker than those reported in sport-related concussion studies. Nonspecific injury factors, and other characteristics common in ED settings, likely affect CNT performance across trauma patients as a whole and thereby diminish the validity of CNTs for assessing mTBI in this patient population. (JINS, 2017, 23, 293–303)

Corresponding author
Correspondence and reprint requests to: Lindsay Nelson, Department of Neurosurgery, Neuroscience Research Center, 8701 West Watertown Plank Road, Milwaukee, WI 53226. E-mail:
Hide All
Babikian, T., Satz, P., Zaucha, K., Light, R., Lewis, R.S., & Asarnow, R.F. (2011). The UCLA longitudinal study of neurocognitive outcomes following mild pediatric traumatic brain injury. Journal of the International Neuropsychological Society, 17, 886895. doi: 10.1017/S1355617711000907
Bazarian, J.J., McClung, J., Cheng, Y.T., Flesher, W., & Schneider, S.M. (2005). Emergency department management of mild traumatic brain injury in the USA. Emergency Medicine, 22, 473477. doi: 10.1136/emj.2004.019273
Bazarian, J.J., McClung, J., Shah, M.N., Cheng, Y.T., Flesher, W., & Kraus, J. (2005). Mild traumatic brain injury in the United States, 1998--2000. Brain Injury, 19, 8591.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Statistical Mehodology), 57, 289300.
Bruera, E., Macmillan, K., Hanson, J., & MacDonald, R.N. (1989). The cognitive effects of the administration of narcotic analgesics in patients with cancer pain. Pain, 39, 1316.
Cassidy, J.D., Carroll, L.J., Peloso, P.M., Borg, J., von Holst, H., Holm, L., . . . WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. (2004). Incidence, risk factors and prevention of mild traumatic brain injury: Results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. Journal of Rehabilitation Medicine, 43(Suppl), 2860.
Centers for Disease Control and Prevention, CDC. (2015). Report to congress on traumatic brain injury in the United States: Epidemiology and rehabilitation. Atlanta, GA: National Center for Injury Prevention and Control; Division of Unintentional Injury Prevention.
Chin, E.Y., Nelson, L.D., Barr, W.B., McCrory, P., & McCrea, M.A. (2016). Reliability and validity of the Sport Concussion Assessment Tool 3 (SCAT3) in high school and collegiate athletes. American Journal of Sports Medicine, 44, 22762285. doi: 10.1177/0363546516648141
Covassin, T., Elbin, R. III, & Stiller-Ostrowski, J.L. (2009). Current sport-related concussion teaching and clinical practices of sports medicine professionals. Journal of Athletic Training, 44, 400404. doi: 10.4085/1062-6050-44.4.400
Delaney, J.S., Abuzeyad, F., Correa, J.A., & Foxford, R. (2005). Recognition and characteristics of concussions in the emergency department population. Journal of Emergency Medicine, 29, 189197.
Derogatis, L.R. (2001). Brief Symptom Inventory 18 (BSI-18): Administration, scoring, and procedures manual. Bloomington, MN: Pearson.
Diener, E., Emmons, R.A., Larsen, R.J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49, 7175. doi: 10.1207/s15327752jpa4901_13
Dischinger, P.C., Ryb, G.E., Kufera, J.A., & Auman, K.M. (2009). Early predictors of postconcussive syndrome in a population of trauma patients with mild traumatic brain injury. Journal of Trauma, 66, 289296. doi: 10.1097/TA.0b013e3181961da2
Eastvold, A.D., Belanger, H.G., & Vanderploeg, R.D. (2012). Does a third party observer affect neuropsychological test performance? It depends. The Clinical Neuropsychologist, 26, 520541.
Faul, M., Xu, L., Wald, M.M., & Coronado, V.G. (2010). Traumatic brain injury in the United States: Emergency department visits, hospitalizations and deaths 2002-2006. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.
Furger, R.E., Nelson, L.D., Lerner, E.B., & McCrea, M.A. (2016). Frequency of factors that complicate the identification of mild traumatic brain injury in level I trauma center patients. Concussion, 1, CNC11. doi: 10.2217/cnc.15.11
Green, P. (2003). Green’s Medical Symptom Validity Test for Windows. Edmonton, Alberta, Canada: Green’s Publishing, Inc.
Guskiewicz, K.M., Ross, S.E., & Marshall, S.W. (2001). Postural stability and neuropsychological deficits after concussion in collegiate athletes. Journal of Athletic Training, 36, 263273.
Helmick, K., Guskiewicz, K., Barth, J., Cantu, R., Kelly, J., McDonald, E., & Warden, D. (2006). Defense and Veterans Brain Injury Center Working Group on the Acute Management of Mild Traumatic Brain Injury in Military Operational Settings: Clinical practice guideline and recommendations. Washington, DC: Defense and Veteran Brain Injury Center. Retrieved from
Iverson, G.L. (2005). Outcome from mild traumatic brain injury. Current Opinion in Psychiatry, 18, 301317. doi: 10.1097/
Iverson, G.L., & Lange, R.T. (2003). Examination of “postconcussion-like” symptoms in a healthy sample. Applied Neuropsychology, 10, 137144. doi: 10.1207/S15324826AN1003_02
Landre, N., Poppe, C.J., Davis, N., Schmaus, B., & Hobbs, S.E. (2006). Cognitive functioning and postconcussive symptoms in trauma patients with and without mild TBI. Archives of Clinical Neuropsychology, 21, 255273. doi: 10.1016/j.acn.2005.12.007
Lichtenstein, J.D., Moser, R.S., & Schatz, P. (2014). Age and test setting affect the prevalence of invalid baseline scores on neurocognitive tests. American Journal of Sports Medicine, 42, 479484. doi: 10.1177/0363546513509225
Luoto, T.M., Tenovuo, O., Kataja, A., Brander, A., Ohman, J., & Iverson, G.L. (2013). Who gets recruited in mild traumatic brain injury research? Journal of Neurotrauma, 30, 1116. doi: 10.1089/neu.2012.2611
Macciocchi, S.N., Seel, R.T., & Thompson, N. (2013). The impact of mild traumatic brain injury on cognitive functioning following co-occurring spinal cord injury. Archives of Clinical Neuropsychology, 28, 684691. doi: 10.1093/arclin/act049
Mathias, J.L., Harman-Smith, Y., Bowden, S.C., Rosenfeld, J.V., & Bigler, E.D. (2014). Contribution of psychological trauma to outcomes after traumatic brain injury: Assaults versus sporting injuries. Journal of Neurotrauma, 31, 658669. doi: 10.1089/neu.2013.3160
McCrea, M., Guskiewicz, K.M., Marshall, S.W., Barr, W., Randolph, C., Cantu, R.C., & Kelly, J.P. (2003). Acute effects and recovery time following concussion in collegiate football players: The NCAA Concussion Study. Journal of the American Medical Association, 290, 25562563. doi: 10.1001/jama.290.19.2556
McCrea, M., Guskiewicz, K., Randolph, C., Barr, W.B., Hammeke, T.A., Marshall, S.W., & Kelly, J.P. (2013). Incidence, clinical course, and predictors of prolonged recovery time following sport-related concussion in high school and college athletes. Journal of the International Neuropsychological Society, 19, 2233. doi: 10.1017/S1355617712000872
McCrea, M., Kelly, J.P., Randolph, C., Kluge, J., Bartolic, E., Finn, G., & Baxter, B. (1998). Standardized assessment of concussion (SAC): On-site mental status evaluation of the athlete. Journal of Head Trauma Rehabilitation, 13, 2735.
McCrory, P., Meeuwisse, W.H., Aubry, M., Cantu, B., Dvorak, J., Echemendia, R.J., & Turner, M. (2013). Consensus statement on concussion in sport: The 4th International Conference on Concussion in Sport held in Zurich, November 2012. British Journal of Sports Medicine, 47, 250258. doi: 10.1136/bjsports-2013-092313
Meehan, W.P. III, d’Hemecourt, P., Collins, C.L., Taylor, A.M., & Comstock, R.D. (2012). Computerized neurocognitive testing for the management of sport-related concussions. Pediatrics, 129, 3844. doi: 10.1542/peds.2011-1972
Meehan, W.P. III, Mannix, R., Monuteaux, M.C., Stein, C.J., & Bachur, R.G. (2014). Early symptom burden predicts recovery after sport-related concussion. Neurology, 83, 22042210. doi: 10.1212/WNL.0000000000001073
Nelson, L.D., LaRoche, A.A., Pfaller, A.Y., Lerner, E.B., Hammeke, T.A., Randolph, C., & McCrea, M.A. (2016). Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): Reliability and validity for the assessment of sport-related concussion. Journal of the International Neuropsychological Society, 22, 2437. doi: 10.1017/S1355617715001101
Nelson, L.D., Pfaller, A.Y., Rein, L., & McCrea, M.A. (2015). Rates and predictors of invalid baseline test performance for three computerized neurocognitive tests (CNTs): ANAM, Axon, and ImPACT. American Journal of Sports Medicine, 43, 20182026. doi: 10.1177/0363546515587714
Nelson, L.D., Tarima, S., LaRoche, A.A., Hammeke, T.A., Barr, W.B., Guskiewicz, K., & McCrea, M.A. (2016). Preinjury somatization symptoms contribute to clinical recovery after sport-related concussion. Neurology, 86, 18561863.
Peterson, S.E., Stull, M.J., Collins, M.W., & Wang, H.E. (2009). Neurocognitive function of emergency department patients with mild traumatic brain injury. Annals of Emergency Medicine, 53, 796–803 e791. doi: 10.1016/j.annemergmed.2008.10.015
Powell, J.M., Ferraro, J.V., Dikmen, S.S., Temkin, N.R., & Bell, K.R. (2008). Accuracy of mild traumatic brain injury diagnosis. Archives of Physical Medicine and Rehabilitation, 89, 15501555. doi: 10.1016/j.apmr.2007.12.035
Rabinowitz, A.R., Li, X., McCauley, S.R., Wilde, E.A., Barnes, A., Hanten, G., & Levin, H.S. (2015). Prevalence and predictors of poor recovery from mild traumatic brain injury. Journal of Neurotrauma, 32, 14881496. doi: 10.1089/neu.2014.3555
Sheedy, J., Geffen, G., Donnelly, J., & Faux, S. (2006). Emergency department assessment of mild traumatic brain injury and prediction of post-concussion symptoms at one month post injury. Journal of Clinical and Experimental Neuropsychology, 28, 755772. doi: 10.1080/13803390591000864
Smith-Seemiller, L., Fow, N.R., Kant, R., & Franzen, M.D. (2003). Presence of post-concussion syndrome symptoms in patients with chronic pain vs mild traumatic brain injury. Brain Injury, 17, 199206.
Vincent, A.S., Roebuck-Spencer, T., Lopez, M.S., Twillie, D.A., Logan, B.W., Grate, S.J., & Gilliland, K. (2012). Effects of military deployment on cognitive functioning. Military Medicine, 177, 248255.
Weathers, J.W., Litz, B.T., Huska, J.A., & Kean, T.M. (1994). The PTSD Checklist--Civilian Version (PCL-C). Boston, MA: National Center for PTSD.
Wechsler, D. (2001). Wechsler test of adult reading: WTAR. San Antonio, TX: The Psychological Corporation.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of the International Neuropsychological Society
  • ISSN: 1355-6177
  • EISSN: 1469-7661
  • URL: /core/journals/journal-of-the-international-neuropsychological-society
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Type Description Title
Supplementary materials

Nelson supplementary material
Tables S1-S3

 Word (30 KB)
30 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed