Hostname: page-component-669899f699-qzcqf Total loading time: 0 Render date: 2025-05-05T14:56:38.798Z Has data issue: false hasContentIssue false
  • English
  • Français

Test–Retest Reliability of a Semi-Structured Interview to Aid in Pediatric Traumatic Brain Injury Diagnosis

Published online by Cambridge University Press:  11 August 2021

Danielle C. Hergert Open the ORCID record for Danielle C. Hergert [Opens in a new window] ,
Veronik Sicard Open the ORCID record for Veronik Sicard [Opens in a new window] ,
David D. Stephenson ,
Sharvani Pabbathi Reddy ,
Cidney R. Robertson-Benta ,
Andrew B. Dodd ,
Edward J. Bedrick ,
Gerard A. Gioia ,
Timothy B. Meier  and
Nicholas A. Shaff
...Show all authors
Danielle C. Hergert
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Veronik Sicard
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
David D. Stephenson
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Sharvani Pabbathi Reddy
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Cidney R. Robertson-Benta
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Andrew B. Dodd
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Edward J. Bedrick
Affiliation:
Department of Epidemiology and Biostatistics, University of Arizona, Tucson, AZ, USA
Gerard A. Gioia
Affiliation:
Department of Pediatrics and Psychiatry & Behavioral Sciences, George Washington University School of Medicine, Washington, DC, USA Division of Pediatric Neuropsychology, Children’s National Hospital, Washington, DC, USA
Timothy B. Meier
Affiliation:
Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI, USA
Nicholas A. Shaff
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Davin K. Quinn
Affiliation:
Department of Psychiatry and Behavioral Sciences, University of New Mexico, Albuquerque, NM, USA
Richard A. Campbell
Affiliation:
Department of Psychiatry and Behavioral Sciences, University of New Mexico, Albuquerque, NM, USA
John P. Phillips
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA Department of Neurology, University of New Mexico, Albuquerque, NM, USA
Andrei A. Vakhtin
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA
Robert E. Sapien
Affiliation:
Department of Emergency Medicine, University of New Mexico Health Sciences Center, Albuquerque, NM, USA
Andrew R. Mayer*
Affiliation:
The Mind Research Network/Lovelace Biomedical Research Institute, Albuquerque, NM, USA Department of Psychiatry and Behavioral Sciences, University of New Mexico, Albuquerque, NM, USA Department of Neurology, University of New Mexico, Albuquerque, NM, USA Department of Psychology, University of New Mexico, Albuquerque, NM, USA
*
Correspondence and reprint requests to: Andrew R. Mayer, Mind Research Network, Pete and Nancy Domenici Hall, 1101 Yale Blvd. NE, Albuquerque, NM87106, USA. Email: amayer@mrn.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

Retrospective self-report is typically used for diagnosing previous pediatric traumatic brain injury (TBI). A new semi-structured interview instrument (New Mexico Assessment of Pediatric TBI; NewMAP TBI) investigated test–retest reliability for TBI characteristics in both the TBI that qualified for study inclusion and for lifetime history of TBI.

Method:

One-hundred and eight-four mTBI (aged 8–18), 156 matched healthy controls (HC), and their parents completed the NewMAP TBI within 11 days (subacute; SA) and 4 months (early chronic; EC) of injury, with a subset returning at 1 year (late chronic; LC).

Results:

The test–retest reliability of common TBI characteristics [loss of consciousness (LOC), post-traumatic amnesia (PTA), retrograde amnesia, confusion/disorientation] and post-concussion symptoms (PCS) were examined across study visits. Aside from PTA, binary reporting (present/absent) for all TBI characteristics exhibited acceptable (≥0.60) test–retest reliability for both Qualifying and Remote TBIs across all three visits. In contrast, reliability for continuous data (exact duration) was generally unacceptable, with LOC and PCS meeting acceptable criteria at only half of the assessments. Transforming continuous self-report ratings into discrete categories based on injury severity resulted in acceptable reliability. Reliability was not strongly affected by the parent completing the NewMAP TBI.

Conclusions:

Categorical reporting of TBI characteristics in children and adolescents can aid clinicians in retrospectively obtaining reliable estimates of TBI severity up to a year post-injury. However, test–retest reliability is strongly impacted by the initial data distribution, selected statistical methods, and potentially by patient difficulty in distinguishing among conceptually similar medical concepts (i.e., PTA vs. confusion).

Keywords

PediatricsAdolescentTraumatic brain injuryBrain concussionSelf-reportsTest–retest reliability
Type
Research Article
Information
Journal of the International Neuropsychological Society , Volume 28 , Issue 7 , August 2022 , pp. 687 - 699
Copyright
Copyright © INS. Published by Cambridge University Press, 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

REFERENCES

Asken, B.M., McCrea, M.A., Clugston, J.R., Snyder, A.R., Houck, Z.M., & Bauer, R.M. (2016). “Playing Through It”: Delayed reporting and removal from athletic activity after concussion predicts prolonged recovery. Journal of Athletic Training, 51(4), 329–335. Retrieved from PM:27111584CrossRefGoogle ScholarPubMed
Barlow, K.M., Crawford, S., Brooks, B.L., Turley, B., & Mikrogianakis, A. (2015). The incidence of postconcussion syndrome remains stable following mild traumatic brain injury in children. Pediatric Neurology, 53(6), 491–497. Retrieved from PM:26421987CrossRefGoogle ScholarPubMed
Bazarian, J.J., Biberthaler, P., Welch, R.D., Lewis, L.M., Barzo, P., Bogner-Flatz, V., … Jagoda, A.S. (2018). Serum GFAP and UCH-L1 for prediction of absence of intracranial injuries on head CT (ALERT-TBI): A multicentre observational study. Lancet Neurology, 17(9), 782–789. Retrieved from PM:30054151CrossRefGoogle ScholarPubMed
Bogner, J. & Corrigan, J.D. (2009). Reliability and predictive validity of the Ohio State University TBI identification method with prisoners. Journal of Head Trauma Rehabilitation, 24(4), 279–291. Retrieved from PM:19625867CrossRefGoogle ScholarPubMed
Bogner, J.A., Whiteneck, G.G., MacDonald, J., Juengst, S.B., Brown, A.W., Philippus, A.M., … Corrigan, J.D. (2017). Test-retest reliability of traumatic brain injury outcome measures: A traumatic brain injury model systems study. Journal of Head Trauma Rehabilitation, 32(5), E1–E16. Retrieved from PM:28195954CrossRefGoogle ScholarPubMed
Caplan, B., Bogner, J., Brenner, L., Arciniegas, D., Silverberg, N.D., Iverson, G.L., … Aquino, A. (2016). The nature and clinical significance of preinjury recall bias following mild traumatic brain injury. Journal of Head Trauma Rehabilitation, 31(6), 388–396.Google Scholar
Carroll, L.J., Cassidy, J.D., Holm, L., Kraus, J., & Coronado, V.G. (2004). Methodological issues and research recommendations for mild traumatic brain injury: The WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. Journal of Rehabilitation Medicine (43 Suppl.), 113–125. Retrieved from PM:15083875CrossRefGoogle ScholarPubMed
Castile, L., Collins, C.L., McIlvain, N.M., & Comstock, R.D. (2012). The epidemiology of new versus recurrent sports concussions among high school athletes, 2005-2010. British Journal of Sports Medicine, 46(8), 603–610. Retrieved from PM:22144000CrossRefGoogle ScholarPubMed
Cicchetti, D.V. (2001). The precision of reliability and validity estimates re-visited: Distinguishing between clinical and statistical significance of sample size requirements. Journal of Clinical and Experimental Neuropsychology, 23(5), 695–700. Retrieved from PM:11778646CrossRefGoogle ScholarPubMed
Corrigan, J.D. & Bogner, J. (2007). Initial reliability and validity of the Ohio State University TBI Identification Method. Journal of Head Trauma Rehabilitation, 22(6), 318–329. Retrieved from PM:18025964CrossRefGoogle ScholarPubMed
Corrigan, J.D., Yang, J., Singichetti, B., Manchester, K., & Bogner, J. (2018). Lifetime prevalence of traumatic brain injury with loss of consciousness. Injury Prevention, 24(6), 396–404.CrossRefGoogle ScholarPubMed
Crowe, L.M., Hearps, S., Anderson, V., Borland, M.L., Phillips, N., Kochar, A., … Babl, F.E. (2018). Investigating the variability in mild traumatic brain injury definitions: A prospective cohort study. Archives of Physical Medicine and Rehabilitation, 99(7), 1360–1369. Retrieved from PM:29407521CrossRefGoogle ScholarPubMed
Delaney, J.S., Lacroix, V.J., Leclerc, S., & Johnston, K.M. (2002). Concussions among university football and soccer players. Clinical Journal of Sport Medicine, 12(6), 331–338. Retrieved from PM:12466687CrossRefGoogle ScholarPubMed
Denning, J.H. (2012). The efficiency and accuracy of the Test of Memory Malingering trial 1, errors on the first 10 items of the test of memory malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27(4), 417–432. Retrieved from PM:22543569CrossRefGoogle ScholarPubMed
Eisenberg, M.A., Meehan, W.P. III, & Mannix, R. (2014). Duration and course of post-concussive symptoms. Pediatrics, 133(6), 999–1006. Retrieved from PM:24819569CrossRefGoogle ScholarPubMed
Gabbe, B.J., Finch, C.F., Bennell, K.L., & Wajswelner, H. (2003). How valid is a self reported 12 month sports injury history? British Journal of Sports Medicine, 37(6), 545–547. Retrieved from PM:14665599CrossRefGoogle ScholarPubMed
Gioia, G.A., Collins, M., & Isquith, P.K. (2008). Improving identification and diagnosis of mild traumatic brain injury with evidence: psychometric support for the acute concussion evaluation. Journal of Head Trauma Rehabilitation, 23(4), 230–242. Retrieved from PM:18650767CrossRefGoogle ScholarPubMed
Gunstad, J. & Suhr, J.A. (2001). “Expectation as etiology” versus “the good old days”: Postconcussion syndrome symptom reporting in athletes, headache sufferers, and depressed individuals. Journal of the International Neuropsychological Society, 7(3), 323–333. Retrieved from PM:11311033CrossRefGoogle ScholarPubMed
Guskiewicz, K.M., McCrea, M., Marshall, S.W., Cantu, R.C., Randolph, C., Barr, W., … Kelly, J.P. (2003). Cumulative effects associated with recurrent concussion in collegiate football players: The NCAA Concussion Study. JAMA , 290(19), 2549–2555.CrossRefGoogle ScholarPubMed
Gwet, K.L. (2008). Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(Pt 1), 29–48. Retrieved from PM:18482474CrossRefGoogle ScholarPubMed
Haarbauer-Krupa, J., Arbogast, K.B., Metzger, K.B., Greenspan, A.I., Kessler, R., Curry, A.E., … Master, C.L. (2018). Variations in mechanisms of injury for children with concussion. The Journal of Pediatrics, 197, 241–248. Retrieved from PM:29627189CrossRefGoogle ScholarPubMed
Haran, H.P., Bressan, S., Oakley, E., Davis, G.A., Anderson, V., & Babl, F.E. (2016). On-field management and return-to-play in sports-related concussion in children: Are children managed appropriately? Journal of Science and Medicine in Sport, 19(3), 194–199. Retrieved from PM:25772997CrossRefGoogle ScholarPubMed
Iverson, G.L. & Lange, R.T. (2003). Examination of “postconcussion-like” symptoms in a healthy sample. Applied Neuropsychology, 10(3), 137–144. Retrieved from PM:12890639CrossRefGoogle Scholar
Iverson, G.L., Silverberg, N.D., Mannix, R., Maxwell, B.A., Atkins, J.E., Zafonte, R., & Berkner, P.D. (2015). Factors associated with concussion-like symptom reporting in high school athletes. JAMA Pediatrics, 169(12), 1132–1140. Retrieved from PM:26457403CrossRefGoogle ScholarPubMed
Kay, T., Harrington, D.E., Adams, R., Anderson, T., Berrol, S., Cicerone, K., … Harley, P. (1993). Definition of mild traumatic brain injury. Journal of Head Trauma Rehabilitation, 8(3), 86–87.CrossRefGoogle Scholar
Kerr, Z.Y., Thomas, L.C., Simon, J.E., McCrea, M., & Guskiewicz, K.M. (2018). Association between history of multiple concussions and health outcomes among former college football players: 15-year follow-up from the NCAA Concussion Study (1999-2001). American Journal of Sports Medicine, 46(7), 1733–1741.CrossRefGoogle Scholar
King, N., Crawford, S., Wenden, F., Moss, N., Wade, D., & Caldwell, F. (1997). Measurement of post-traumatic amnesia: How reliable is it? Journal of Neurology, Neurosurgery, and Psychiatry, 62(1), 38–42.CrossRefGoogle Scholar
Kristman, V.L., Borg, J., Godbolt, A.K., Salmi, L.R., Cancelliere, C., Carroll, L.J., … Cassidy, J.D. (2014). Methodological issues and research recommendations for prognosis after mild traumatic brain injury: Results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Archives of Physical Medicine and Rehabilitation, 95(3 Suppl), S265–S277. Retrieved from PM:24581912.CrossRefGoogle ScholarPubMed
Lange, R.T., Iverson, G.L., & Rose, A. (2010). Post-concussion symptom reporting and the “good-old-days” bias following mild traumatic brain injury. Archives of Clinical Neuropsychology, 25(5).CrossRefGoogle ScholarPubMed
Lexell, J.E. & Downham, D.Y. (2005). How to assess the reliability of measurements in rehabilitation. American Journal of Physical Medicine & Rehabilitation, 84(9), 719–723.CrossRefGoogle ScholarPubMed
Lezak, M.D. (1995). Neuropathology for neuropsychologists. In Neuropsychological Assessment (pp. 170–276).Google Scholar
Mailer, B.J., Valovich-McLeod, T.C., & Bay, R.C. (2008). Healthy youth are reliable in reporting symptoms on a graded symptom scale. Journal of Sport Rehabilitation, 17(1), 11–20. Retrieved from PM:18270383CrossRefGoogle ScholarPubMed
Manley, G., Gardner, A.J., Schneider, K.J., Guskiewicz, K.M., Bailes, J., Cantu, R.C., … Iverson, G.L. (2017). A systematic review of potential long-term effects of sport-related concussion. British Journal of Sports Medicine, 51(12), 969–977. Retrieved from PM:28455362CrossRefGoogle ScholarPubMed
Mayer, A.R., Hanlon, F.M., Claus, E.D., Dodd, A.B., Miller, B., Mickey, J., … Hutchison, K.E. (2018). An examination of behavioral and neuronal effects of comorbid traumatic brain injury and alcohol use. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 3(3), 294–302. Retrieved from PM:29486871Google ScholarPubMed
Mayer, A.R., Kaushal, M., Dodd, A.B., Hanlon, F.M., Shaff, N.A., Mannix, R., … Meier, T.B. (2018). Advanced biomarkers of pediatric mild traumatic brain injury: Progress and perils. Neuroscience & Biobehavioral Reviews, 94, 149–165. doi: 10.1016/j.neubiorev.2018.08.002 CrossRefGoogle ScholarPubMed
Mayer, A.R., Quinn, D.K., & Master, C.L. (2017). The spectrum of mild traumatic brain injury: A review. Neurology, 89(6), 623–632.CrossRefGoogle ScholarPubMed
Mayer, A.R., Stephenson, D.D., Dodd, A.B., Robertson-Benta, C.R., Pabbathi Reddy, S., Shaff, N.A., … Quinn, D.K. (2020). Comparison of methods for classifying persistent post-concussive symptoms in children. Journal of Neurotrauma, 37(13), 1504–1511. doi: 10.1089/neu.2019.6805 CrossRefGoogle ScholarPubMed
Mayou, R.A., Black, J., & Bryant, B. (2000). Unconsciousness, amnesia and psychiatric symptoms following road traffic accident injury. British Journal of Psychiatry, 177(6), 540–545.CrossRefGoogle ScholarPubMed
McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., … Vos, P.E. (2017). Consensus statement on concussion in sport-the 5(th) international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51(11), 838–847. Retrieved from PM:28446457Google Scholar
McMillan, T., Jongen, E., & Greenwood, R. (1996). Assessment of post-traumatic amnesia after severe closed head injury: Retrospective or prospective? Journal of Neurology, Neurosurgery, and Psychiatry 60(4), 422–427.CrossRefGoogle ScholarPubMed
Menon, D.K., Schwab, K., Wright, D.W., & Maas, A.I. (2010). Position statement: Definition of traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 91(11), 1637–1640. Retrieved from PM:21044706CrossRefGoogle ScholarPubMed
Nelson, N.W., Anderson, C.R., Thuras, P., Kehle-Forbes, S.M., Arbisi, P.A., Erbes, C.R., & Polusny, M.A. (2015). Factors associated with inconsistency in self-reported mild traumatic brain injury over time among military personnel in Iraq. British Journal of Psychiatry, 206(3), 237–244. Retrieved from PM:25614533CrossRefGoogle ScholarPubMed
Rice, S.G. (2008). Medical conditions affecting sports participation. Pediatrics, 121(4), 841–848. Retrieved from PM:18381550CrossRefGoogle ScholarPubMed
Roberts, C.M., Spitz, G., Mundy, M., & Ponsford, J.L. (2019). Prospective evaluation of first and last memory reports following moderate to severe traumatic brain injury. Journal of Clinical and Experimental Neuropsychology, 41(2), 109–117.CrossRefGoogle ScholarPubMed
Roberts, C.M., Spitz, G., & Ponsford, J.L. (2016). Comparing prospectively recorded posttraumatic amnesia duration with retrospective accounts. Journal of Head Trauma Rehabilitation, 31(2), E71–E77.CrossRefGoogle ScholarPubMed
Ruff, R.M., Iverson, G.L., Barth, J.T., Bush, S.S., & Broshek, D.K. (2009). Recommendations for diagnosing a mild traumatic brain injury: A National Academy of Neuropsychology education paper. Archives of Clinical Neuropsychology, 24(1), 3–10. Retrieved from PM:19395352CrossRefGoogle ScholarPubMed
Ruff, R.M. & Jurica, P. (1999). In search of a unified definition for mild traumatic brain injury. Brain Injury, 13(12), 943–952. Retrieved from PM:10628500CrossRefGoogle ScholarPubMed
Sady, M.D., Vaughan, C.G., & Gioia, G.A. (2014). Psychometric characteristics of the postconcussion symptom inventory in children and adolescents. Archives of Clinical Neuropsychology, 29(4), 348–363. Retrieved from PM:24739735CrossRefGoogle ScholarPubMed
Sarmiento, K., Gioia, G.A., Kirkwood, M.W., Wade, S.L., & Yeates, K.O. (2019). A commentary for neuropsychologists on CDC’s guideline on the diagnosis and management of mild traumatic brain injury among children. Clinical Neuropsychology, 1–19. Retrieved from PM:31530221Google ScholarPubMed
Sherer, M., Sander, A.M., Maestas, K.L., Pastorek, N.J., Nick, T.G., & Li, J. (2015). Accuracy of self-reported length of coma and posttraumatic amnesia in persons with medically verified traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 96(4), 652–658. Retrieved from PM:25461819CrossRefGoogle ScholarPubMed
Shrout, P.E. & Fleiss, J.L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. Retrieved from PM:18839484CrossRefGoogle ScholarPubMed
Stuss, D.T., Binns, M.A., Carruth, F.G., Levine, B., Brandys, C.E., Moulton, R.J., … Schwartz, M.L. (1999). The acute period of recovery from traumatic brain injury: Posttraumatic amnesia or posttraumatic confusional state? Journal of Neurosurgery, 90(4), 635–643. Retrieved from PM:10193606CrossRefGoogle ScholarPubMed
Vanderlei, F.M., Barbosa, D.A., Machado, A.F., Bastos, F.N., Vanderlei, L.M., Junior, J.N., & Pastre, C.M. (2017). Analysis of recall bias of information on soccer injuries in adolescents. Motriz: Revista de Educação Física, 23(2), e101777.Google Scholar
Vanderploeg, R.D., Groer, S., & Belanger, H.G. (2012). Initial developmental process of a VA semistructured clinical interview for TBI identification. Journal of Rehabilitation Research and Development, 49(4), 545–556. Retrieved from PM:22773258CrossRefGoogle Scholar
Walker, W.C., Cifu, D.X., Hudak, A.M., Goldberg, G., Kunz, R.D., & Sima, A.P. (2015). Structured interview for mild traumatic brain injury after military blast: Inter-rater agreement and development of diagnostic algorithm. Journal of Neurotrauma, 32(7), 464–473. Retrieved from PM:25264909CrossRefGoogle ScholarPubMed
Wilmoth, K., LoBue, C., Clem, M.A., Reddy, R., Hynan, L.S., Didehbani, N., … Cullum, C.M. (2018). Consistency of traumatic brain injury reporting in older adults with and without cognitive impairment. The Clinical Neuropsychologist, 32(3), 524–529. doi: 10.1080/13854046.2017.1378371 CrossRefGoogle ScholarPubMed
Yeates, K.O., Taylor, H.G., Rusin, J., Bangert, B., Dietrich, A., Nuss, K., … Jones, B.L. (2009). Longitudinal trajectories of postconcussive symptoms in children with mild traumatic brain injuries and their relationship to acute clinical status. Pediatrics, 123(3), 735–743.CrossRefGoogle ScholarPubMed
Zemek, R.L., Grool, A.M., Rodriguez, D.D., DeMatteo, C., Rothman, L., Benchimol, E.I., … Macpherson, A.K. (2017). Annual and seasonal trends in ambulatory visits for pediatric concussion in Ontario between 2003 and 2013. The Journal of Pediatrics, 181, 222–228.e222. Retrieved from PM:27843008CrossRefGoogle ScholarPubMed
Supplementary material: File

Hergert et al. supplementary material

Hergert et al. supplementary material 1

Download Hergert et al. supplementary material(File)
File 41.7 KB
Supplementary material: File

Hergert et al. supplementary material

Hergert et al. supplementary material 2

Download Hergert et al. supplementary material(File)
File 41.9 KB