Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2024-12-23T05:00:29.039Z Has data issue: false hasContentIssue false
  • English
  • Français

Test–Retest Reliability of Concussion Baseline Assessments in United States Service Academy Cadets: A Report from the National Collegiate Athletic Association (NCAA)–Department of Defense (DoD) CARE Consortium

Published online by Cambridge University Press:  16 June 2020

Megan N. Houston Open the ORCID record for Megan N. Houston [Opens in a new window] ,
Kathryn L. Van Pelt ,
Christopher D’Lauro ,
Rachel M. Brodeur ,
Darren E. Campbell ,
Gerald T. McGinty ,
Jonathan C. Jackson ,
Tim F. Kelly ,
Karen Y. Peck  and
Steven J. Svoboda
...Show all authors
Megan N. Houston*
Affiliation:
John A. Feagin Jr. Sports Medicine Fellowship, Keller Army Hospital, West Point, NY10996, USA
Kathryn L. Van Pelt
Affiliation:
University of Kentucky, Lexington, KY40526, USA
Christopher D’Lauro
Affiliation:
United States Air Force Academy, Colorado Springs, CO80840, USA
Rachel M. Brodeur
Affiliation:
United States Coast Guard Academy, New London, CT06320, USA
Darren E. Campbell
Affiliation:
Logan Regional Orthopedics, Logan, UT84341, USA
Gerald T. McGinty
Affiliation:
United States Air Force Academy, Colorado Springs, CO80840, USA
Jonathan C. Jackson
Affiliation:
United States Air Force Academy, Colorado Springs, CO80840, USA
Tim F. Kelly
Affiliation:
United States Military Academy, West Point, NY10996, USA
Karen Y. Peck
Affiliation:
United States Military Academy, West Point, NY10996, USA
Steven J. Svoboda
Affiliation:
MedStar Orthopaedic Institute, Washington, DC20036, USA
Thomas W. McAllister
Affiliation:
Department of Psychiatry, Indiana University, Indianapolis, IN46202, USA
Michael A. McCrea
Affiliation:
Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, WI53226, USA
Steven P. Broglio
Affiliation:
Michigan Concussion Center, University of Michigan, Ann Arbor, MI48109, USA
Kenneth L. Cameron
Affiliation:
John A. Feagin Jr. Sports Medicine Fellowship, Keller Army Hospital, West Point, NY10996, USA
*
*Correspondence and reprint requests to: Megan N. Houston, John A. Feagin Jr. Sports Medicine Fellowship, Keller Army Hospital, 900 Washington Road, West Point, NY10996, USA. Tel: +1 845 938 6826. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

In response to advancing clinical practice guidelines regarding concussion management, service members, like athletes, complete a baseline assessment prior to participating in high-risk activities. While several studies have established test stability in athletes, no investigation to date has examined the stability of baseline assessment scores in military cadets. The objective of this study was to assess the test–retest reliability of a baseline concussion test battery in cadets at U.S. Service Academies.

Methods:

All cadets participating in the Concussion Assessment, Research, and Education (CARE) Consortium investigation completed a standard baseline battery that included memory, balance, symptom, and neurocognitive assessments. Annual baseline testing was completed during the first 3 years of the study. A two-way mixed-model analysis of variance (intraclass correlation coefficent (ICC)3,1) and Kappa statistics were used to assess the stability of the metrics at 1-year and 2-year time intervals.

Results:

ICC values for the 1-year test interval ranged from 0.28 to 0.67 and from 0.15 to 0.57 for the 2-year interval. Kappa values ranged from 0.16 to 0.21 for the 1-year interval and from 0.29 to 0.31 for the 2-year test interval. Across all measures, the observed effects were small, ranging from 0.01 to 0.44.

Conclusions:

This investigation noted less than optimal reliability for the most common concussion baseline assessments. While none of the assessments met or exceeded the accepted clinical threshold, the effect sizes were relatively small suggesting an overlap in performance from year-to-year. As such, baseline assessments beyond the initial evaluation in cadets are not essential but could aid concussion diagnosis.

Keywords

Mild traumatic brain injuryNeuropsychological testsBalanceSymptomsNeurocognitiveChange score
Type
Regular Research
Information
Journal of the International Neuropsychological Society , Volume 27 , Issue 1 , January 2021 , pp. 23 - 34
Copyright
Copyright © INS. Published by Cambridge University Press, 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Barr, W.B. & McCrea, M. (2001). Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. Journal of the International Neuropsychological Society, 7(6), 693–702.CrossRefGoogle ScholarPubMed
Barth, J.T., Alves, W.M., Ryan, T.V., Macciocchi, S.N., Rimel, R.W., Jane, J.A., & Nelson, W.E. (1989). Mild head injury in sports: Neuropsychological sequelae and recovery of function. Mild Head Injury, 257–275.Google Scholar
Bell, D.R., Guskiewicz, K.M., Clark, M.A., & Padua, D.A. (2011). Systematic review of the balance error scoring system. Sports Health, 3(3), 287–295.CrossRefGoogle ScholarPubMed
Broglio, S.P., Cantu, R.C., Gioia, G.A., Guskiewicz, K.M., Kutcher, J., Palm, M., & Valovich McLeod, T.C. (2014). National Athletic Trainers’ Association position statement: Management of sport concussion. Journal of Athletic Training, 49(2), 245–265.CrossRefGoogle ScholarPubMed
Broglio, S.P., Ferrara, M.S., Macciocchi, S.N., Baumgartner, T.A., & Elliott, R. (2007). Test-retest reliability of computerized concussion assessment programs. Journal of Athletic Training, 42(4), 509–514.Google ScholarPubMed
Broglio, S.P., Ferrara, M.S., Sopiarz, K., & Kelly, M.S. (2008). Reliable change of the sensory organization test. Clinical Journal of Sport Medicine, 18(2), 148–154.CrossRefGoogle ScholarPubMed
Broglio, S.P., Harezlak, J., Katz, B., Zhao, S., McAllister, T., McCrea, M., & Investigators, C.C. (2019). Acute sport concussion assessment optimization: A prospective assessment from the CARE consortium. Sports Medicine, 49(12), 1977–1987.CrossRefGoogle ScholarPubMed
Broglio, S.P., Katz, B.P., Zhao, S., McCrea, M., & McAllister, T. (2018). Test-retest reliability and interpretation of common concussion assessment tools: Findings from the NCAA-DoD CARE consortium. Sports Medicine, 48(5), 1255–1268.CrossRefGoogle ScholarPubMed
Broglio, S.P., McCrea, M., McAllister, T., Harezlak, J., Katz, B., Hack, D., & Hainline, B. (2017). A national study on the effects of concussion in collegiate athletes and US Military Service Academy members: The NCAA-DoD concussion assessment, research and education (CARE) consortium structure and methods. Sports Medicine, 47(7), 1437–1451.CrossRefGoogle Scholar
Broglio, S.P., Zhu, W., Sopiarz, K., & Park, Y. (2009). Generalizability theory analysis of balance error scoring system reliability in healthy young adults. Journal of Athletic Training, 44(5), 497–502.CrossRefGoogle ScholarPubMed
Cameron, K.L., Marshall, S.W., Sturdivant, R.X., & Lincoln, A.E. (2012). Trends in the incidence of physician-diagnosed mild traumatic brain injury among active duty U.S. military personnel between 1997 and 2007. J Neurotrauma, 29(7), 1313–1321.CrossRefGoogle ScholarPubMed
Centers for Disease Control and Prevention. (2016). Basic information about traumatic brain injury and concussion. Retrieved from http://www.cdc.gov/traumaticbraininjury/basics.htmlGoogle Scholar
Chan, M., Vielleuse, J.V., Vokaty, S., Wener, M.A., Pearson, I., & Gagnon, I. (2013). Test-retest reliability of the sport concussion assessment tool 2 (SCAT2) for uninjured children and young adults. British Journal of Sports Medicine, 47(5), e1.CrossRefGoogle Scholar
Chin, E.Y., Nelson, L.D., Barr, W.B., McCrory, P., & McCrea, M.A. (2016). Reliability and validity of the Sport Concussion Assessment Tool-3 (SCAT3) in high school and collegiate athletes. The American Journal of Sports Medicine, 44(9), 2276–2285.CrossRefGoogle ScholarPubMed
Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Rev. ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
Cole, W.R., Arrieux, J.P., Schwab, K., Ivins, B.J., Qashu, F.M., & Lewis, S.C. (2013). Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population. Archives of Clinical Neuropsychology, 28(7), 732–742.CrossRefGoogle Scholar
Defense and Veterans Brain Injury Center. (2018). Department of Defense worldwide numbers for traumatic brain injury. Retrieved from http://dvbic.dcoe.mil/files/tbi-numbers/WorldwideTotals2000-2017Q1-Q3Nov%2014-2017508.pdfGoogle Scholar
Department of Defense. (2015). Department of Defense Instruction 6490.13: Comprehensive policy on traumatic brain injury-related neurocognitive assessments by the military services. Retrieved from https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/649013p.pdfGoogle Scholar
Farnsworth, J.L., Dargo, L., Ragan, B.G., & Kang, M. (2017). Reliability of computerized neurocognitive tests for concussion assessment: A meta-analysis. Journal of Athletic Training, 52(9), 826–833.CrossRefGoogle ScholarPubMed
Grilo, L., & Grilo, H. (2012). Comparison of clinical data based on limits of agreement. Biomedical Letters, 49(1), 45–56.Google Scholar
Herring, S.A., Cantu, R.C., Guskiewicz, K.M., Putukian, M., Kibler, W.B., Bergfeld, J.A., … Indelicato, P.A. (2011). Concussion (mild traumatic brain injury) and the team physician: A consensus statement – 2011 update. Med Sci Sports Exerc, 43(12), 2412–2422.Google Scholar
Hinton-Bayre, A.D., Geffen, G.M., Geffen, L.B., McFarland, K.A., & Frijs, P. (1999). Concussion in contact sports: Reliable change indices of impairment and recovery. Journal of Clinical and Experimental Neuropsychology, 21(1), 70–86.CrossRefGoogle ScholarPubMed
Hoffman, J.I. (2015). Linear regression. In Hoffman, J.I. (Ed.), Biostatistics for medical and biomedical practitioners (pp. 451–500). Amsterdam: Elsevier.CrossRefGoogle Scholar
Iverson, G.L. (2011). Reliable change index. In Kreutzer, J.S., DeLuca, J., & Caplan, B. (Eds.), Encyclopedia of clinical neuropsychology (pp. 2150–2153). New York, NY: Springer.CrossRefGoogle Scholar
Iverson, G.L., Lovell, M.R., & Collins, M.W. (2003). Interpreting change on ImPACT following sport concussion. The Clinical Neuropsychologist, 17(4), 460–467.CrossRefGoogle ScholarPubMed
Koo, T.K., & Li, M.Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.CrossRefGoogle ScholarPubMed
Lancaster, M.A., McCrea, M.A., & Nelson, L.D. (2016). Psychometric properties and normative data for the Brief Symptom Inventory-18 (BSI-18) in high school and collegiate athletes. The Clinical Neuropsychologist, 30(2), 338–350.CrossRefGoogle ScholarPubMed
Langlois, J.A., Rutland-Brown, W., & Wald, M.M. (2006). The epidemiology and impact of traumatic brain injury: A brief overview. Journal of Head Trauma Rehabilitation, 21(5), 375–378.CrossRefGoogle ScholarPubMed
Leahy, S. (2011, April 27). Peyton Manning admits to tanking NFL’s baseline concussion test. USA Today.Google Scholar
Lichtenstein, J.D., Moser, R.S., & Schatz, P. (2014). Age and test setting affect the prevalence of invalid baseline scores on neurocognitive tests. The American Journal of Sports Medicine, 42(2), 479–484.CrossRefGoogle ScholarPubMed
McCrea, M., Kelly, J.P., Randolph, C., Kluge, J., Bartolic, E., Finn, G., & Baxter, B. (1998). Standardized assessment of concussion (SAC): On-site mental status evaluation of the athlete. Journal of Head Trauma Rehabilitation, 13(2), 27–35.CrossRefGoogle ScholarPubMed
McCrory, P., Meeuwisse, W., Dvorak, J., Aubry, M., Bailes, J., Broglio, S., …Vos, P.E. (2017). Consensus statement on concussion in sport-the 5th international conference on concussion in sport held in Berlin, October 2016. British Journal of Sports Medicine, 51(11), 838–847.Google Scholar
McCrory, P., Meeuwisse, W.H., Aubry, M., Cantu, B., Dvorak, J., Echemendia, R.J., Engebretsen, L., … Turner, M. (2013). Consensus statement on concussion in sport: The 4th International Conference on Concussion in Sport held in Zurich, November 2012. British Journal of Sports Medicine, 47(5), 250–258.CrossRefGoogle Scholar
McLeod, T.C., & Leach, C. (2012). Psychometric properties of self-report concussion scales and checklists. Journal of Athletic Training, 47(2), 221–223.CrossRefGoogle ScholarPubMed
Meachen, S.J., Hanks, R.A., Millis, S.R., & Rapport, L.J. (2008). The reliability and validity of the brief symptom inventory-18 in persons with traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 89(5), 958–965.CrossRefGoogle ScholarPubMed
Mehta, S., Bastero-Caballero, R.F., Sun, Y., Zhu, R., Murphy, D.K., Hardas, B., & Koch, G. (2018). Performance of intraclass correlation coefficient (ICC) as a reliability index under various distributions in scale reliability studies. Statistics in Medicine, 37(18), 2734–2752.CrossRefGoogle ScholarPubMed
Nelson, L.D., LaRoche, A.A., Pfaller, A.Y., Lerner, E.B., Hammeke, T.A., Randolph, C., … McCrea, M.A. (2016). Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): Reliability and validity for the assessment of sport-related concussion. Journal of the International Neuropsychological Society, 22(1), 24–37.CrossRefGoogle ScholarPubMed
Portney, L.G., & Watkins, M.P. (2009). Foundations of clinical research: Applications to practice. Upper Saddle River, NJ: Prentice Hall.Google Scholar
Ratcliff, R., Smith, P.L., Brown, S.D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20(4), 260–281.CrossRefGoogle ScholarPubMed
Register-Mihalik, J.K., Guskiewicz, K.M., Mihalik, J.P., Schmidt, J.D., Kerr, Z.Y., & McCrea, M.A. (2013). Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery: Implications for caution in clinical practice. Journal of Head Trauma Rehabilitation, 28(4), 274–283.CrossRefGoogle ScholarPubMed
Resch, J., Driscoll, A., McCaffrey, N., Brown, C., Ferrara, M.S., Macciocchi, S., … & Walpert, K. (2013). ImPact test-retest reliability: Reliably unreliable? Journal of Athletic Training, 48(4), 506–511.CrossRefGoogle ScholarPubMed
Riemann, B.L., Guskiewicz, K.M., & Shields, E.W. (1999). Relationship between clinical and forceplate measures of postural stability. Journal of Sport Rehabilitation, 8(2), 71–82.CrossRefGoogle Scholar
Shrout, P.E., & Fleiss, J.L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.CrossRefGoogle ScholarPubMed
Snell, F.I., & Halter, M.J. (2010). A signature wound of war: Mild traumatic brain injury. Journal of Psychosocial Nursing and Mental Health Services, 48(2), 22–28.CrossRefGoogle ScholarPubMed
Valovich McLeod, T.C., Barr, W.B., McCrea, M., & Guskiewicz, K.M. (2006). Psychometric and measurement properties of concussion assessment tools in youth sports. Journal of Athletic Training, 41(4), 399–408.Google ScholarPubMed
Supplementary material: File

Houston et al. supplementary material

Houston et al. supplementary material

Download Houston et al. supplementary material(File)
File 140.6 KB