i Örebro University
School of Medicine Degree project 15 ECTS January 2017
Evaluation of the current
diagnostic tools for concussion
in sports
Version 2
Author: Johanna Wallinder
Supervisor: Pashtun Shahim MD PhD Department of Neurochemistry
Sahlgrenska University Hospital, Mölndal, Sweden
2
Abstract
Background: Concussion is a common injury in sports. The need for accurate diagnosis and follow up is critical to avoid too early return to play and minimize the number of repeated concussions obtained which can result in post concussions symptoms and chronic traumatic encephalopathy.
Purpose: The objective of this study was to examine the support for 3 diagnostic tools used for evaluation of sports-related concussion.
Methods: The tools investigated were Sports Associated Concussion Assessment Tool (SCAT3), Rivermead Post Concussion Symptoms Questionnaire (RPQ) and Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT). This was done by searching three databases, PubMED, PsychInfo and SportDiscus. The search words were the name of the specific test, validity and reliability.
Results: A total of 18 articles were selected, 1 for SCAT 3, 3 for RPQ and 14 for ImPACT. Of the studies evaluated in this study half (7/14) supported the reliability and validity of ImPACT and the other approximate half (6/14) did not support the reliability and validity of the
ImPACT. SCAT 3 only yielded one article that supported the use of SCAT 3, primarily within the first week of getting a concussion. Of the 3 articles yielded in RPQ search, two questioned the construct validity and the third article, which contained two studies supported it.
Conclusion: None of these diagnostic tests have an unquestionable support within current literature. Further research regarding the validity and reliability is needed to further develop the use and understanding of these tools.
3 Abbreviations:
mTBI-mild traumatic brain injury
SCAT3- Sport Concussion Assesment Tool 3
RPQ- Rivermead Post-Concussion Symptoms Questionnaire
ImPACT- Immediate Post-Concussion Assessment and Cognitive Testing SAC- Standardized Assessment of Concussion
PCS- Post-Concussion Syndrome, GPA-grade point average ADHD-attention deficit hyperactivity disorder
4
CONTENT
Background ... iv
Material and Methods ... iv
Diagnostic tests investigated ... iv
ImPact ... iv
SCAT 3 ... iv
RPQ ... v
Method... v
Inculsion and Exclusion Criteria ... v
Data extraction ... vi Results ... vi Discussion ... x Conclusion ... xii References ... xiii
BACKGROUND
Concussion is a common injury in contact sports such as boxing, American football, rugby and ice hockey. In the USA, the Centers for Disease Control estimate that 1,6-3,8 million sports-related concussions occur annually in the United States alone1,2. This figure is,
iii however, probably underestimated since not all who obtain a mild concussion seek medical care3.
Concussion is a type of mild traumatic brain injury (mTBI) and is defined as a complex pathophysiological disturbance of brain function induced by mechanical force or trauma2 4. When the brain is subjected to a rapid change of motion, as when receiving a tackle or hit to the head, it starts a cascade in the brain cells which includes a change in ionic
concentrations in the cell, glutamate release, energy crisis, axonal damage, cytoskeletal dysfunction and altered transmission. Inflammation and cell death has also been implicated as part of the pathophysiology of concussion5.
The symptoms of a concussion include fatigue, headaches, visual disturbances, loss of
memory, trouble concentrating/paying attention and emotional disturbances as irritability and depression.
At present, there are no valid blood biomarkers or imaging techniques for detection and diagnosis of concussion. The current clinical imaging biomarkers, such as computed tomography or magnetic resonance imaging may be performed to rule out intracranial
hemorrhage but contribute little to the diagnostic process. The concussion diagnosis is largely based on clinical symptoms. Behavioral and cognitive tests may be used to aid the diagnosis, but with limited specificity6.
Recent data suggest that athletes who have repeated concussions, especially unresolved concussions with early return to play may be at risk of developing both short-term and long-term neurological symptoms 2. Short-long-term risks include delayed reaction time which may increase risk of further injury and second impact syndrome which includes swelling of the brain and can therefore be lethal. It is, however, very uncommon7. Long-term risks are those associated with cognitive impairment and long term neurological consequences, however, more studies concerning this matter is needed.8
Considering the risks associated with concussions there is an urgent need for objective and reliable diagnostic and prognostic tools. At present, there are many qualitative and semi-qualitative tools used to aid in concussion diagnosis including Sports Associated Concussion Assessment Tool (SCAT3), Rivermead Post Concussion Symptoms Questionnaire (RPQ) and Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT).
iv The purpose of this paper was to investigate the evidence of reliability and validity of these three tests.
MATERIAL AND METHODS
DIAGNOSTIC TESTS INVESTIGATED
ImPact
ImPACT is a computerized concussion management tool and consists of 4 parts, verbal memory, visual memory, visual motor speed and reaction time. There is also a questionnaire covering self-reported symptoms. This tool is primarily utilized for baseline testing in the beginning of the season and post-injury testing. ImPACT was developed from the Pittsburg Steelers model which was a paper and pencil based test used to monitor Football players in the National Football League 9. The Pittsburg Steelers model was used until 1998. ImPACT baselines are usually administered during pre-season and follow up test 1-3 days after the injury 10.
SCAT 3
SCAT is a standardized test for evaluating athletes with concussion. It was first developed in 2005 and has since been edited and updated to SCAT2 and now SCAT3. The idea for
developing a standardized assessment tool was developed during the consensus meetings held 2001 in Vienna for medical professionals organized by International Ice Hockey Federation, Fédération Internationale de Football Association and the medical comission of the
International Olympic Committee. The latest version, SCAT3, was one of the outcomes from the consensus meeting in Zurich 20124.
SCAT3 is mainly used as a sideline assessment tool. Baseline values may or may not be obtained during preseason. It contains 8 major parts, Glasgow coma scale (testing alertness), Maddocks score (testing short term memory), self-reported symptoms, Cognitive assessment (using Standardized Assessment of Concussion, SAC.), neck examination, balance testing
v (using modified Balance Error Scoring System), coordination and delayed recall. The SCAT 3 form is found in appendix no. 1.
RPQ
Rivermead is a symptom questionnaire where symptoms are self-assessed on a scale of 0-4. The ratings are subjective comparisons of how the patient was feeling prior to the accident or within the past 24 hours. The symptoms covered by the questionnaire are headache, dizziness, nausea/vomiting, noise sensitivity, sleeping disturbance, fatigue, irritability, depression, frustration/patients, memory, concentration, thought processing, vision and restlessness. It was developed by N.S. King’s research group in 1995 and is designed to assess the presence of post-concussion syndrome11. Post-concussion syndrome, PCS, is a when symptoms such as headache, dizziness appear as a consequence of a mTBI. The RPQ questionnaire is found in appendix no. 2.
METHOD
Databases PubMed, PsychInfo, SportDiscus were searched using the name of the test (i.e. SCAT 3, Rivermead or ImPACT) together with realibility and validity.
Since this is a literature study there is no need for ethical considerations or statistical analysis.
Inculsion and Exclusion Criteria
1. Published Articles after 1990
2. Reported outcomes include an analysis of reliability and/or validity of SCAT 3, RPQ or ImPACT
3. Individuals older than 13years of age
4. Data collected either at baseline or postconcussion 5. English language
6. SCAT, RPQ or ImPACT was used to evaluate sports related concussions 7. Articles must be original studies therefore review articles were rejected
The inclusion criteria no. 6 was disregarded for RPQ since the search yielded no results related to concussions in sports.
vi
Data extraction
The articles were evaluated with help of the first section of “Mall för kvalitetsgranskning av observationsstudier” (the GRADE scale was not used) from SBU’s website. The form is found in appendix no. 3.
RESULTS
A total of 18 articles were selected, 1 for SCAT 3, 3 for RPQ and 14 for ImPACT. A table of all included articles is found in appendix no. 4
Figure 1: Search for SCAT 3
vii Figure 3: Search for ImPACT
viii
ImPACT
The search yielded 14 articles investigating the validity and reliability of the ImPACT. 10 were within subject cohort studies testing baseline re-testing reliability, three were cross sectional studies which tested the subject only once and one was a cohort study investigating differences between concussed and non-concussed athletes.
Of the studies evaluated in this study half (7/14) supported the reliability and validity of ImPACT 12-17,24 and the other approximate half (6/14) did not support the reliability and validity of the ImPACT 18-22,25. The reliability, that is that the test will score an athlete with similar results when tested several times, is tested primarily by baseline testing and retesting. The validity, meaning that the test measures what it is intended to measure, is harder to conduct studies since there is no gold standard to compare with. There are studies
comparing ImPACT to other previously validated tests24,25, but as previously mentioned none of the tests are considered gold standard for diagnosing concussion. The study
examining the rates and predictors for invalid baseline didn’t say whether it supported the test or not but examined what factors can influence baseline results other than the construct of the test19.
Baseline testing
There is a debate about whether comparison to baseline testing is the best way to evaluate the scores of ImPACT or if it is better to compare the scores obtained post-injury to normative data.
Many of the studies have investigated the test-retest reliability12-22, most of them only with healthy participants. Study participants have both been athletes and non-athletes12,13, of varying ages, from high school (13-18) and college students’ (18-24) to professional
players14-25. The results of these studies have, however, been inconclusive. Some studies have shown high test-retest reliability12-17,24 while others have shown poor reliability18-22,25. The author Schatz who has conducted several studies12,16 showing reliable test-retesting within-subjects have criticized, primarily one study showing unreliable test-test-retesting conducted by Broglio21, primarily because of the study method in investigating several neurocognitive tests during the same session. That is, however, not the only study showing varying results when it comes to testing the reliability of baseline testing 18-20, 22,25.
ix For baseline-testing to be useful there is need for a valid test-result from the athlete taking the baseline. ImPACT has as part of the data analysis a function for flagging tests as invalid showing “++” on the scorecard. That is, if an athlete would for example score 30 or more on Impulse Control composite their effort would be questioned25. Several predictors for obtaining an invalid baseline have been identified by Nelson et. al. Several predictors were studied such as lower GPA (grade point average), diagnosis such as ADHD (attention deficiency hyperactivity disorder) or learning disability and age. The only predictor for invalid baseline testing with ImPACT was lower age and that was only a modest association which became non-significant after adjustment19. There have also been other questions raised such as how primary language22, prior night’s sleep and so forth can affect the results of not only baseline but also re-testing26, most of these studies did not, however, turn up in the searches conducted in this study. Jared Bruce et. al22 raises the question of primary language in his study of ImPACT’s test-retest reliability in professional hockey players. The results indicate that the reliability may vary depending on language administration.
Follow up
The use of ImPACT as a tool to follow the concussed athlete’s recovery and as part of the return to play decision is also not entirely supported in the literature. One study conducted by L. D. Nelson et. al.23 where concussed athletes completed the ImPACT within 24 hours of injury and 8, 15 and 45 days’ post-injury and were compared with non-concussed, matched controls showed only significant differences within the first 24 hours’ and day 8 post-injury.
One study conducted by G. Iverson et. al. tested athletes within 21 days of sustaining a concussion and found that the ImPACT composite socres testing Processing Speed and Reaction Time did correlate with a traditional and previously validated test, the Symbol Digit Modalities Test 24.
SCAT 3
The search for articles investigating the reliability and validity yielded only one article, “Reliability and Validity of the Sport Concussion Assessment Tool -3 (SCAT3) in High School and Collegiate Athletes by Esther Y. Chin et. al. In the study, concussed athletes were
x matched with non-concussed controls and were evaluated 24 hours, 8, 15, and 45 days post-injury (healthy controls being evaluated at similar times. The results indicated that significant differences between concussed and healthy controls could be observed primarily 24 hours and 8 days post-injury27.
RPQ
The search for articles investigating the reliability and validity yielded three articles investigating the construct validity and reliability of the RPQ.
The construct validity of the RPQ, that is if the RPQ is constructed so that it measures what it is intended to, has been questioned by both S. Eyres et. al. and M. Lannsjö et. al. Both studies suggest that instead of using one composite score the questions should be divided in to two or more groups and these groups of questions be scored independent of each other since they don’t measure the same thing28-29.
The reliability has been tested, as far as the results of this study, only by the developer, N. King. There are two studies presented in one article, one where the RPQ was administered twice, once by a professional and once self-administered and one study where the RPQ was administered twice within one week as a 6 months’ post-injury follow up. Both studies were considered proving the reliability of the RPQ11 it is, however, questioned by S. Eyres et. al 29.
DISCUSSION
ImPACT
ImPACT is one of the most widely used concussion diagnostic/follow-up tools used in the USA. It is clear, however, that there is no consensus on the reliability and validity of this test. Studies show varied results both within and between studies.
The advantages of ImPACT are, it is easy to administer on the computer, it comes in different languages, and with the online versions scores can be accessed at any computer with access to
xi internet (older desktop versions are, however, bound to the computer). The different
translations of ImPACT are, however, not studied in depth in terms of reliability. Of note is also that most of the studies conducted have had any other primary language than English as an exclusion criteria12-16,20,21,25. Further research of the translations of the ImPACT is therefore warranted.
Another issue with baseline testing is the risk of some athletes intentionally trying to underperform on their baseline, so called “sandbagging”19 to be allowed to return to play sooner since their scores are seemingly not affected by the concussion. This search, however, did not include those articles and are there -fore not further discussed.
It is a factor to keep in mind that this test is developed by a company and is therefore a product made to be marketed and sold10. That can be both an advantage, they need a good product to be able to sell it and remain a good reputation. It can also be a disadvantage in that there is an interest in making the test look good and perhaps overstating its utility in order to market it and sell more.
SCAT 3
Since the search only generated one article it is hard to make any conclusive statements. SCAT 3 has many strengths such as being easy to administer on the side line by filling out the form with paper and pencil. It doesn’t require a computer or tablet and it assesses more than neurocognitive functions, for example consciousness and balance which are also affected by concussions. It is also developed at a consensus meeting by medical professionals and is available free on the internet, therefore there is no need to sell the test. It does, however, also have weaknesses in not being evaluated for how all these different tests are to be interpreted together. As far as this study could illustrate only one study has been made on the entire SCAT 3, not only the individual parts. There is also no visual testing which is a major part of other concussion diagnostic tests27.
xii
RPQ
The RPQ is not primarily used to diagnose concussion but is used to evaluate the presence of post concussive symptoms. The positive aspects is that it is easy to administer and seems to have a good reliability, whether self-administered or by a professional11. The reliability and validity is, however questioned by Eyres et. al. 29 due to the low fit to the statistical Rasch model used. It could also be questioned though because the studies with the greatest support for the reliability were performed by the developer of the test and therefore it could constitute a conflict of interest.
RPQ’s validity is however questionable since many post-concussive symptoms are vague and includes common symptoms such as headache, nausea and fatigue. The studies support that all these symptoms can’t be lumped together since the symptoms fit together in a more complex way28,29.
Limitations
One limitation of this study is that only one person reviewed the articles. Further evaluating the studies with traditional forms for evaluating diagnostic studies was not optimal since there is no test that is considered gold standard for concussions. This makes it difficult to design studies and to evaluate when you have nothing to compare the new diagnostic method to. The “Mall för kvalitetsgranskning av observationsstudier” wasn’t optimal either since many of the questions were not applicable or unclear, which made it hard to score the studies. Another more specific weakness of this study is that this study only searched studies assessing the whole SCAT 3 test. There are more studies1§ testing the individual parts of SCAT 3 which could be useful, however that doesn’t mean that the way all these tests are interpreted together as a whole needs to be further investigated. Also, SCAT 3 has been developed from the SCAT 2 which is similar to the SCAT 3
CONCLUSION
None of the tests discussed in this paper could be considered optimal for diagnosis and prognosis of concussion in athletes. One test should not be the single base of clinical decisions made. Consideration of the athletes’ background and circumstances during both baseline testing and follow up should be part of the return to play decision. Concussions are complex and the ramifications of too early return to play have not yet been fully understood.
xiii Also, the determination of what is considered too early return to play is not fully determined. Is it when symptoms have resolved or when the neurocognitive function is back to baseline values? Further research in this area is needed to fully understand this problem. The
development and research has also focused a lot on development of imaging methods and blood biomarkers which could also aid in diagnosis. Perhaps the best way for diagnosing is in a future where imaging techniques and neurocognitive testing can be combined to avoid the possible devastating ramifications of too many concussions or too early return to play.
ACKNOWLEDGEMENTS
Special thanks to my supervisor Pashtun Shahim for taking the time to help me write this paper, also to Yelverton Tegner who helped me get in touch with Pashtun.
REFERENCES
1. Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of traumatic brain injury: a brief overview. J Head Trauma Rehabil. 2006 Sep-Oct;21(5):375–378.
2. Harmon KG, Drezner JA, Gammons M, Guskiewicz KM, Halstead M, Herring SA, et al. American Medical Society for Sports Medicine position statement: concussion in sport. Br J Sports Med. 2013 Jan;47(1):15-26
3. U.S DEPARTMENT OF HELATH AND HUMAN SERVICES, Centers for Disease Control and Pervention, National Center for Injury Prevention and Control. Traumatic Brain Injury in the United States Emergency Department Visits, Hospitalizations and Deaths 2002-2006. Atlanta (GA): Centers for Disease Control and Prevention, National Center for Injury Prevention and Control; 2010.
4. McCrory P, Meeuwisse WH, Aubry M, Cantu RC, Dvořák J, Echemendia RJ et al. Consensus Statment on Concussion in Sport: the 4th Internation Concerence on Concussion in Sport held in Zurich, November 2012. J Athl Train. 2013 Jul-Aug;48(4):554-575
5. Giza CC, Hovda D. The New Neurometabolic Cascade of Concussion. Neurosurgery. 2014 Oct;75 (04):24-33
6. Svensson M, Bellander B-M, Undén J, Johnsson H. Traumatiska hjärnskador (TBI) hos vuxna Internet… Utgivningsort: Internetmedicin 2016 Updated 2016-09-19; cited 2016-11-30 Available from: http://www.internetmedicin.se/page.aspx?id=1178
xiv 7. Bey T, Ostick B. Second impact syndrome. West J Emerg Med. 2009;10(1):6–10) 8. Gaetz M, Weinberg H. Electrophyiological indices of persistent post-concussion
symptoms. Brain Inj. 2000 Sep; 14(9):815-832
9. Vanderbildt Medical School. The ImPACT Program: History and Development Internet. Nashville TN 2013 Cited 2016-12-02 Available from:
https://medschool.vanderbilt.edu/vscc-research/files/vscc-research/public_files/Lovell_Nashville_November2013_.pdf
10. ImPACT Applications, Inc. ImPACT Recourses for Parents and Athletes Internet. Pittsburg PA 2016 Cited 2016-12-02. Availabel from:
https://www.impacttest.com/audience/?parents-3
11. King NS, Crawford s, Wenden FJ, Moss NE, Wade DT. The Rivermead Post
Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. JNeurol. 1995 Sep; 242(9):587-592.
12. Schatz P, Ferris CS. One-Month Test—Retest Reliability of the ImPACT Test Battery. Arch Clin Neruopsychol. 2013 Aug; 28(5):499-504.
13. Nakayama Y, Covassin T, Schatz P, Nogle S, Kovan J. Examination of the Test— Retest Reliability of a Computerized Neurocognitive Test Battery. Am J Sports Med. 2015 Aug; 43(8):2018-26.
14. Tsushima WT, Siu AM, Pearce AM, Zhang G. Oshiro RS. Two-year Test—Retest Reliability of ImPACT in High School Athletes. Arch Clin Neruospychol. 2016 Feb:31(1):105-111.
15. Brett BL, Solomon GS. The Influenve of validity criteria on Immediate Post-Concussion Assessment and Covgnitive Testing (ImPACT) test—retest reliability among high school athletes. J Clin Exp Neuropsychol. 2016 Sep 2:1-10
16. Elbin R.J, Schatz P, Covassin T. One-Year Test-Retest Reliability of the Online Version of ImPACT in High School Athletes. Am J Sports Med. 2011 Nov; 39 (11):2319-2324.
17. Schatz P. Long-Term Test-Retest Reliability of Baseline Cognitive Assessments Using ImPACT. Am J Sports Med. 2010 Jan; 38(1):47-53.
18. Resch J, Driscoll A, McCaffrey, Brown C, Ferrara MS, Macciocchi S et. al. ImPACT Test-Retest Reliability: Reliably Unreliable? J Athl Train. 2013 Jul-Aug;48(4):506-511
xv 19. Nelson LD, LaRoche AA, Pfaller AY, Lerner EB, Hammeke TA, Randolph C et. al.
Prospective, Head-toHead Study of Three Computerized Neurocognitive Assesment Tools (CNTs): Reliability and Validity for the Assessment of Sport-Related
Concussion. J Int Neuropsychol Soc. 2016 Jan;22(1):24-37
20. Register.Mihalik JK, KontosDK, Guskiewicz KM, Mihalik JP, Conder R, Shields EW. Age-Related Differences and Reliability on Computerized and Paper-and-Pencil Neurocognitive Assessment Batteries. J Athl Train. 2012 May-Jun;47(3):297.305 21. Broglio SP, Ferrara MS, Macciocchi SN, Baumgartner TA, Elliot R. Test-Retest
Reliability of Computerized Concussion Assessment Programs. J Athl Train 2007 Oct-Dec;42(4)509-514
22. Bruce J, Echemendia R, Meeuwisse W, Compier P, Sisco A. 1 year test-retest reliability of ImPACT in professional ice hockey players. Clin Neuropsychol. 2013 Dec;28(1):14-25
23. Nelson LD, Pfaller AY, Rein LE, McCrea MA. Rates and Predictors of Invalid Baseline Test Performance in High School and Collegiate Athletes for 3
Computerized Neurocognitive Tests:ANAM, Axon Sports and ImPACT. Am J Sports Med. 2015 Aug;43(8);2018-2026
24. Iverson GK, Lovell MR, Collins MW. Validity of ImPACT for Measuring Processing Speed Following Sports-Related Concussion. J Clin Exp Neuropsycol. 2005
Aug;27(6):683-698.
25. Allen BJ, Gfeller JD. The Immediate Post-Concussion Assessment and Cognitive Testing battery and traditional neuropsychological measures: A construct and concurrent validity study. Brain Inj. 2011;25(2):179-191.
26. McClure DJ, Zuckerman SL, Kutscher SJ, Gregory AJ, Solomon GS. Baseline neurocognitive testing in sports-related concussion: the importance of a prior night’s sleep. Am J Sports Med. 2014 Feb; 42(2):472-478.
27. Chin EY, Nelson LD, Barr WB, McCrory P, McCrea MA. Reliability and Validity of the Sport Concussion Assessment Tool—3 (SCAT3) in High School and Collegiate Athletes. Am J Sports Med. 2016 Sep;44(9):2276-2285.
28. Lannsjö M, Borg J, Björklund G, Af Geijerstam JL, Lundgren.Nilsson A. J Rehabil Med. 2011 Nov;43(11):997-1002.
29. Eyres S, Carey A, Gilworth G, Neumann V, Tennant A. Construct validity and reliability of the Rivermead Post-Concussion Sumptoms Questionnaire. Clin Rehabil. 2005 Dec;19(8)878-887
xvi Appendix no. 1.
xx Appendix no. 2.
xxi Appendix no. 3.
ii Appendix no 4. Publication (Author, year, referencenumber) Type of Study Patient characteristics Exposure or intervention
Compared to Measure of effect Results Quality assessment Schatz P, 2010,
17
Case series College Athletes Baseline testing with ImPACT approximately 2 years apart Differences between individuals 2 baseline scores measured with Intra Class Correlation (ICC)
Baseline values remain fairly stable over a 2-year period
Risk for systemic bias: Moderately high Nelson LD, 2015, 23 Prospective head-to-head High School and collegiate athletes Concussed athletes completing ImPACT Non-concussed, matched controls completing ImPACT Reliable change indices (RCI) Computerized neurocognitive testing may contribute to identification within 24h of injury but little after symptom reliefe
Risk for systemic bias: Moderately high
iii Nelson LD, 2016, 19 Cross-sectional High School and collegiate athletes Baseline testing with ImPACT Number of baselines classified as invlaid Validity criteria may not be sensitive to detect the cause of invalid baseline scores. Risk for systemic bias: Moderately high Iverson GL, 2005, 24 Cross-sectional Amature athletes ImPACT testing post Injury Symbol Digit Modalities Test (SDMT) Correlation between the two test-scores
Speed and reaction time composite scores of ImPACT seem to measure similar construct as SDMT Risk for systemic bias: Moderately high Broglio SP, 2007, 21 Repeated-measure University students Baseline Test and Re-testing of ImPACT Differences between individuals 2 baseline scores measured with Intra Class Correlation (ICC)
Tests showed low to moderate test-retest reliability Risk for systemic bias: Low Bruce J, 2013, 22 Repeated-measure Professional ice hockey players Baseline Test and Re-testing of ImPACT Differences between individuals 2 baseline scores measured with Intra Class Correlation (ICC)
Mixed support for visual motor and reaction time composite scores, verbal and visual composite scores may not be sensitive to clinical change Risk for systemic bias: Low
iv Register-Mihalik, 2012, 20 Cohort study High school and collegiate athletes
Test and Re-testing of ImPACT
Differences in composite scores between age-groups and between test sessions Performance may vary between sessions Risk for systemic bias: Low Resch J, 2013, 18 Cross-sectional Men and women ages 18-24
Test and Re-testing of ImPACT
Differences
between individuals 2 baseline scores measured with Intra Class Correlation (ICC) Variable test-retest reliability. Risk for systemic bias: Moderately high Brett BL, 2016, 15 Repeated measure High school athletes
Test and re-testing of ImPACT
Differences
between individuals 2 baseline scores measured with Intra Class Correlation (ICC)
Application of strict validity criteria does not affect rest-retest reliability Risk for systemic bias: Moderately High Tsushima WT, 2016, 14 Repeated measure High school athletes
Test and re-testing of ImPACT
Differences
between individuals 2 baseline scores measured with Intra Class Correlation (ICC), Reliable change index(RCI)
Study indicated that baseline values were stable within a 2 year interval
Risk for systemic bias: Moderately High
v and regression based method(BM) Nakayama Y, 2014, 13 Repeated measure College students
Test and re-testing of ImPACT
Differences
between individuals 2 baseline scores measured with Intra Class Correlation (ICC),
Study indicated that ImPACT has good reliability Risk for systemic bias: Moderately High Allen BJ, 2011 Cross-sectional College students ImPACT testing NFL’s traditional neurospychological tests Factor analysis examined component structure. Correlations analyses examines relationships between variables within and in-between tests.
Study indicated that there is both overlapping in their constructs as well as notable differences Risk for systemic bias: Low
vi Schatz P, 2013,12 Repeated measure College students
Test and re-testing of ImPACT
Differences
between individuals 2 baseline scores measured with Intra Class Correlation (ICC) and Pearson’s correlation coefficients Repeated testing may lead to significant improvement on composites such as visual motor speed but does not lead to practice effect on memory or reaction time. Risk for systemic bias: Moderately High Elbin R.J, 2011, 16 Repeated measure High school athletes
Test and re-testing of ImPACT
Differences
between individuals 2 baseline scores measured with Intra Class Correlation (ICC)
Reliability of the online version of ImPACT is stable and better than the 2 year reported from desktop version- Risk for systemic bias: Low Eyres, 2004, 29 Cross-sectional Adults visiting the emergency room after head trauma Completing RPQ post injury RPQ score analysed with Rasch model.
RPQ is not up to modern psychometric standards. Risk for systemic bias: Moderately high
vii Lannsjö, 2011,28 Cross sectional Patients with mild traumatic brain injury (mTBI) visiting the emergency room, ages 6 years and older Completing RPQ post-injury RPQ score analysed with Rasch model
RPQ is not optimal for patients with mTBI. It is, however, unsensitive to age and gender. Risk of systemic bias: Low King NS, 1995, 11 Repeated measure Head injury patients Completing RPQ twice post-injury
Comparing the two RPQ scores with Spearman rank correlation RPQ shows good reliability Risk for systemic bias: Moderately high Chin EY, 2016 Cohort
diagnostic study High school and collegiat athletes Concussed athletes completing SCAT 3 at baseline and 4 times post injury Non-injured matched controls at same time as injured athletes. Comparing injured with non-injured matched controls with RCI When interpreting the SCAT3 results, individual differences should be considered Risk for systemic bias: Moderately high
