Valid, invalid, or somewhere in between? Baseline ImPACT and stand-alone performance validity testing in collegiate athletes

Abstract

ObjectiveBaseline cognitive testing is important for sport concussion management. Assessing baseline data for both the validity and reliability is vital to ensuring its utility. Stand-alone performance validity tests (PVT) and embedded validity indicators (EVI) (collectively called "validity measures" hereafter) are commonly used in neuropsychological testing and screening. This study further investigates efficient ways to assess validity of baseline cognitive testing. Methods231 NCAA athletes completed baseline assessment with ImPACT and one of two PVTs: Medical Symptom Validity Test or Rey Dot Counting Test. The relationship between scores on validity measures and cognitive performance, symptoms, and sleep was assessed. ResultsAthletes who failed two or more validity measures performed worse on the Visual Motor Speed Composite while athletes failing three or more measures performed worse on the Reaction Time Composite. Those athletes who failed validity measures and also reported poor sleep performed worse on both composites. Self-reported symptoms and hours of sleep were not independently associated with failing validity measures. Lastly, athletes with self-reported ADHD and/or LD were more likely to fail two+ validity measures (46% versus 27% of neurotypical athletes). ConclusionsConsistent with previous research, ImPACT Baseline++ only identified 1.7% of the samples data as invalid despite 4.8% of the dataset failing a combination of PVT and EVI and an additional 42.9% failing at least EVI alone. Results suggest that assessing validity on a continuum and using multiple validity measures may be useful to identify data validity that falls in the middle of the continuum. Public Significance StatementBaseline concussion testing is common and often mandated prior to sport participation, however, a baseline only has utility if it is both a reliable and valid representation of the athletes healthy and optimal functioning. This study adds to the growing body of literature demonstrating that baseline testing may frequently have questionable validity. It also provides support to the value of conceptualizing validity as a spectrum, rather than dichotomy and is the first to apply the concept to baseline concussion testing data.