Hidden Aspects of the Research-ADOS are Bound to Affect Autism Science

The research-grade ADOS is a broadly used instrument that informs and steers much of the science of Autism. Despite its broad use, little is known about the empirical variability inherently present in the scores of the ADOS scale, or their appropriateness to define change, to repeatedly use this test to characterize neurodevelopmental trajectories. Here we examine the empirical distributions of research-grade ADOS scores from 1,324 records in a cross-section of the population comprising participants with autism between 5-65 years of age. We find that these empirical distributions violate the theoretical requirements of normality and homogeneous variance, essential for independence between bias and sensitivity. Further, we assess a subset of 52 typical controls vs. those with autism and find lack of proper elements to characterize neurodevelopmental trajectories in a coping nervous system changing at non-uniform, non-linear rates. Lastly, longitudinally repeating the assessments over 4 visits in a subset of the participants with autism for whom verbal criteria kept the same appropriate ADOS modules over the timespan of the 4 visits, reveals that switching the clinician, changes the cutoff scores, and consequently, influences the diagnosis, despite maintaining fidelity in the same tests modules, room conditions and tasks fluidity per visit. Given the changes in probability distribution shape and dispersion of these ADOS scores, the lack of appropriate metric spaces, and the impact that these elements have on sensitivity-bias co-dependencies, and on longitudinal tracking of autism, we invite a discussion on the use of this test for scientific purposes.