Data Heterogeneity Limits the Scaling Effect of Pretraining in Neural Data Transformers
Linxing Preston Jiang, Shirui Chen, Emmanuel Tanumihardja, Xiaochuang Han, Weijia Shi, Eric Shea‐Brown, Rajesh P. N. Rao
This study shows that data heterogeneity limits the scaling benefits of pretraining neural data transformers in neuroscience datasets, suggesting that careful session selection can improve performance.
bioRxiv · May 12, 2025 · PreprintNeuroscience