Spatio-temporal transformers for decoding neural movement control

Abstract

Deep learning tools applied to high-resolution neurophysiological data have significantly progressed, offering enhanced decoding, real-time processing, and readability for practical applications. However, the design of artificial neural networks to analyze neural activity remains a challenge, requiring a delicate balance between efficiency in low-data regimes and the interpretability of the results. To this end, we introduce a novel specialized transformer architecture to analyze single-neuron spiking activity. We test our model on multi electrodes recordings from the dorsal premotor cortex (PMd) of non-human primates while performing a motor inhibition task. The proposed architecture provides a very early prediction of the correct movement direction - no later than 230ms after the Go signal presentation across animals - and can accurately forecast whether the movement will be generated or withheld before a Stop signal, unattended, is actually presented. We also analyze the internal dynamics of the model by computing the predicted correlations between time steps and between neurons at successive layers of the architecture. We find that their evolution mirrors previous theoretical analyses. Overall, our framework provides a comprehensive use case for the practical implementation of deep learning tools in motor control research.