Scalable gaussian process inference of neural responses to natural images

Abstract

1Predicting the responses of sensory neurons is a long-standing neuroscience goal. However, while there has been much progress in modeling neural responses to simple and/or artificial stimuli, predicting responses to natural stimuli remains an ongoing challenge. One the one hand, deep neural networks perform very well on certain data-sets, but can fail when data is limited. On the other hand, gaussian processes (GPs) perform well on limited data, but are generally poor at predicting responses to high-dimensional stimuli, such as natural images. Here we show how structured priors, e.g. for local and smooth receptive fields, can be used to scale up GPs to high-dimensional stimuli. We show that when we do this, a GP model largely outperforms a deep neural network trained to predict retinal responses to natural images, with largest differences observed when both models are trained on a very small data-set. Further, since GPs compute the uncertainty in their predictions, they are well-suited to closed-loop experiments, where stimuli are chosen actively so as to collect informative neural data. We show how this can be done in practice on our retinal data-set, so as to: (i) efficiently learn a model of retinal responses to natural images, using little data, and (ii) rapidly distinguish between competing models (e.g. a linear vs a non-linear model). In the future, our approach could be applied to other low-level sensory areas, beyond the retina.