Convolutional neural network models describe the encoding subspace of local circuits in auditory cortex

Abstract

Convolutional neural networks (CNNs) provide powerful models of neural sensory encoding, but their complexity makes it difficult to discern computations that support their performance. To address this limitation, we developed a linear-nonlinear subspace model that identifies the most informative sensory dimensions captured by a CNN. A CNN was trained on single-neuron data recorded from auditory cortex of ferrets during presentation of a large natural sound set. Each neurons linear tuning subspace was computed by applying dimensionality reduction to the gradient of CNN output relative to input. Subspace projections were combined nonlinearly to predict neural activity. The resulting model was functionally equivalent to the CNN. Analysis of trained models showed that responses of local neural populations sparsely tiled a shared stimulus subspace. Encoding properties also differed between cell types and layers, reflecting their position in the cortical circuit. More generally, these results establish a framework for interpreting deep learning-based encoding models. Significance statementAuditory cortex mediates the representation and discrimination of complex sound features. Many models have been proposed for cortical sound encoding, varying in their generality, interpretability, and ease of fitting. It has been difficult to determine if and what different functional properties are captured by different models. This study shows that two families of encoding models, convolutional neural networks (CNNs) and tuning subspace models, account for the same functional properties, providing an important analytical link between accurate models that are easy to fit (CNNs) and models that are straightforward to interpret (tuning subspace).