High-level prediction errors in low-level visual cortex

Abstract

Perception and behaviour are significantly moulded by expectations derived from our prior knowledge. Hierarchical predictive processing theories provide a principled account of the neural mechanisms underpinning these processes, casting perception as a hierarchical inference process. While numerous studies have shown stronger neural activity for surprising inputs, in line with this account, it is unclear what predictions are made across the cortical hierarchy, and therefore what kind of surprise drives this upregulation of activity. Here we leveraged fMRI and visual dissimilarity metrics derived from a deep neural network to arbitrate between two hypotheses: prediction errors may signal a local mismatch between input and expectation at each level of the cortical hierarchy, or prediction errors may incorporate feedback signals and thereby inherit complex tuning properties from higher areas. Our results are in line with this second hypothesis. Prediction errors in both low- and high-level visual cortex primarily scaled with high-level, but not low-level, visual surprise. This scaling with high-level surprise in early visual cortex strongly diverges from feedforward tuning, indicating a shift induced by predictive contexts. Mechanistically, our results suggest that high-level predictions may help constrain perceptual interpretations in earlier areas thereby aiding perceptual inference. Combined, our results elucidate the feature tuning of visual prediction errors and bolster a core hypothesis of hierarchical predictive processing theories, that predictions are relayed top-down to facilitate perception.