Neural representations in visual and parietal cortex differentiate between imagined, perceived, and illusory experiences

Abstract

Humans constantly receive massive amounts of information, both perceived from the external environment and imagined from the internal world. To function properly, the brain needs to correctly identify the origin of information being processed. Recent work has suggested common neural substrates for perception and imagery. However, it has remained unclear how the brain differentiates between external and internal experiences with shared neural codes. Here we tested this question by systematically investigating the neural processes underlying both the generation and maintenance of information from voluntary imagery, veridical perception, and illusion. The inclusion of illusion allowed us to differentiate between objective and subjective internality: while illusion has an objectively internal origin and can be viewed as involuntary imagery, it is also subjectively perceived as having an external origin like perception. Combining fMRI, eye-tracking, multivariate decoding and encoding approaches, we observed superior orientation representations in parietal cortex during imagery compared to perception, and conversely in early visual cortex. This imagery dominance gradually developed along a posterior-to-anterior cortical hierarchy from early visual to parietal cortex, emerged in the early epoch of imagery and sustained into the delay epoch, and persisted across varied imagined contents. Moreover, representational strength of illusion was more comparable to imagery in early visual cortex, but more comparable to perception in parietal cortex, suggesting content-specific representations in parietal cortex differentiate between subjectively internal and external experiences, as opposed to early visual cortex. These findings together support a domain-general engagement of parietal cortex in the generation and maintenance of internally-generated experience. Significance StatementHow does the brain differentiate between imagined and perceived experiences? Combining fMRI, eye-tracking, multivariate decoding and encoding approaches, the current study revealed enhanced stimulus-specific representations in visual imagery originating from IPS, supporting the subjective experience of imagery. This neural principle was further validated by evidence from visual illusion, wherein illusion resembled perception and imagery at different levels of cortical hierarchy. Our findings provide direct evidence for the critical role of parietal cortex as a domain-general source region for the generation and maintenance of content-specific imagery, and offer new insights into the neural mechanisms underlying the differentiation between subjectively internal and external experiences.