Turning an object into a scene: buildings activate scene-selective visual cortex independently of visual features

Human visual cortex contains regions that selectively respond to both scenes and large objects, particularly buildings. The cortical overlap between buildings and scenes has been attributed to shared visual features (e.g., cardinal orientations, rectilinearity). Alternative accounts propose that buildings may also activate scene representations indirectly, independently of specific visual features, for example because buildings evoke a sense of space. Here, we tested for such feature-independent activation by comparing EEG and fMRI responses in human participants (both sexes) to buildings and visually-matched boxes, and relating these responses to scene-selective responses. Buildings and boxes were matched using image-based metrics, deep neural networks, and a perceptual similarity task. Time-resolved EEG decoding showed that buildings and boxes evoked discriminable responses from 360ms post-stimulus onset, incompatible with feedforward visual feature processing. Importantly, the building-box classifier generalized to discriminate scenes from chairs, providing the first EEG evidence for a representational overlap between buildings and scenes. Temporal generalization analyses further showed that the late building-selective response corresponded to an earlier scene-selective response, with a temporal offset of ~130ms. Finally, ultra-fast fMRI (TR=140ms) revealed that the scene-selective parahippocampal place area (PPA) similarly showed a feature-independent building-selective response that was delayed and prolonged relative to the scene-selective response. These results clarify the nature of building selectivity in visual cortex by showing that this selectivity is feature-independent and delayed relative to scene selectivity. We propose that intrinsic associations between object categories (e.g., buildings and scenes, tools and hands) may account for part of the categorical organization of ventral temporal cortex.