Human EEG and artificial neural networks reveal disentangled representations and processing timelines of object real-world size and depth in natural images

Abstract

Remarkably, human brains have the ability to accurately perceive and process the real-world size of objects, despite vast differences in distance and perspective. While previous studies have delved into this phenomenon, distinguishing the processing of real-world size from other visual properties, like depth, has been challenging. Using the THINGS EEG2 dataset with human EEG recordings and more ecologically valid naturalistic stimuli, our study combines human EEG and representational similarity analysis to disentangle neural representations of object real-world size from retinal size and perceived depth, leveraging recent datasets and modeling approaches to address challenges not fully resolved in previous work. We report a representational timeline of visual object processing: object real-world depth processed first, then retinal size, and finally, real-world size. Additionally, we input both these naturalistic images and object-only images without natural background into artificial neural networks. Consistent with the human EEG findings, we also successfully disentangled representation of object real-world size from retinal size and real-world depth in all three types of artificial neural networks (visual-only ResNet, visual-language CLIP, and language-only Word2Vec). Moreover, our multi-modal representational comparison framework across human EEG and artificial neural networks reveals real-world size as a stable and higher-level dimension in object space incorporating both visual and semantic information. Our research provides a temporally resolved characterization of how certain key object properties - such as object real-world size, depth, and retinal size - are represented in the brain, which offers further advances and insights into our understanding of object space and the construction of more brain-like visual models.