The scope and limits of fine-grained image and category information in the ventral visual pathway

Abstract

Humans can easily abstract incoming visual information into discrete semantic categories. Previous research employing functional MRI (fMRI) in humans has identified cortical organizing principles that allow not only for coarse-scale distinctions such as animate versus inanimate objects but also more fine-grained distinctions at the level of individual objects. This suggests that fMRI carries rather fine-grained information about individual objects. However, most previous work investigating fine-grained category representations either additionally included coarse-scale category comparisons of objects, which confounds fine-grained and coarse-scale distinctions, or only used a single exemplar of each object, which confounds visual and semantic information. To address these challenges, here we used multisession human fMRI (female and male) paired with a broad yet homogenous stimulus class of 48 terrestrial mammals, with 2 exemplars per mammal. Multivariate decoding and representational similarity analysis (RSA) revealed high image-specific reliability in low- and high-level visual regions, indicating stable representational patterns at the image level. In contrast, analyses across exemplars of the same animal yielded only small effects in the lateral occipital complex (LOC), indicating rather subtle category effects in this region. Variance partitioning with a deep neural network and shape model showed that across exemplar effects in EVC were largely explained by low-level visual appearance, while representations in LOC appeared to also contain higher category-specific information. These results suggest that representations typically measured with fMRI are dominated by image-specific visual or coarse-grained category information but indicate that commonly employed fMRI protocols may reveal subtle yet reliable distinctions between individual objects. Significance StatementWhile it has been suggested that functional MRI (fMRI) responses in ventral visual cortex carry fine-grained information about individual objects, much previous research has confounded fine-grained with coarse-scale category information or only used individual visual exemplars, which potentially confounds semantic and visual object information. Here we address these challenges in a multisession fMRI study where participants viewed a highly homogenous stimulus set of 48 land mammals with 2 exemplars per animal. Our results reveal a strong dominance of image-specific effects and additionally indicate subtle yet reliable category-specific effects in lateral occipital complex, underscoring the capacity of commonly employed fMRI protocols to uncover fine-grained visual information.