Deep-learning-based fMRI decoding of real-world size for hand-held objects

Abstract

Real-world object size is a fundamental dimension of visual cognition, supporting effective interaction with the environment and object manipulation. However, neural mechanisms encoding size have largely been inferred from extreme size comparisons, leaving the neural representation of subtle size differences within a manipulable, "hand-scale" range poorly understood. Here, we applied a three-dimensional deep neural network (3D DNN) to decode real-world size from whole-brain fMRI data (N = 50) using objects that all fall within a graspable range. The 3D DNN successfully decoded subtle size differences, achieving predictive accuracies comparable to multivariate pattern analysis. Crucially, Guided Gradient-weighted Class Activation Mapping (Guided Grad-CAM) revealed that discriminative size information was not confined to the ventral occipito-temporal cortex but extended to a distributed network. Notably, these regions spatially overlap with the specific brain areas implicated in size-perception distortions following brain damage. Our findings suggest that the brain represents subtle variations in object size through a non-linear, distributed network that transcends the traditional visual hierarchy. Specifically, this system likely orchestrates the integration of visual properties with semantic scaling and the multimodal convergence of vision, space, and memory.