Material category of visual objects computed from specular image structure

Recognising materials and their properties from visual information is vital for successful interactions with our environment, from avoiding slippery floors to handling fragile objects. Yet there is no simple mapping of retinal image intensities to the physical properties that define materials. While studies have investigated how material properties like surface gloss are perceived from regularities in image structure, such as the size, sharpness, contrast, and position of bright patches caused by specular reflections, little is known how this translates to the recognition of different material classes like plastic, pearl, satin, or steel, and the underlying mechanisms involved. We investigated this by collecting human psychophysical judgments about complex glossy objects rendered in natural illumination fields. We found that variations in specular image structure - produced either by different reflectance properties or direct manipulation of image features - caused categorical shifts in material appearance, suggesting that specular reflections provide diagnostic information about a wide range of material classes, including many that should be defined by more complex scattering functions. Moreover, differences in material category were predicted by, but also appeared to mediate, cues for surface gloss, providing evidence against a traditional feedforward view of neural processing that assumes combinations of mid-level properties mediate our holistic, categorical impressions. Instead, our results suggest that the image structure that triggers our perception of surface gloss plays a direct role in visual categorisation and, importantly, that the perception and neural processing of stimulus properties should not be studied in isolation but rather in the context of recognition.