Perception of soft materials relies on physics-based object representations: Behavioral and computational evidence

When encountering objects, we readily perceive not only low-level properties (e.g., color and orientation), but also seemingly higher-level ones - including aspects of physics (e.g., mass). Perhaps nowhere is this contrast more salient than in the perception of soft materials such as cloths: the dynamics of these objects (including how their three-dimensional forms vary) are determined by their physical properties such as stiffness, elasticity, and mass. Here we hypothesize that the perception of cloths and their physical properties must involve not only image statistics, but also abstract object representations that incorporate "intuitive physics". We provide behavioral and computational evidence for this hypothesis. We find that humans can visually match the stiffness of cloths with unfamiliar textures from the way they undergo natural transformations (e.g. flapping in the wind) across different scenarios. A computational model that casts cloth perception as mental physics simulation explains important aspects of this behavior.