A Unified Account of Lightness Illusions via Edge-BasedReconstruction of Natural Images

Abstract

The human visual system transforms patterns of light into rich perceptual experiences, where what we see is a construction that goes beyond simple measurement. Lightness illusions--where identical parts of an image can appear dramatically different depending on context--provide a window into these processes. Here we leverage a deep learning framework to investigate the constructive processes that give rise to lightness illusions, introducing the core computational goal of edge-based image reconstruction. Specifically, we demonstrate that autoencoder models trained to reconstruct natural images based only on an edge-based image representation naturally recapitulate a wide range of lightness illusions, which were previously assumed to require distinct mechanisms, inference over lighting sources, and explicit three-dimensional scene representation. These results offer a simpler, unified account of diverse lightness phenomena as emerging naturally from surface filling-in mechanisms, and broadly provide a framework for understanding the computational principles that underlie our perception of the visual world. SIGNIFICANCE STATEMENTThe human visual system shows remarkably stable perception of objects under different viewing conditions, but it uses strategies that can be thwarted by clever visual illusions - for instance, the exact same object can appear as either white or black in different contexts. The most complex of these lightness illusions have long been taken as evidence that perception involves explicit inference about 3D scene geometry and lighting conditions. However, here we show that these illusions also emerge in deep learning models, trained simply to reconstruct natural images from sparse edge signals. Thus, our perception of the lightness of surfaces in our world may instead arise from a much more primitive computation -- reconstructing surface appearance from edge responses.