Energy saving in vision at the first synapse: The ON and OFF pathways measure temporal differences

The inner plexiform layer (IPL) of mammalian retina has a precise bisublaminar organization in an inner on- and an outer off-layer, innervated by spatially segregated on- and off-cone bipolar cell inputs. Also, the processes of starburst amacrine cells are segregated into on and off sublaminae of the IPL. Distances between overlapping on-off pair retinal ganglion cell dendritic tree centers are markedly smaller than between on-on or off-off centers, indicating simultaneously sampling the same space. Despite dekades of research, no good model exists for the role of the on- and off pathways. Here I propose that the on- and off pairs are temporally subtracted, with one channel delayed in time, likely in a higher cortical center. The on- and off receptive fields give at every retinal location an I+ and I-signal, where I is intensity, velocity, color. Subsequent frame subtraction is a basis function of every surveillance camera for vision, and in MPEG video/sound compression. The model explains the many phenomena observed when the retinal image is stabilized. The separation of layers in the LGN fits with the notion of a time delay at higher cortical level. The directionalty observed in micro-saccades is typically perpendicular to the main edges in the scene. Precise measurement of spatio-temporal receptive field kernels shows that time is processed in the visual system as a real-time process, i.e. with a logarithmic time axis. As only contours and textures are transmitted, it is a very effective design strategy of the visual system to conserve energy, in a brain that typically uses 25 Watt and very low neuron firing frequencies. The higher visual centers perform the fill-in (inpainting) with such efficiency, that the subtraction always goes unnoticed.