Target Identification Under High Levels of Amplitude, Size, Orientation and Background Uncertainty

Many natural tasks require the visual system to classify image patches accurately into target categories, including the category of no target. Natural target categories often involve high levels of within-category variability (uncertainty), making it challenging to uncover the underlying computational mechanisms. Here, we describe these tasks as identification from a set of exhaustive, mutually exclusive target categories, each partitioned into mutually exclusive subcategories. We derive the optimal decision rule and present a computational method to simulate performance for moderately large and complex tasks. We focus on the detection of an additive wavelet target in white noise with five dimensions of stimulus uncertainty: target amplitude, orientation, scale, background contrast, and spatial pattern. We compare the performance of the ideal observer with various heuristic observers. We find that a properly normalized heuristic MAX observer (SNN-MAX) approximates optimal performance. We also find that a convolutional neural network trained on this task approaches but does not reach optimal performance, even with extensive training. We measured human performance on a task with three of these dimensions of uncertainty (orientation, scale, and background pattern). Results show that the pattern of hits and correct rejections for the ideal and SNN-MAX observers (but not a simple MAX observer) aligns with the data. Additionally, we measured performance under low uncertainty (without scale and orientation uncertainty) and found that the effect of uncertainty on the performance is smaller than any of the models predicted. This smaller-than-expected effect can largely be explained by including biologically plausible levels of intrinsic position uncertainty. PrecisWe describe target identification tasks in terms of mutually exclusive categories and subcategories and derive the optimal decision rule. Simulations of ideal and heuristic observers were compared to human data under high and low levels of extrinsic uncertainty.