Real-world visual search goes beyond eye movements: Active searchers select 3D scene viewpoints too

Abstract

Visual search is a ubiquitous task; people search for objects on a daily basis. However, the majority of the existing visual search literature focuses on passive search on a 2D computer screen, a far cry from emulating a real-world environment. Search is a real-world task that involves active observation. Search targets may be occluded, completely out of the observers line of sight, or oriented in unconventional ways. This is typically mitigated by actively selecting viewpoints, an important aspect of search behaviour with limited scope on a computer screen. Our goal was to explore viewpoint selection in active visual search. Subject eye and head movements were tracked as they moved freely while searching for toy objects in a controlled 3-dimensional environment, yielding the first such record of search-driven viewpoint selection. We found that subjects utilized their full range of eye and head motion to move from viewpoint to viewpoint, apparently employing a variety of objectives including changing viewing height and pose depending on object 3D pose. Subjects were also adept at selecting unobstructed views to search through otherwise occluded areas with objects. Furthermore, subjects completed the search task with high accuracy, even with no training on the environment. Although no learning was found in terms of accuracy over the duration of the experiment, increases in efficiency were found for other metrics such as response time, number of fixations, and distance travelled, particularly in target present trials where the target was not visible from the starting location. These results paint the story of a visual system that selects and moves to useful and informative views to facilitate the successful execution of an active visual search task, and stresses the significance of active vision research in understanding how vision is used in naturalistic environments.