Eye tracking insights into movement preparation and execution under nonstandard visual movement feedback

Abstract

For eye-hand coordination, predictions of sensory movement consequences may already be issued, and adjusted, during action preparation. In this pre-registered study, we combined a delayed-movement paradigm with a virtual reality-based hand-eye tracking task to investigate the oculomotor correlates of planning and executing coordinated hand-eye movements under standard vs nonstandard visual hand movement feedback. We measured pupil dilation and gaze-hand tracking during action preparation and subsequent task execution, where visual movement feedback violated or matched cued expectations: Participants prepared and, after a delay period, executed hand movements. Their movements were reflected by congruent or incongruent (inverted) movements of a glove-controlled virtual hand model, which they had to follow with their gaze. In the preceding delay period, visual cues could specify the to-be-executed movement (or leave it unspecified), and the visuomotor mapping (congruent or incongruent, 75% cue validity). We found that during the delay, pupil diameter increased more strongly when the movement was pre-cued (compared to left unspecified), and when nonstandard compared to standard visual movement feedback was expected. During execution, gaze-hand tracking performance decreased under nonstandard mappings, but significantly less so when the to-be-executed movement was pre-cued. Expectation violation trials produced a strong pupil dilation, particularly when congruent (standard) visuomotor expectations were violated, but also when incongruent mappings were cued but congruent ones observed. Furthermore, expectation violation impaired tracking performance; again, stronger for pre-cued movements with standard mapping. Our results indicate that oculomotor responses during delay encode processes related to motor planning and flexible forward prediction of sensory action consequences ahead of execution, i.e. increased mental effort and expectations of sensory conflict. Moreover, the results demonstrate that the strength of these (updated) predictions affects eye-hand coordination and pupillary responses during subsequent execution of the planned action.