Low-Dimensional Representations of Visuomotor Coordination in Natural Behavior

Abstract

Understanding how the eyes, head, and hands coordinate in natural contexts is a critical challenge in visuomotor coordination research, often constrained by sedentary tasks, cued actions, or restricted settings. To address this gap, we conducted an experiment where participants could self-generate pick-and-place actions on a life-size shelf in a virtual environment, recording concurrent gaze and body movements. Subjects exhibited intricate translation and rotation movements of the eyes, head, and hands during the task. We employed a time-dependent principal component analysis to study the relationship between the movements of the eyes, head, and hands relative to the onset of the action. We reduced the overall dimensionality into 2D representations, capturing up to 65% of the movement variance just in time with the actions. Our analysis revealed a synergistic coupling of the eye-head and eye-hand systems, as well as a strong coupling within the head-hand system. The eyes synchronized with the head and hands close to the action onset, with variations in coupling observed in horizontal and vertical planes, indicating distinct mechanisms for coordination in the brain. Crucially, the head and hands remained tightly coupled throughout the observation period, suggesting a shared neural code driving these effectors. Notably, the low-dimensional representations of the eye-head-hand movement vectors showed maximum predictive accuracy of the actions location ~200ms before the action onset, highlighting just-in-time coordination among the three effectors. Furthermore, the predictive accuracy was significantly influenced by the location of the upcoming action. Our study emphasizes the differential visuomotor control subject to the task structure, providing insights into the dynamic interplay of eye, head, and hand movements during natural behavior. NEW & NOTEWORTHYStudying natural, self-initiated, complex visuomotor coordination, we observe low-dimensional dynamics with distinct patterns along horizontal and vertical axes. The eyes horizontal movement showed notable independence, aligning with head and hand movements just in time for action. Notably, around critical events, the dimensionality of the complex movements is further reduced, indicating dynamic correspondence of eye-head-hand coordination.