Analyzing combined eye-tracking/EEG experiments with (non)linear deconvolution models

Fixation-related potentials (FRPs), neural responses aligned to saccade offsets, are a promising tool to study the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/EEG experiments: (i) the synchronization of data streams, (ii) the removal of ocular artifacts, (iii) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, (iv) and the fact that numerous low-level stimulus and saccade properties also influence the post-saccadic neural responses. While effective solutions exist for the first two problems, the latter ones are only beginning to be addressed. In the current paper, we present and review a unified framework for FRP analysis that allows us to deconvolve overlapping potentials and control for linear and nonlinear confounds on the FRPs. An open software implementation is provided for all procedures. We then demonstrate the advantages of this approach for data from three commonly studied paradigms: face perception, scene viewing, and natural sentence reading. First, for a traditional ERP face recognition experiment, we show how deconvolution can separate stimulus-ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in scene viewing, we isolate multiple non-linear influences of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable fixation-related brain potentials during natural vision.