Entrained audiovisual speech integration implemented by two independent computational mechanisms: Redundancy in left posterior superior temporal gyrus and Synergy in left motor cortex

Abstract

Information integration is fundamental to many aspects of human behavior, and yet its neural mechanism remains to be understood. For example, during face-to-face communication we know that the brain integrates the auditory and visual inputs but we do not yet understand where and how such integration mechanisms support speech comprehension. Here we show that two independent mechanisms forge audiovisual representations for speech comprehension in different brain regions. With a novel information theoretic measure, we found that theta (3-7 Hz) oscillations in the posterior superior temporal gyrus/sulcus (pSTG/S) code speech information that is common (i.e. redundant) to the auditory and visual inputs whereas the same oscillations in left motor and inferior temporal cortex code synergistic information between the same inputs. Importantly, redundant coding in the left pSTG/S and synergistic coding in the left motor cortex predict behavior - i.e. speech comprehension performance. Our findings therefore demonstrate that processes classically described as integration effectively reflect independent mechanisms that occur in different brain regions to support audiovisual speech comprehension.