A Deterministically Synchronized Widefield Imaging and Virtual Reality Platform for Multimodal Brain Behavior Recording

Abstract

ObjectiveSimultaneous recording of brain activity, behaviour, and virtual environments is essential for understanding large-scale neural dynamics during behaviour. However, existing systems often rely on software-based synchronization or post hoc alignment, introducing latency, jitter, and drift that obscure fast brain-behavior interactions. ApproachHere, we present a deterministically synchronized widefield calcium imaging platform that unifies neural imaging, high-speed behavioural monitoring, and closed-loop virtual reality (VR) under a shared hardware-defined clock. This system enables millisecond-precision temporal alignment across modalities, including dual-wavelength hemodynamic correction, pupil and orofacial tracking, locomotion sensing, and VR rendering. Main resultsThe platform achieves stable hardware-level synchronization across neural imaging, behavioural recordings, and VR rendering without reliance on software timestamps. It supports widefield imaging rates up to 100 Hz and integrates seamlessly with both ViRMEn and Blender VR engines, exhibiting a mean locomotion-to-VR update latency of [~]1.5 ms. Multimodal recordings during VR navigation demonstrate robust temporal alignment between cortical activity, facial dynamics, pupil signals, and locomotion. SignificanceThis system provides a deterministic multimodal framework for studying brain-behaviour relationships during active behaviour. By enabling millisecond-precision synchronization across neural imaging, behaviour, and virtual environments, this platform enables causal investigation of brain-behaviour interactions at millisecond precision and provides a foundation for next-generation closed-loop neuroengineering experiments.