Millisecond-scale detection of mouse ultrasonic vocalizations enables closed-loop experiments

Abstract

Mouse ultrasonic vocalizations (USVs) provide a rapidly evolving readout of social interaction but are typically analysed only after acquisition. Here we introduce DeepFisFis, a waveform-based neural network that detects USVs while they are being produced. DeepFisFis classifies consecutive 5 ms audio segments directly from the waveform with high accuracy, processing each segment in approximately 2.5 ms and therefore faster than the incoming audio stream. This enables ongoing vocalizations to guide experimental interventions in real time. In a deployed closed-loop system, detections triggered an external stimulus, demonstrating online control of ongoing vocal behaviour. DeepFisFis also enables event-triggered data acquisition: gating storage around detected calls preserved more than 99% of vocalization time while retaining only approximately 22% of the continuous recording. DeepFisFis thus transforms USVs from a retrospective behavioural readout into a real-time experimental signal for selective acquisition and closed-loop causal interrogation of vocal communication and its underlying neural circuits.