Cheese3D: Sensitive Detection and Analysis of Whole-Face Movement in Mice

Abstract

Facial expressions and movements, from a subtle and ephemeral grimace, to vigorous and rapid chewing, offer direct insights into the moment-to-moment change of neural and physiological processes. Mice, with discernible facial responses and evolutionarily conserved mammalian facial movement control circuits, provide an ideal model to unravel the link between facial movement and underlying physiological states. However, existing frameworks lack the spatial or temporal resolution to sensitively track all movements of the mouse face, due to its small and conical form factor. We introduce Cheese3D, a computer vision system that captures high-speed 3D motion of the entire mouse face (including ears, eyes, whisker pad, and jaw, while covering both sides of the face) using a calibrated six-camera array. The interpretable framework extracts dynamics of anatomically-meaningful 3D facial features in absolute world units at sub-millimeter precision. The precise face-wide motion data generated by Cheese3D provides clear physiological insights, as shown by proof-of-principle experiments predicting time under general anesthesia from changing facial patterns, inferring tooth and muscle anatomy from fast ingestion motions across the entire face, measuring minute differences in movements evoked by brainstem stimulation, and relating neural activity to spontaneous facial movements, including expressive features only measurable in 3D (e.g., angles of ear motion). Cheese3D can serve as a discovery tool that renders subtle mouse facial movements highly interpretable as a readout of otherwise hidden internal states.