Discovering compositional, interpretable structure in social behavior

Abstract

Quantitative assessment of behavior is vital for rigorous analysis of how animals interact in the world and how behavior is generated from brain activity. Recent advances in automated pose detection have made possible data-driven discovery of movement action sequences. While many algorithms have been developed for this purpose, their categorical nature, i.e., the assumption that behavior is one action at a time, prohibits expressive quantification of social behavior. Such behavior is combinatorial and compositional, with multiple animals performing actions that may be either independent or coordinated over time. Here we show how developments in decomposed linear dynamical systems (dLDS) can instead represent action-generating dynamics as a signed, graded mixture of shared linear operators acting directly on keypoint positions. We demonstrate our approach across both single-mouse and two-mouse datasets, comparing against a state-of-the-art switching model, keypoint-MoSeq. In single-mouse data, we find that a composition of three operators can capture the full repertoire of 21 keypoint-MoSeq syllables, showing that even for a single animal, the action space is more parsimoniously described by a continuous subspace than clustered motifs. In a two-mouse resident--intruder assay, we identify, without behavioral labels, separate primitives for self-action, inter-animal-geometry, and social-action. The latter couples an animal's action to its partner's relative bearing, enabling the action to incorporate the social geometry in which it occurs. dLDS further decodes human-labeled social behaviors (i.e., "attack", "investigation", "mount", and "other") more accurately than behavior-selective keypoint-MoSeq syllable sets using fewer units. Together, these results demonstrate that compositional, time-varying linear dynamics can provide a path towards flexible and interpretable modeling of action-level behavior for both individual and social behavior.