From pose to behavior: SABER integrates identity-resolved multi-animal pose tracking with large-language-model-based behavioral factor discovery

Abstract

Quantifying social behavior requires accurately assigning posture and actions to individual animals, but close contact, occlusion, and identity switches often hinder this. Meanwhile, current behavioral recognition pipelines still lack stable predictive accuracy, especially when using a unified multi class model. Here we developed SABER, a locally deployable framework that couples multi animal pose estimation with identity preserving tracking, interpretable behavioral factor mining, and multiscale temporal behavior prediction from a single overhead video stream. Across spontaneous social interaction, mating, aggression, and four mouse recordings, SABER improved pose-estimation accuracy and tracking continuity relative to comparator pipelines. Its behavioral factor mining procedure identified interpretable kinematic, postural, and social descriptors, and temporal integration refined behavioral category classification. SABER offers an intuitive, open source interface to facilitate use. Applied to social defeat stress mice, SABER detected reduced approach behavior and a multivariate behavioral profile that distinguished depression susceptible from control animals. SABER outputs could also be synchronized with miniscope calcium recordings, enabling joint analysis of behavioral states and neuronal population activity. SABER therefore provides an accessible, identity resolved route from single view social interaction video to behavioral phenotyping and brain behavior analysis.