AMADEUS: Annotation-free multi-animal direction estimation using self-supervised learning

Markerless multi-animal tracking has advanced rapidly, yet measuring social interactions among visually similar animals remains challenging during occlusion and crowding. Here we present AMADEUS (Annotation-free Multi-Animal Direction Estimation Using Self-supervised learning), a tracking system that requires neither manual training annotation nor physical marking. AMADEUS extracts masks representing single animals, uses movement direction to assign head direction, and then synthesizes interaction images using copy-paste augmentation. A detector trained on the synthetic dataset estimates oriented bounding boxes and head direction, from which center, front and rear keypoints are derived. For a video containing instances of almost complete occlusion, AMADEUS additionally uses contrastive learning for identity verification. Across benchmark videos of 10 mice, 10 fish, 32 beetles, 32 ants with one cricket, 60 flies, and 368 ants, AMADEUS outperformed state-of-the-art methods in identity tracking and keypoint localization. AMADEUS enables scalable quantification of interactions across diverse species.