Video-Based Cattle Behavior Detection for Digital Twin Development in Precision Dairy Systems

Abstract

Digital twins in dairy systems require reliable behavioral inputs. We develop a video-based framework that detects and tracks individual cows and classifies seven behaviors under commercial barn conditions. From 4,964 annotated clips, expanded to 9,600 through targeted augmentation, we couple YOLOv11 detection with ByteTrack for identity persistence and evaluate SlowFast versus TimeSformer for behavior recognition. TimeSformer achieved 85.0% overall accuracy (macro-F1 0.84) and real-time throughput of 22.6 fps on RTX A100 hardware. Attention visualizations concentrated on anatomically relevant regions (head/muzzle for feeding and drinking; torso/limbs for postures), supporting biological interpretability. Structured outputs (cow ID, start-end times, durations, confidence) enable downstream use in nutritional modeling and 3D digital-twin visualization. The pipeline delivers continuous, per-animal activity streams suitable for individualized nutrition, predictive health, and automated management, providing a practical behavioral layer for scalable dairy digital twins.