PrimateFace: A Machine Learning Resource for Automated Face Analysis in Human and Non-human Primates

Key results

1. ShownFace detectors trained on PrimateFace transfer to human faces almost as well as human-trained detectors. Human-trained detectors lose accuracy on primate faces.

   RetinaNet trained on PrimateFace reached 0.340 mAP@0.50 on human WIDERFace validation images vs 0.390 for a WIDERFace-trained RetinaNet. On the PrimateFace test set, the WIDERFace-trained model scored 0.585 mAP vs 0.775 for the PrimateFace-trained one. In qualitative comparisons, GroundingDINO prompted with 'face' found primate faces inconsistently. · PrimateFace models generalize effectively to…; Human Dataset Evaluation Protocols; PrimateFace detectors outperform human-face and…; PrimateFace is an integrated ecosystem for…

2. ShownTop-down landmark models trained on PrimateFace stay close to human-trained baselines on human faces. Human-trained landmark models degrade sharply on primate faces and fail on distant taxa.

   HRNet-W18-Dark trained on PrimateFace: 0.061 NME (normalized by inter-ocular distance) on human COCO-WholeBody-Face validation vs 0.053 for the COCO-WholeBody-trained model. On PrimateFace data, the human-trained model had 0.122 NME vs 0.029 for the PrimateFace-trained one. Qualitatively, human models misplaced nose and mouth points in Cercopithecoidea and failed on Daubentonia. · PrimateFace models maintain human facial…; Human Dataset Evaluation Protocols; PrimateFace landmarks outperform human and…

3. ShownAccuracy varies by taxon, but detection works for every genus. Landmark error is highest for tarsiers, which the authors attribute to limited training data and their distinctive facial morphology.

   Every genus scored above 0.60 mAP@0.50:0.95 for detection. Tarsioidea scored best and Hominoidea worst, which the authors attribute to image context rather than face difficulty. Landmark estimation was worst on Tarsioidea and best on Hominoidea and Lemuroidea. Alouatta was hard because of fast mouth movement while vocalizing. Per-taxon values appear only in figures. · Face detection maintains high performance across…; Facial landmark estimation maintains consistent…

4. ShownPicking training images for visual diversity using DINOv2 embeddings beats random picking when the annotation budget is small. The gap closes as the budget grows.

   Cascade R-CNN detectors trained on DINOv2-sampled subsets gained 2 to 4 points of mAP@[.50:.95] over random subsets, across budgets of 500 to 8,000 images. The gain peaked near 500 images, and the two methods converged at larger sizes. This led to a target of about 500 to 850 images per genus. · PrimateFace dataset provides a foundation for…; Sampling Efficiency Benchmark for Detector…

5. SuggestedPrimateFace models serve as the front end (off the shelf, or fine-tuned for howler monkeys) for macaque identity recognition, howler monkey mouth-voice coupling, human infant gaze synchrony and unsupervised discovery of macaque facial movements.

   An SVM on ArcFace embeddings of PrimateFace-aligned faces reached 0.858 top-1 accuracy over 62 rhesus macaques (chance 1/62 ≈ 0.016). After fine-tuning on howler data, howler monkey mouth aperture correlated with the vocal envelope at r = 0.33. Two human infants' gaze yaw angles correlated 0.85 at 0.64 s lag. A modified MotionMapper pipeline found over 80 recurring macaque movement patterns. · Scientific Application 2: Rapid Face Recognition…; Scientific Application 3: Fine-Grained…; Scientific Application 4: Quantifying Gaze…; Scientific Application 5: Data-Driven Discovery…