Human Pose and Action Recognition

An end-to-end platform for pose estimation and real-time edge-AI deployment

This paper introduces an integrated platform for pose estimation that supports both offline analysis and real-time deployment on edge devices without requiring workstation-grade hardware.

From video to behaviour: an LSTM-based approach for automated nest behaviour recognition in the wild

This paper introduces an LSTM-based framework for automated nest behaviour recognition in wild birds, outperforming human annotators and YOLO in accuracy and speed.

AVATAR: AI Vision Analysis for Three-dimensional Action in Real-time

The paper introduces AVATAR, an AI system that virtualizes 3D mouse motions from body part detection, enabling real-time action classification and closed-loop manipulation.

PoseR - A deep learning toolbox for decoding animal behavior

PoseR is a deep learning tool that classifies animal behavior using pose estimation data, achieving accurate and efficient analysis across multiple species.

Deep Imputation for Skeleton Data (DISK) for Behavioral Science

DISK uses deep learning to impute missing skeleton data in animal behavior studies, improving reliability and enabling more accurate motion analysis.

Assessing Attentiveness and Cognitive Engagement across Tasks using Video-based Action Understanding in Non-Human Primates

This paper introduces a video-based analysis pipeline to classify cognitive states like attentiveness and engagement in non-human primates during tasks, achieving >90% decoding accuracy.

Mapping the landscape of social behavior

This paper introduces a 3D tracking technique to analyze social behavior in rodents, revealing new insights into autism and amphetamine effects through high-resolution postural data.

Forestwalk: A machine learning workflow brings new insights into posture and balance in rodent beam walking

Forestwalk uses machine learning to analyze rodent beam walking, revealing new insights into posture and balance through automated feature extraction and classification.

YOLO-Behaviour: A simple, flexible framework to automatically quantify animal behaviours from videos

YOLOv8 detection with SORT event grouping and grid-search tuning quantifies visually distinct behaviours from video across 5 case studies, reaching f1-scores of 0.62 to 0.94.

Marker Data Enhancement For Markerless Motion Capture

This paper improves motion capture accuracy by enhancing sparse keypoints with a more generalizable marker enhancer trained on a larger dataset.

refineDLC: an advanced post-processing pipeline for DeepLabCut outputs

RefineDLC improves DeepLabCut outputs by cleaning, filtering, and interpolating data to produce reliable kinematic analyses across species and recording conditions.

REVEALS: An Open Source Multi Camera GUI For Rodent Behavior Acquisition

REVEALS is an open-source GUI for rodent behavior data acquisition using multiple cameras, validated with DeepLabCut for pose estimation in behavioral assays.

Unsupervised decomposition of natural monkey behavior into a sequence of motion motifs

This paper introduces SMP, an unsupervised algorithm for decomposing primate behavior into motion motifs, outperforming existing methods and visualizing neural manipulation effects.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

SUBTLE: An unsupervised platform with temporal link embedding that maps animal behavior

SUBTLE maps animal behavior from 3D action skeletons using an unsupervised framework, achieving superior performance in capturing behavioral repertoires and identifying inter-group differences through temporal representation.

Anti-drift pose tracker (ADPT): A transformer-based network for robust animal pose estimation cross-species

ADPT is a transformer-based pose tracker that reduces tracking drift in animal behavior analysis, achieving high accuracy across species and conditions.

Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

The paper introduces GlowTrack, a method to generate large-scale training data for markerless motion capture, enabling generalizable deep learning models for animal behavior analysis.

DeepAction: A MATLAB toolbox for automated classification of animal behavior in video

DeepAction is a MATLAB toolbox that uses deep learning to automate animal behavior classification in video, achieving high accuracy with minimal training data and reducing annotation time.

A layered, hybrid machine learning analytic workflow for mouse risk assessment behavior

The paper develops a layered hybrid machine learning workflow to improve the analysis of mouse risk assessment behavior by combining random forest and hidden Markov models.

Identifying Behavioral Structure from Deep Variational Embeddings of Animal Motion

The paper introduces VAME, a deep learning framework for identifying hierarchical behavioral structures from animal motion data, demonstrating its ability to detect discrete motifs and community-specific usage patterns in mouse models.

BehaviorDEPOT: a tool for automated behavior classification and analysis in rodents

BehaviorDEPOT automates behavior classification and analysis in rodents, providing tools for tracking, classification, and experimental setup with reduced costs and improved reliability.

Selfee: Self-supervised Features Extraction of animal behaviors

Selfee extracts comprehensive and discriminative features from raw animal behavior videos using self-supervised learning, enabling anomaly detection and temporal analysis.

The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice

The paper introduces MARS, a software pipeline for automated analysis of social behaviors in mice, achieving human-level performance in pose estimation and behavior classification.

Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders

This paper introduces a video analysis tool that combines supervised pose estimation with unsupervised dimensionality reduction to extract interpretable behavioral features from animal videos, improving downstream analyses.

DeepEthogram: a machine learning pipeline for supervised behavior classification from raw pixels

DeepEthogram uses machine learning to automatically classify animal behaviors from video frames with high accuracy, reducing manual effort and increasing reproducibility.

Anipose: a toolkit for robust markerless 3D pose estimation

Anipose provides a toolkit for markerless 3D pose estimation using 2D video, enabling analysis of animal movement and motor control.

B-SOiD: An Open Source Unsupervised Algorithm for Discovery of Spontaneous Behaviors

B-SOiD is an open-source unsupervised algorithm that identifies spontaneous behaviors from pose data, improving processing speed and generalization across subjects.

MacaquePose: A novel 'in the wild' macaque monkey pose dataset for markerless motion capture

The paper introduces MacaquePose, a dataset for markerless motion capture of macaques, showing neural network performance comparable to human-level pose estimation.

Real-time, low-latency closed-loop feedback using markerless posture tracking

This paper introduces a real-time, low-latency pose estimation tool with feedback capabilities for behavioral neuroscience experiments.

OpenMonkeyStudio: Automated Markerless Pose Estimation in Freely Moving Macaques

OpenMonkeyStudio provides a deep learning-based system for markerless 3D pose estimation in freely moving macaques, enabling automatic tracking and action recognition in unconstrained environments.