Evaluation and primer for using DeepLabCut to track small animals in large spaces

Abstract

DeepLabCut (DLC), a machine learning-based pose estimation tool, has dramatically increased the throughput of behavioral tracking. Yet its implication remains limited by lack of validation data from diverse scenarios and users' unfamiliarity with the machine learning workflow. Here, we provide a practical primer on DLC terminology and workflow and performance evaluation of DLC in a novel scenario where the target is much smaller than those in previous validation data. We assessed model performance in a total of 36 model configurations across three training proportions, six training iterations, and two recording resolutions. Performance was evaluated on training convergence, prediction error, prediction certainty, and generalizability. Additionally, we selected an optimal model based on a balance between model performance and training time and applied it to 20 recordings at each resolution. In the test frames, 15 and 6 out of 18 model configurations met all performance criteria under high- and low-resolution, respectively. Of these, DLC was able to achieve low prediction error, high prediction confidence, and low train-test loss. Prediction error and confidence varied with training proportion, iteration, and resolution. Inference using the optimal models achieved similar high confidence predictions, with low-likelihood predictions predominantly associated with visual obstruction, low light, and reduced contrast between the subject and the background. Notably, the optimal models for high- and low-resolutions performed similarly despite a multi-fold difference in the cost of the recording devices, providing validation data for DLC in low-budget research, teaching and outreach scenarios.