LabGym: quantification of user-defined animal behaviors using learning-based holistic assessment

Quantifying animal behavior is important for many branches of biological research. Current computational tools for behavioral quantification typically rely on a few pre-defined, simplified features to identify a behavior. However, such an approach restricts the information used and the tools applicability to a limited range of behavior types or species. Here we report a new tool, LabGym, for quantifying animal behaviors without such limitations. Combining a novel approach for effective evaluation of animal motion with customizable convolutional recurrent networks for capturing spatiotemporal details, LabGym provides holistic behavioral assessment and accurately identify user-defined animal behaviors without restrictions on behavior types or animal species. It then provides quantitative measurements of each behavior, which quantify the behavior intensity and the body kinematics during the behavior. LabGym requires neither any intermediate step for processing features that causes information loss nor programming knowledge from users for post-hoc analysis. It tracks multiple animals simultaneously in various experimental settings for high-throughput and versatile analysis. It also provides users a way to generate visualizable behavioral datasets that are valuable resources for the research community. We demonstrate its efficacy in capturing subtle behavioral changes in animals ranging from soft-bodied invertebrates to mammals.