Closing the loop: tracking and perturbing behaviour of individuals in a group in real-time

Abstract

Quantitative description and selective perturbation of individual animals in a social group is prerequisite for understanding complex social behaviors. Tracking behavioral patterns of individuals in groups is an active research field, however, reliable software tools for long-term or real-time tracking are still scarce. We developed a new open-source platform, called xyTracker, for online tracking and recognition of individual animals in groups. Featuring a convenient MO_SCPLOWATLABC_SCPLOW-based interface and a fast multi-threading C++ core, we achieved an > 30x speed-up over a popular existing tracking method without loss in accuracy. Moreover, since memory usage is low, many hours of high-resolution video files can be tracked in reasonable time, making long-term observation of behavior possible. In a number of exemplary experiments on zebra fish, we show the feasibility of long-term observations and how to use the software to perform closed-loop experiments, where the tracked position of individuals is fed-back in real-time to a stimulus presentation screen installed below the fish-tank. Visual stimulation capabilities is incorporated into xyTracker and can be based on any behavioral features of all members of the group, such as, collective location, speed, or direction of movement, making interesting closed-loop experiments for investigating group behavior in a virtual reality setting possible.