Integrating real-time data analysis into automatic tracking of social insect behavior

Abstract

Automatic video tracking has become a standard tool for investigating the social behavior of insects. The recent integration of computer vision in tracking technologies will likely lead to fully automated behavioral pattern classification within the next few years. However, most current systems rely on offline data analysis and use computationally expensive techniques to track pre-recorded videos. To address this gap, we developed BACH (Behavior Analysis maCHine), a software that performs video tracking of insect groups in real time. BACH uses object recognition via convolutional neural networks and identifies individually tagged insects via an existing matrix code recognition algorithm. We compared the tracking performances of BACH and a human observer across a series of short videos of ants moving in a 2D arena. We found that, concerning computer vision-based ant detection only, BACH performed only slightly worse than the human observer. Contrarily, individual identification only attained human-comparable levels when ants moved relatively slow, and fell when ants walked relatively fast. This happened because BACH had a relatively low efficiency in detecting matrix codes in blurry images of ants walking at high speeds. BACH needs to undergo hardware and software adjustments to overcome its present limits. Nevertheless, our study emphasizes the possibility of, and the need for, integrating real time data analysis into the study of animal behavior. This will accelerate data generation, visualization and sharing, opening possibilities for conducting fully remote collaborative experiments.