Improving the workflow to crack Small, Unbalanced, Noisy, but Genuine (SUNG) datasets in bioacoustics: the case of bonobo calls

Abstract

Despite the accumulation of data and studies, deciphering animal vocal communication remains highly challenging. While progress has been made with some species for which we now understand the information exchanged through vocal signals, researchers are still left struggling with sparse recordings composing Small, Unbalanced, Noisy, but Genuine (SUNG) datasets. SUNG datasets offer a valuable but distorted vision of communication systems. Adopting the best practices in their analysis is therefore essential to effectively extract the available information and draw reliable conclusions. Here we show that the most recent advances in machine learning applied to a SUNG dataset succeed in unraveling the complex vocal repertoire of the bonobo, and we propose a workflow that can be effective with other animal species. We implement acoustic parameterization in three feature spaces along with three classification algorithms (Support Vector Machine, xgboost, neural networks) and their combination to explore the structure and variability of bonobo calls, as well as the robustness of the individual signature they encode. We underscore how classification performance is affected by the feature set and identify the most informative features. We highlight the need to address data leakage in the evaluation of classification performance to avoid misleading interpretations. Finally, using a Uniform Manifold Approximation and Projection (UMAP), we show that classifiers generate parsimonious data descriptions which help to understand the clustering of the bonobo acoustic space. Our results lead to identifying several practical approaches that are generalizable to any other animal communication system. To improve the reliability and replicability of vocal communication studies with SUNG datasets, we thus recommend: i) comparing several acoustic parameterizations; ii) adopting Support Vector Machines as the baseline classification approach; iii) explicitly evaluating data leakage and possibly implementing a mitigation strategy; iv) visualizing the dataset with UMAPs applied to classifier predictions rather than to raw acoustic features.