Exploiting facial side similarities to improve AI-driven sea turtle photo-identification systems

AO_SCPLOWBSTRACTC_SCPLOWAnimal Photo-identification (photo-ID) denotes the process of identifying individual animals based on their unique and stable morphological characteristics. It has been proven to be a particularly useful tool for a variety of studies on sea turtles increasing the knowledge of their ecology and informing conservation efforts. The photo-ID process in sea turtles is predominantly based on the geometric patterns of the scales of their two head sides which are unique to every individual, and also different from side to side. As such, both manual and automated photo-ID techniques are performed under a side-specific image retrieval setting. There, a query image showing a single profile of an unknown individual, either left or right, is only compared to images of previously identified individuals showing the same side. Here, by employing the recently introduced state-of-the-art deep feature extractor MegaDescriptor, we show for the first time an inherit deep visual similarity between left and right facial profiles of the same individuals in three sea turtle species. We show that the similarity between the left and right profiles of the same individual with respect to geometry, coloration and pigmentation, is on average higher than the similarity between profiles of different individuals. The similarity is detectable even when images are taken years apart and under diverse settings and conditions. We perform several image retrieval experiments under scenarios which mimic realistic sea turtle photo-ID matching processes, where we also allow comparisons of opposite sides in the matching process which have no spatial overlap. We show that the detection and exploitation of this similarity is translated to improved accuracies when compared to the traditional side-specific image retrieval setting. Notably this similarity cannot be detected and thus neither explored by the so-far state-of-the-art sea turtle photo-ID automated methods such as those based on SIFT. Our work leads to a change of paradigm for the sea turtle photo-ID workflows which will be further facilitated by the constant improvement of deep feature-based re-identification methods and paves the path for adopting similar workflows in other animal species as well.