Reducing Manual Labour in Forensic Microtrace Recognition with Deep Learning

Abstract

Forensic microtrace investigation relies on time- and labor-intensive microscopic analyses. To aid forensic experts in their investigations, an image recognition model for microtrace localisation and classification is needed. In this work, we use deep learning to automate trace recognition in images captured with automated microscopy. We localise and classify fibres, hairs, skin, glass and sand in microscopy scans through pixel-wise classification of tape-lift samples. As deep learning requires extensive amounts of annotated training data, we additionally investigate various pretraining strategies to minimise the required annotation workload. We compare ImageNet pretraining, pretraining with self-supervised learning and a sequential application of these approaches. We find that pretrained models are able to reduce the required annotated data twofold compared to models trained from scratch while retaining the prediction accuracy. While our ImageNet-pretrained models outperform our self-supervised-pretrained models, we achieve the highest accuracy by combining the two approaches, resulting in a factor of 4 reduction of manual annotated microtraces or a 65 % improvement in recognition and localisation accuracy (mean intersection over union increases from 0.34 to 0.56 due to pretraining) when training on only 2.2 dm2 of annotated tape lift scans. Our model is, therefore, the method of choice for the automatic analysis of large forensic microtrace scans.