Benchmarking AI-based plasmid annotation tools for antibiotic resistance genes mining from metagenome of the Virilla River, Costa Rica

Bioinformatics and Artificial Intelligence (AI) stand as rapidly evolving tools that have facilitated the annotation of mobile genetic elements (MGEs), enabling the prediction of health risk factors in polluted environments, such as antibiotic resistance genes (ARGs). This study aims to assess the performance of four AI-based plasmid annotation tools (Plasflow, Platon, RFPlasmid, and PlasForest) by employing defined performance parameters for the identification of ARGs in the metagenome of one sediment sample obtained from the Virilla River, Costa Rica. We extracted and sequenced complete DNA from the sample, assembled the metagenome, and then performed the plasmid prediction with each bioinformatic tool, and the ARGs annotation using the Resistance Gene Identifier web portal. Sensitivity, specificity, precision, negative predictive value, accuracy, and F1score were calculated for each ARGs prediction result of the evaluated plasmidomes. Notably, Platon emerged as the highest performer among the assessed tools, exhibiting exceptional scores. Conversely, Plasflow seems to face difficulties distinguishing between chromosomal and plasmid sequences, while PlasForest has encountered limitations when handling small contigs. RFPlasmid displayed diminished specificity and was outperformed by its taxon-dependent workflow. We recommend the adoption of Platon as the preferred bioinformatic tool for resistome investigations in the taxon-independent environmental metagenomic domain. Meanwhile, RFPlasmid presents a compelling choice for taxon-dependent prediction due to its exclusive incorporation of this approach. We expect that the results of this study serve as a guiding resource in selecting AI-based tools for accurately predicting the plasmidome and its associated genes.