Benchmarking Hayai-Annotation Plants: A Re-evaluation Using Standard Evaluation Metrics

Abstract

The rapid growth of next-generation sequencing (NGS) technology has led to a surge in the determination of whole genome sequences in plants. This has created a need for functional annotation of newly predicted gene sequences in the assembled genomes. To address this, "Hayai-Annotation Plants" was developed as a gene functional annotation tool for plant species. In this report, we compared Hayai-Annotation Plants with Blast2GO and TRAPID, focusing on the three primary gene-ontology (GO) domains: Biological Process (BP), Molecular Function (MF), and Cellular Component (CC). Using the Arabidopsis thaliana GO annotation as a benchmark, we evaluated each tool using two approaches: the area under the precision-recall curve (AUC-PR) and the metrics used at the critical assessment of functional annotation (CAFA). In the latter case, a CAFA-evaluator, was used to determine the F-score, weighted F-score, and S-score for each domain. Hayai-Annotation Plants showed better performances in all three GO domains. Our results thus reaffirm the effectiveness of Hayai-Annotation Plants for functional gene annotation in plant species. In this era of extensive whole genome sequencing, Hayai-Annotation Plants will serve as a valuable tool that facilitates simplified and accurate gene function annotation for numerous users, thereby making a significant contribution to plant research.