SBMate: A Framework for Evaluating Quality of Annotations in Systems Biology Models

Abstract

The interests in repurposing and reusing systems biology models have been growing in recent years. Semantic annotations play an important role for this, as they provide crucial information on the meanings and functions of models. However, there are a limited number of tools that evaluate the existence or quality of such annotations. In this paper, we introduce SBMate, a python package that would serve as a framework for evaluating the quality of annotations in systems biology models. Three default metrics are provided: coverage, consistency, and specificity. Coverage checks whether annotations exist in a model. Consistency tests if the annotations are appropriate for the given model element. Finally, specificity represents how detailed the annotations are. We analyzed 1,000 curated models from the BioModels repository using the three metrics and discussed the results. Additional metrics can be easily added to extend the current version of SBMate.