PharaCon: A new framework for identifying bacteriophages via conditional representation learning

MotivationIdentifying bacteriophages (phages) within metagenomic sequences is essential for understanding microbial community dynamics. Transformer-based foundation models have been successfully employed to address various biological challenges. However, these models are typically pre-trained with self-supervised tasks that do not consider label variance in the pre-training data. This presents a challenge for phage identification as pre-training on mixed bacterial and phage data may lead to information bias due to the imbalance between bacterial and phage samples. ResultsTo overcome this limitation, this study proposed a novel conditional BERT framework that incorporates labels during pre-training. We developed an approach using a conditional BERT model for pre-training labeled data, incorporating label constraints with modified language modeling tasks. This approach allows the model to acquire label-conditional sequence representations. Additionally, we proposed a solution that utilizes conditional BERT in the fine-tuning phase as a classifier. We applied this conditional BERT framework to identify phages using a novel fine-tuning strategy, introducing PharaCon. We evaluated PharaCon against several existing methods on both simulated sequence datasets and real metagenomic contig datasets. The results demonstrate PharaCon's potential as an effective and efficient method for phage identification, highlighting the effectiveness of conditional B ERT as a solution for learning label-specific representations during pre-training on mixed sequence data. AvailabilityThe codes of PharaCon are now available in: https://github.com/Celestial-Bai/PharaCon. Contactyaozhong@ims.u-tokyo.ac.jp and imoto@hgc.jp