Predicting evolutionary rate as a pretraining task improves genome language model representations
Micaela Elisa Consens, Kevin K. Yang, Jimmy Hall…·Alex Xijie Lu
This paper introduces a pretraining task for genome language models that predicts evolutionary rate, showing improved performance over sequence-only pretraining.