Convolutions are competitive with transformers for protein sequence pretraining
K. K. Yang, N. K. Fusi, A. X. Lu
This study shows that convolutional neural networks can be as effective as transformers for protein sequence pretraining, offering better scalability and performance on long sequences.
bioRxiv · May 19, 2022Bioengineering