xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
Bo Chen, Xingyi Cheng, Pan Li, Yangli‐ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Ying Zeng, Chi-Ming Liu, Aohan Zeng, Yuxiao Dong, Jie Tang, Le Song
A 100-billion-parameter protein language model trained with masked and autoregressive objectives together, topping prior results on 15 of 18 benchmarks and powering a fast structure predictor.
bioRxiv · Jul 5, 2023 · PreprintSynthetic Biology✦ In-depth digest