High-resolution dissection of concept acquisition in different families of protein language models
Shawn Whitfield, Tom Marty, Robert McCoy Vernon, CHRISTOPHER JAMES LANGMEAD, Dhanya Sridhar, Quentin Fournier
This study analyzes how biological concepts are encoded in protein language models across their layers and finds that complexity increases with depth, with pretraining data and compute having a greater impact than parameter scaling.
bioRxiv · Jul 23, 2026 · PreprintBioinformatics