FragLlama: Next-fragment prediction for molecular design

The emergence of ChatGPT has drawn significant attention to Large Language Models (LLMs) due to their impressive performance. While LLMs primarily focus on next token/word prediction, we apply this principle to molecular design by reframing the task as predicting the next token/fragment. We present FragLlama, a large language model trained for molecular design, featuring custom tokens that represent molecular fragments and functional groups. The model is for generating molecules given one or two fragments, for application scenarios like general hit-to-lead and lead optimization stage drug design, PROTAC linker design; mapping to commonly used drug design strategies like fragment growing and scaffold hopping. In the pre-training stage, we adapted the Llama 3 architecture to create FragLlama, training it to learn conditional probabilities of these fragment-level tokens. The subsequent alignment stage employed fine-tuning to guide the model towards generating molecules with desired properties. The effectiveness of FragLlama is demonstrated through its applications in designing molecular glue libraries, PROTAC linkers and EGFR binders. FragLlama demonstrates proficiency in reproducing expert-level designs while also exploring novel and promising chemical spaces, highlighting its potential to augment the capabilities of medicinal chemists in drug design.