Geometry-based BERT: an experimentally validated deep learning model for molecular property prediction in drug discovery

Various deep learning based methods have significantly impacted the realm of drug discovery. The development of deep learning methods for identifying novel structural types of active compounds has become an urgent challenge. In this paper, we introduce a self-supervised representation learning framework, i.e., GEO-BERT. GEO-BERT considers the information of atoms and chemical bonds in chemical structures as the input, and integrates the positional information of the three-dimensional conformation of the molecule for training. Specifically, GEO-BERT enhances its ability to characterize molecular structures by introducing three different positional relationships: atom-atom, bond-bond, and atom-bond. By benchmarking study, GEO-BERT has demonstrated optimal performance on multiple benchmarks. We also performed prospective study to validate the GEO-BERT model, with screening for DYRK1A inhibitors as a case. Two potent and novel DYRK1A inhibitors (IC50: View larger version (24K): org.highwire.dtl.DTLVardef@1e9bc54org.highwire.dtl.DTLVardef@4b6832org.highwire.dtl.DTLVardef@a5d8eeorg.highwire.dtl.DTLVardef@44f7be_HPS_FORMAT_FIGEXP M_FIG C_FIG