Knowledge Graph-based Thought: a knowledge graph enhanced LLMs framework for pan-cancer question answering

BackgroundIn recent years, Large Language Models (LLMs) have shown promise in various domains, notably in biomedical sciences. However, their real-world application is often limited by issues like erroneous outputs and hallucinatory responses. ResultsWe developed the Knowledge Graph-based Thought (KGT) framework, an innovative solution that integrates LLMs with Knowledge Graphs (KGs) to improve their initial responses by utilizing verifiable information from KGs, thus significantly reducing factual errors in reasoning. The KGT framework demonstrates strong adaptability and performs well across various open-source LLMs. Notably, KGT can facilitate the discovery of new uses for existing drugs through potential drug-cancer associations, and can assist in predicting resistance by analyzing relevant biomarkers and genetic mechanisms. To evaluate the Knowledge Graph Question Answering (KGQA) task within biomedicine, we utilize a pan-cancer knowledge graph to develop a pan-cancer question answering benchmark, named the Pan-cancer Question Answering (PcQA). ConclusionsThe KGT framework substantially improves the accuracy and utility of LLMs in the biomedical field. This study serves as a proof-of-concept, demonstrating its exceptional performance in biomedical question answering. Key PointsO_LIWe introduce a framework combining LLMs with KGs to improve factual accuracy in LLM reasoning. C_LIO_LIOur system is a flexible architecture that seamlessly integrates various LLMs. C_LIO_LIUtilizing a pan-cancer knowledge graph, we have proposed the first KGQA benchmark in the field of biomedicine. C_LIO_LICase studies reveal our method enhanced LLMs in addressing biomedical challenges such as drug repositioning, resistance research, individualized treatment, and biomarker analysis. C_LIO_LIThe method performs favorably in comparison to existing methods. C_LI