henhaobio
Sign in

bioRxivBioinformaticsPreprint

Human-supervised Agentic AI for Hypothesis Generation and Experimental Assistance in Drug Repurposing

Dinh-Long HuynhiD, Elin AspiD, Flavio BallanteiD, Jordi Carreras PuigvertiD, Alisa Nicole DeGraveiD, Reagon KarkiiD, Kristen Michelle NaderiD, Päivi ÖstlingiD, B.K. PokhareliD, Jonne RietdijkiD, Lars SchlotawaiD, Lina Schmidt, Srijit SealiD, Brinton A. Seashore-LudlowiD, Tero A. AittokallioiD, Ola SpjuthiD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Computational drug repurposing has largely been focused on rapid hypothesis generation, yet real-world applications span a far broader lifecycle, from drug candidate suggestion to designing experiments, analyzing assay data, and iteratively refining candidates. Here, we demonstrate that agentic AI can operate throughout this lifecycle. To this end, we developed RepurAgent, a hierarchical multi-agent AI system comprising a supervisor agent and a planning agent that coordinate four specialized sub-agents (research, prediction, data, and report), through a human-in-the-loop design, with episodic memory and retrieval-augmented generation. The system is grounded in data, tools, and standard operating procedures specific for drug repurposing, developed within the REMEDi4ALL consortium. We validated the agentic system across three scenarios spanning the various stages within the repurposing lifecycle: in Acute Myeloid Leukemia, a blinded expert evaluation indicated that RepurAgent produced substantially more novel and mechanistically credible candidates compared to a vanilla LLM baseline; in a retrospective COVID-19 antiviral screen, RepurAgent acted as an adaptive experimental collaborator, prioritizing compounds with AUC-ROC up to 0.99 without predefined thresholds and flagging confounders missed in manual review; and for Multiple Sulfatase Deficiency, it prioritized 81 high-confidence candidates from 5000 compounds, which were further corroborated by domain experts. These results demonstrate that agentic AI can support across the drug repurposing lifecycle, from hypothesis generation to experimental analysis. RepurAgent is open source and deployed at https://repuragent.serve.scilifelab.se/.

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 2 (17 Aug 2026) · CC BY 4.0

A human-supervised multi-agent LLM system covered drug repurposing from knowledge graph building to assay analysis in three case studies; retrospective COVID-19 screen AUC-ROC 0.986 without predefined thresholds.

Per the paper, existing repurposing tools (knowledge graphs, ML predictors, literature mining, ADMET and enrichment tools) are fragmented, and no framework joins hypothesis generation with experimental data analysis and confounder detection. RepurAgent coordinates these steps in one workflow under human review.

Key results

  1. ShownAdding SOP retrieval, PubMed search and episodic memory to the planning agent raised plan quality from 0.492 (vanilla LLM) to 0.651. The authors conclude that plans still need human review.

    8 configurations (2^3 design), 15 requests (5 from the case studies), 10 runs each, scored by an LLM judge. Every tool-augmented configuration beat the baseline (adjusted p < 10^-10). Single-tool gains: SOP +0.038, PubMed +0.054, episodic memory +0.066. · Planning agent performance is maximized by tool…; Ablation study for Planning Agent

  2. SuggestedFor AML, the agent built a knowledge graph from one prompt. Its top-20 drugs covered the pathways of Google Co-Scientist's five drugs significantly more than vanilla LLMs did on recall, precision and Jaccard. The comparison is at pathway level only.

    Up to 97% pathway coverage in 45 to 60 minutes. Recall ~0.82, precision ~0.28, Jaccard ~0.26, higher than six vanilla GPT baselines (Mann-Whitney U, Holm). Vanilla LLMs proposed 15% to 39% already-known AML drugs in their top 20. · RepurAgent executes end-to-end workflow to…; Evaluation of Pathways overlap with Co-Scientist…; Figure legends

  3. ShownIn a retrospective COVID-19 antiviral screen, the agent ranked compounds with high concordance to the original expert hit lists. It also flagged confounders such as cytotoxicity (infection inhibition with more than 20% viability loss) and cell painting rescue without a matching infection reduction.

    Primary screen of 5275 compounds (324 hits): AUC-ROC 1.0 with supplied thresholds, 0.986 with self-inferred thresholds. Validation screen (74 hits among 324): AUC-ROC 0.906. · RepurAgent as an active assistant for COVID-19…; Discussion; Benchmarking RepurAgent’s compound…

  4. SuggestedFor Multiple Sulfatase Deficiency, with very little annotated data, the agent used literature retrieval to define three mechanistic directions and tiered a compound library. One high-confidence compound was independently flagged by external experts.

    Initial knowledge graph had 46 nodes and 4 genes. Of about 5000 compounds, 82 were high confidence and 115 medium. High-confidence compounds were all proteostasis/ER control; 85.2% of medium ones were lysosomal/autophagy/SRT. · RepurAgent prioritises promising drugs to…; Figure legends

  5. ShownThe retriever over the SOP corpus found the right source document for most queries. Locating the exact supporting passage was harder, especially for software-related queries.

    200 queries, k = 4. File level: Recall@4 0.950, MRR 0.883. Evidence level: Recall@4 0.795, MRR 0.607. Software lookup was lowest (Recall@4 0.375, MRR 0.292). · Evaluation of RAG performance

Methods, data and reproducibility

A hierarchical LLM agent system (planning, supervisor, research, prediction, data and report agents) with ReAct tool use, episodic memory and retrieval over SOPs. A human approves plans and supervises execution. It was evaluated by ablation, by pathway overlap with Co-Scientist, and by ROC concordance with expert selections.

  • Planning benchmark: 15 task requests with reference decompositions (5 from case studies, 10 synthetic) · n = 15 requests x 10 runs x 8 configurations
  • AML case: agent-built knowledge graph (Open Targets, ChEMBL, UniProt, GWAS) and a 435-drug initial space; top-20 candidates' pathways compared with those of Google Co-Scientist's five published drugs. · human · n = 435 initial drugs; 10 runs per model for overlap metrics
  • COVID-19 screen data from Asp et al. (2025): cytopathic effect, antibody-based and cell painting assays. The primary screen used Vero E6 cells; validation used A549-ACE2 cells. · human (A549-ACE2 validation cells); Vero E6 cells for the primary screen · n = 5275 compounds; 324 primary hits; 74 validation hits
  • MSD compound library prioritized into confidence tiers · human (SUMF1-linked disease) · n = about 5000 compounds
  • SOP retrieval benchmark of stratified queries · n = 200 queries
Methods & tools
LangChain, LangGraph, Gradio, ChromaDB, LangMem, CPSign, RDKit, GSEApy/Enrichr, NCBI LitSense 2.0, KGG, RepurposeDrugs, Chemical Annotator
Software & environment
LangChain v0.3.24, LangGraph v0.6.3, Gradio v5.49.1, ChromaDB v0.6.3, LangMem v0.2.9, CPSign v2.0.0, GSEApy v1.1.11, Unstructured v0.18.4; embeddings text-embedding-3-small; GPT-5.1 used for SOP chunk descriptions.
Experimental conditions
AML workflow completed in 45 to 60 minutes; COVID-19 reanalysis in minutes (versus weeks manually); no hardware stated.
Reproducibility gaps
The corroborated MSD compound is undisclosed for intellectual property reasons. The per-agent LLM choices (Table 1) and supplementary tables were not in the text provided.
Limitations (5)
  • No prospective wet-lab validation: the AML candidates are untested, the COVID-19 study is retrospective, and the corroborated MSD compound cannot be disclosed. · stated by the authors
  • The three case studies measure different kinds of utility and cannot be reduced to one metric, and there are no community-accepted benchmarks for end-to-end agentic systems. · stated by the authors
  • Literature-aware agents may retrieve published answers rather than infer them, which complicates benchmarks built from known drug-disease pairs. · stated by the authors
  • The AML comparison uses pathway overlap with only five Co-Scientist drugs as a proxy, since no drug-level comparison was possible. The COVID-19 AUC-ROC measures agreement with the original authors' selections, not independent correctness. · noted in this summary
  • Planning quality is scored by an LLM judge against reference decompositions built by the authors, with no human rater agreement reported in the text. · noted in this summary

Read this if you want to see how an LLM agent system with SOP retrieval and human review was tested across repurposing tasks, from hypothesis generation to screening-data analysis.

Citations
2

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISZhang '25Sanz '25Matsumoto '26Pham '24Llompart '26Aamer '26
70% semantic matchBioinformatics
Fan Zhang, Yalong Zhao, Weihan Zhang+1 more·bioRxiv·Aug 8, 2025· 2 citations