henhaobio
Sign in

bioRxivGeneticsPreprint

TigerAI: An AI-powered genetic evidence platform to support clinical development

Zichen ZhangiD, Wanheng ZhangiD, Xiaochen YangiD, Yujue Li, Jinjie Lin, Chong WuiD, Bingxin ZhaoiD

PDFDOIData
In-depth full-text digest available · Jump to digest ↓

Abstract

Genetic evidence is a major determinant of clinical success in drug development, yet its aggregation has long relied on laborious human curation. Large language models (LLMs) have the potential to rapidly synthesize knowledge across biomedical resources, providing a route to scalable AI-driven genetic evidence generation. Here we develop a novel domain-grounded instruction framework to systematically evaluate GPT-5 for producing genetic evidence relevant to clinical trial success. Using 13,022 target-indication pairs from a comprehensive drug development database, we benchmark LLM-derived evidence against a recent exhaustive human expert-curated study. We find that GPT-5 yields genetic evidence that is at least as informative as expert curation for inferring clinical success, while substantially expanding coverage relative to traditional curation resources. Building on these results, we introduce TigerAI (https://tigerai.bio/), a dual-purpose platform for AI-powered genetic evidence that (i) benchmarks emerging state-of-the-art LLMs and (ii) provides an accessible service for querying reliable AI-generated genetic evidence. These contributions outline a practical, domain-grounded pathway for integrating AI-powered genetic evidence into drug development pipelines and for realizing the potential of LLMs to inform clinical success.

In-depth summary

Claude Opus 5.5 · xhigh effort · 7 Nov 2026 · from the full text of version 1 (1 Sept 2026) · CC BY 4.0

Domain-grounded prompting lets GPT-5 produce genetic evidence for 13,022 drug target-indication pairs that predicts clinical success as well as expert curation, with wider coverage.

Genetic support raises the chance that a drug mechanism succeeds clinically (2.6-fold in the benchmark expert-curated study), yet assembling it takes years of manual curation, and whether LLM-generated evidence matches trusted resources was untested. The work benchmarks GPT-5 against that curation on drug development outcomes and turns the protocol into TigerAI, a platform for ongoing LLM benchmarking and on-demand queries.

Key results

  1. ShownGPT-5 genetic support (ICES ≥ 4.0) reproduced the known enrichment of clinical success among genetically supported target-indication pairs, with relative success (RS) slightly above expert curation and higher coverage among launched programmes.

    Phase I to Launched RS 2.74 (GPT-5) vs 2.63 (Minikel et al. curation); Phase II to III 1.68 vs 1.57; Phase III to Launched 1.40 vs 1.28. Launched support 312 (20.5%) vs 189 (12.4%) of 1,519; Phase I 140 (5.4%) vs 93 (3.6%) of 2,569. · Benchmarking AI-generated genetic evidence…; Figure legends

  2. ShownPairs supported by both approaches form the highest-confidence subset, and pairs supported only by GPT-5 remain more enriched for success than curation-only pairs, so GPT-5 adds informative coverage, mostly for single-therapeutic-area indications.

    Launched: 66 human-only, 189 GPT-5-only, 123 both; Phase I to Launched RS 1.74 (human-only), 2.22 (GPT-5-only), 3.33 (both). Union covered 378/1,519 (24.9%) launched pairs, RS 2.64. Single-area indications: 97 vs 210 supported pairs, RS 2.52 vs 2.53. · Benchmarking AI-generated genetic evidence…; Figure legends

  3. ShownTherapeutic-area patterns were largely preserved: GPT-5 and curation agreed on launched-phase coverage and Phase I to Launched RS across areas, diverging mainly in endocrine, oncology, metabolic, psychiatry and infection.

    Across 17 therapeutic areas, Spearman ρ = 0.62 for launched P(G) and ρ = 0.60 for Phase I to Launched RS. GPT-5 coverage higher in endocrine and oncology, lower in psychiatry; curation gave higher RS in psychiatry and infection, GPT-5 higher in endocrine and metabolic. · Dissecting AI-generated genetic evidence by…; Figure legends

  4. ShownThe integrated causal evidence score (ICES) works as a tunable stringency setting and gave stronger enrichment for success than any single evidence channel.

    Launched supported pairs: 497 at ICES ≥3.0 (RS 2.17), 312 at ≥4.0 (RS 2.74), 175 at ≥5.0 (RS 2.94). Single channels gave Phase I to Launched RS 2.41 to 2.68; biological coherence had the broadest launched coverage (529/1,519, 34.8%) but the lowest RS. · Evidence-dimension interpretability and ICES…; Figure legends

  5. ShownUnder the same fixed protocol, GPT-4o was more permissive than GPT-5, with over twice the supported pairs in Phase I-III and higher verdicts overall, yet similar RS (below curation's). GPT-3.5 Turbo failed to return valid structured output.

    Phase I to Launched RS 2.74 (GPT-5) vs 2.60 (GPT-4o); the verdict confusion matrix had 4,445 pairs below vs 1,637 above the diagonal (GPT-4o assigning higher verdicts). GPT-3.5 Turbo (gpt-3.5-turbo-0125) was excluded from analysis. · Operationalizing AI-generated genetic evidence…; Implementations in LLMs; Figure legends

Methods, data and reproducibility

A fixed developer prompt casts the LLM as a human-genetics expert, bans drug and trial information, requires cited evidence across four channels and returns JSON sub-scores plus an integrated causal evidence score (ICES, 0-6). ICES ≥ 4.0 defines support, and relative success (risk ratio, Katz CIs) across phase transitions is compared with expert curation.

  • Minikel et al. curated Citeline Pharmaprojects monotherapy target-indication pairs added since 2000 that reached at least Phase I, with MeSH-mapped indications · human (human gene targets) · n = 13,022 T-I pairs
  • Benchmark strata by highest clinical phase reached · human · n = Phase I 2,569; Phase II 4,153; Phase III 665; Launched 1,519
  • Human expert-curated genetic support labels (Minikel et al.) built from OMIM, Open Targets, PICCOLO colocalization, Genebass exomes and IntOGen somatic data · human · n = 13,022 T-I pairs
  • GPT-5 and GPT-4o evidence outputs (ICES, channel sub-scores, verdicts, evidence ledgers) for benchmark pairs, browsable on TigerAI · human · n = 13,022 T-I pairs
Methods & tools
GPT-5 (gpt-5-2025-08-07), GPT-4o (gpt-4o-2024-08-06), GPT-3.5 Turbo (gpt-3.5-turbo-0125; excluded), OpenAI Python API library v2.14.0 (Responses, Streaming and Batch APIs), OpenAI Structured Outputs (strict JSON Schema), Integrated causal evidence score (ICES) rubric, MeSH ontology (therapeutic-area mapping), Katz log method for risk-ratio confidence intervals, R 4.3.3, Minikel et al. genetic_support code, Django 5.0 with Graphene-Django (GraphQL), SQLite, Apache Parquet and PostgreSQL on AWS
Reagents & environment
OpenAI Python API library v2.14.0 via client.responses.create with json_schema output; GPT-5 tools disabled, reasoning_effort and verbosity high; queries run November 2025; R 4.3.3; Django 5.0.
Experimental conditions
Inference via the OpenAI API only; approximately $0.12 per query (Table S1); 1,800 s per-request timeout for GPT-5; TigerAI hosted on AWS. No local hardware reported.
Reproducibility gaps
No source repository for TigerAI or the full developer prompt is linked; study-generated data are offered only through the tigerai.bio platform; model settings, cost and latency are given in supplementary Table S1.
Limitations (5)
  • Output quality depends on how complete the literature and resources behind LLM training are, so the generated evidence may inherit their biases and gaps despite the evidence constraints. · stated by the authors
  • The absence of drug or trial leakage cannot be guaranteed in principle. A curated-term scan flagged 253 of 13,022 pairs (2%), and manual review of all 253 found legitimate biology rather than leakage. · stated by the authors
  • LLM outputs can hallucinate and vary with prompting configuration. Robustness was checked by rerunning GPT-5 at scale through a different API (Fig. S5), and outputs are framed as decision support that complements expert judgment. · stated by the authors
  • With tools disabled, GPT-5 relies on internal knowledge that likely reflects the larger body of research on approved drug targets. A keyword scan cannot detect this kind of implicit outcome signal, which could inflate launched-programme coverage (312 vs 189). · noted in this summary
  • The analyses do not report auditing evidence-ledger citations for existence or accuracy, and GPT-5 vs curation RS differences (e.g., 2.74 vs 2.63) are not formally tested, so the RS advantage is descriptive. · noted in this summary

Read this if you prioritize drug targets using human genetic evidence and want to know whether prompted LLMs such as GPT-5 can extend or substitute for expert curation in predicting clinical success.

Citations
0

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISTu '23Yin '25Zhang '26Gazzo '26Penadés '25Furlan '25
55% semantic matchGenetics
Tao Tu, Zhuoqing Fang, Zhuanfen Cheng+5 more·bioRxiv·Nov 9, 2023· 8 citations