henhaobio
Sign in

bioRxivScientific Communication & EducationPreprint

React-to-Me: A Conversational Interface for Interactive Exploration of the Reactome Pathway Knowledgebase.

Helia MohammadiiD, Fatemeh AlmodaresiiD, Gregory F J Hogue, Adam WrightiD, Marija Orlic-MilaciciD, Nancy T. LiiD, Amin MawaniiD, Lincoln David SteiniD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

The Reactome Pathway Knowledgebase (www.reactome.org) provides expert-curated information on human biological pathways, molecular interactions, and disease mechanisms. However, its complex data model and keyword-based search interface present accessibility barriers for non-expert users. In contrast, general-purpose conversational AI systems offer intuitive natural language interfaces but lack the domain-specificity, transparent sourcing, and factual reliability required for scientific applications. To address this gap, we developed React-to-Me (https://reactome.org/chat), a domain-specific conversational assistant that enables users to query Reactome using natural language while maintaining scientific rigor and source traceability. React-to-Me integrates hybrid retrieval-augmented generation (RAG) with constrained language model generation to ensure that all responses are grounded in curated Reactome content and directly linked to corresponding knowledgebase entries. When internal coverage is insufficient, the system defers to trusted external biomedical sources rather than generating speculative or unverified content. Computational benchmarking confirmed that combining semantic vector search with keyword-based matching substantially improved contextual grounding and factual precision relative to dense-only retrieval baselines. In blinded expert evaluations, grounded responses were more likely to receive higher quality ratings than ungrounded counterparts, with significant gains in factual accuracy, biological specificity, and mechanistic depth. User surveys further indicated strong satisfaction with ease of use, citation reliability, and factual accuracy. These findings demonstrate that domain-specific grounding can markedly improve the reliability and usability of conversational AI for biological knowledge exploration. React-to-Me provides a transparent and scientifically robust interface for accessing and exploring Reactome content and is freely available at https://reactome.org/chat. Graphical Abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=130 SRC="FIGDIR/small/693752v1_ufig1.gif" ALT="Figure 1"> View larger version (37K): org.highwire.dtl.DTLVardef@1ef0d73org.highwire.dtl.DTLVardef@15c2d4dorg.highwire.dtl.DTLVardef@a2338corg.highwire.dtl.DTLVardef@e639fd_HPS_FORMAT_FIGEXP M_FIG C_FIG

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (12 Dec 2025) · CC BY 4.0

React-to-Me, a retrieval-augmented chatbot grounding GPT-4o-mini in curated Reactome pathways with hyperlinked citations, earned higher blinded expert ratings than ungrounded GPT-4o-mini (odds ratio 2.01).

Reactome's legacy keyword search needs exact nomenclature and returns fragmented entity hits without a synthesized explanation, while general-purpose chatbots hallucinate and lack source transparency. React-to-Me pairs conversational LLM access with retrieval from curated Reactome records and per-claim citations, and is evaluated with retrieval benchmarking, blinded expert ratings and a user survey.

Key results

  1. ShownBlinded experts rated answers higher when the model was grounded in Reactome than when the same model ran without retrieval, pooled across all metrics and questions.

    Cumulative-link mixed model on 450 paired ratings (10 experts × 15 questions × 3 metrics): OR = 2.01 [95% CI 1.50-2.69], Holm-adj p < 0.01; mean gain +0.26 points on a 4-point scale; CLES = 0.60 ± 0.03; React-to-Me preferred in 71.6% of paired comparisons (154 vs. 61; 235 ties); all ten evaluators preferred it overall. · Expert Evaluation of Grounded Vs. Open-domain…; Grounding yields significant system wide…; Tables

  2. ShownFactual accuracy and granularity improved the most. Relational depth improved less, but the gain was still significant.

    Factual accuracy OR = 2.97 [1.68-5.24], better in 82% of pairs, 'Excellent' 52% vs 38% for GPT-4o-mini; granularity OR = 2.88 [1.62-5.11], 71% of pairs, mean Δ +0.33, rater-level preference not significant (8 of 10 raters, p = 0.0547); relational depth OR = 1.83 [1.08-3.09], Holm-adj p < 0.05, 65% of pairs. · Grounding yields significant system wide…; Figure legends; Tables

  3. SuggestedAdding BM25 lexical search to dense retrieval and merging the two with Reciprocal Rank Fusion raised context utilization and faithfulness over dense-only retrieval. Relevance was similarly high for both.

    Ragas evaluation on 1,000 synthetic Reactome-derived questions in four types (Simple, Reasoning, Multi-context, Conditional): Advanced RAG beat Basic RAG on context utilization, relevance and faithfulness; dense-only showed consistent faithfulness deficits. Numeric scores are not given in the text. · RAG Module; Computational Benchmarking of Knowledge…; Figure legends

  4. ShownGrounding helped on both factual-recall questions and multi-step reasoning questions. The mean gain was slightly larger on reasoning items.

    Query-like: OR = 1.89 [1.34-2.66], p < 0.01, mean gain +0.24, better in 71% of pairs; reasoning: OR = 2.20 [1.17-4.16], p < 0.05, mean gain +0.29, better in 72.5% of pairs; nine of ten evaluators favored React-to-Me for each question type. · Grounding yields significant system wide…; Tables

  5. SuggestedCommunity users gave the highest marks to ease of use, link correctness and factual accuracy. Of all the metrics, factual accuracy correlated most strongly with overall satisfaction.

    27 respondents over 29 weeks: favourable ratings for ease of use 92%, link correctness 88%, factual accuracy 85%; Pearson correlation with overall satisfaction 0.81 (factual accuracy), 0.72 (relevance), 0.69 (link correctness), 0.63 (ease of use) vs 0.21 (conciseness) and 0.33 (logical consistency). · User Satisfaction; Discussion

Methods, data and reproducibility

Five-stage pipeline: queries are normalized and screened for scope by an LLM classifier, then hybrid dense-embedding plus BM25 retrieval over Reactome records is merged by Reciprocal Rank Fusion and passed to GPT-4o-mini at temperature 0.0; an LLM sufficiency check triggers fallback links to external resources. Evaluated with Ragas, blinded expert ratings and a user survey.

  • Reactome Pathway Knowledgebase records (pathways, reactions, molecular entities, annotations) indexed for dense and sparse retrieval · human (Reactome curates human pathways)
  • Synthetic question set generated from Reactome content with the Ragas test set workflow, spanning four query types · human (derived from Reactome content) · n = 1,000 questions
  • Blinded expert evaluation of paired React-to-Me vs GPT-4o-mini responses to molecular biology questions, scored on a four-point rubric · n = 10 experts; 15 questions; 450 paired ratings · 10.5281/zenodo.17602575
  • Online user satisfaction survey across eight metrics, recruited via institutional bulletins and scientific networks · n = 27 anonymous participants
Methods & tools
GPT-4o-mini (OpenAI), Retrieval-Augmented Generation (RAG-Fusion), Dense semantic vector embeddings, BM25 sparse lexical retrieval, Reciprocal Rank Fusion (RRF), Ragas (Retrieval-Augmented Generation Assessment), Cumulative-link mixed-effects ordinal regression, ICC(2,1) inter-rater reliability, Wilcoxon and sign tests, Holm multiple-testing correction, Pearson correlation, R (statistical analysis scripts)
Software & environment
Production deployment uses OpenAI GPT-4o-mini with generation temperature 0.0; source code and deployment instructions are under the MIT license; statistics and figures are scripted in R (/analysis directory). No library versions or containers are stated.
Reproducibility gaps
Depends on the commercial GPT-4o-mini API. The main text names neither the embedding model nor the prompts and defers module details to Supplementary Methods, not included here. It does not say whether the 1,000-question Ragas test set or the user survey responses are released.
Limitations (6)
  • Scope is limited to human pathways already curated in Reactome, so model organism, comparative and cross-species questions are unsupported. Coverage also trails newly reported mechanisms and rare interactions that have not yet been curated. · stated by the authors
  • Inconsistent or hallucinated statements can still occur because generation is probabilistic, especially for questions near the edge of Reactome's curated content or with ambiguous biological terminology. · stated by the authors
  • The deployment relies on the commercial GPT-4o-mini API, which raises cost, data governance and sustainability concerns. Moving to open-source or locally hosted models depends on their domain performance and on infrastructure support. · stated by the authors
  • The survey subgroup analyses by biology education and Reactome experience were underpowered because the sample was small (27 respondents), so the expertise trends are descriptive only. Respondents also flagged clarity, conciseness and logical consistency as areas to improve. · stated by the authors
  • Inter-rater reliability among experts was low (ICC(2,1) -0.05 to 0.18). The text also does not say whether Reactome hyperlinks were removed from React-to-Me answers; if they were not, blinded raters could have identified the grounded system. · noted in this summary
  • The retrieval benchmark uses synthetic questions generated from the same Reactome corpus being indexed, and the text reports no numeric scores or significance tests for it. The expert study compares only against ungrounded GPT-4o-mini, with no other grounded system as a baseline. · noted in this summary

Read this if you build or evaluate retrieval-augmented chatbots over curated biological knowledgebases, or want a blinded expert-rating design analyzed with mixed-effects ordinal models.

Citations
0

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISPunuru '26Scheibe '25O’Neil '24Darnell '25Yip '25Yu '26
64% semantic matchBioinformatics
Pranav Punuru, Nabil Ibtehaz, Swagarika Jaharlal Giri+3 more·Journal of Biomedical Semantics·Aug 12, 2026· 1 citations