henhaobio
Sign in

bioRxivBioengineeringPreprint

BoneGraph: A Domain-Specialised, Self-Correcting Reasoning System for Bone Science Retrieval, Grounded Inference, and Image Mechanics

Jon ValijonoviD, Peter SoariD, James Le HouxiD, Gianluca TozziiD

PDFDOICode
In-depth full-text digest available · Jump to digest ↓

Abstract

Bone science literature spans biology, mechanics, materials science, and clinical medicine, and its volume makes reliable knowledge synthesis increasingly difficult. General-purpose large language models (LLMs) answer fluently but under-represent this niche domain, cannot cite specific evidence, and offer no mechanism to be corrected durably. Here we present BoneGraph, a domain-specialised system for bone science delivered as a five-tab web application over a shared substrate: a curated full-text corpus of 7,449 documents embedded into 248,629 passage vectors using SPECTER2, a scientific-paper embedding model, and a bone knowledge graph of 1,597 concepts with 1,699 causal relations. The five tabs are: (I) Chat, retrieval-augmented question answering with server-rebuilt inline citations; (II) Search, raw semantic retrieval with no LLM in the loop; (III) Reasoning, a self-correcting loop in which a deterministic physics check and a literature/knowledge-graph critic constrain the answer, and a user's feedback becomes a durable, per-user rule; (IV) Vision, a bone-region classifier trained on frozen BiomedCLIP features that grounds a vision-language model, guarded against out-of-distribution inputs and augmented with image-embedding correction memory; and (V) Mechanics, integrating our previous data-driven image mechanics (D2IM) model that predicts displacement and strain fields from a single undeformed micro-CT image. All inference is performed locally, without third-party API calls, and the public beta is served at bonegraph.org. Retrieval attains a mean reciprocal rank (MRR) of 0.928 on a 30-question, seven-domain benchmark, and the Vision classifier attains 92.6% accuracy on the held-out MURA (MUsculoskeletal RAdiographs) dataset. A grounded-reasoning benchmark shows that, with the correct passage, BoneGraph raises answer accuracy from 42% to 78%. BoneGraph makes a major contribution to bone-science informatics: to our knowledge it is the first domain-specialised system to unify curated retrieval, deterministic physics-grounded self-correction, and durable per-user learning for bone science.

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (18 Sept 2026) · CC BY 4.0

BoneGraph combines a SPECTER2-indexed bone corpus, knowledge graph, deterministic physics checks, and user-taught rules in a local five-tab system; oracle passages raise accuracy 42% to 78%.

General-purpose large language models (LLMs) under-represent bone science, cannot cite specific evidence, and lack a durable way to accept expert corrections, while retrieval-augmented generation grounds answers without checking physical consistency. BoneGraph pairs bone-curated retrieval with deterministic physics checks, an evidence-grounded critic, and editable per-user rules, all served on local hardware without third-party API calls.

Key results

  1. ShownGiving HuatuoGPT-o1-8B the passage that holds the answer clearly improves its accuracy on calculation questions built from the corpus. Neither keyword retrieval condition differed significantly from answering with no retrieval.

    50 multiple-choice questions, structured prompt, temperature 0: oracle passage 39 of 50 vs no retrieval 21 of 50 (18 questions, 36 percentage points; significant by exact McNemar test); BM25 keyword 26 of 50 and LLM reranking 29 of 50; reranking vs oracle gap still significant. · 2. Results and Discussion; Tables

  2. SuggestedThe authors trace the retrieval shortfall to evidence delivery. Keyword conditions supplied the deciding quantity for only 31 of 50 questions, and under the structured prompt accuracy followed this needle-presence, which MRR does not capture.

    Structured prompt: reranking correct on 24 of 31 needle-present vs 5 of 19 needle-absent questions; keyword 20 of 31 vs 6 of 19. Direct prompt: 16 of 31 vs 7 of 19 and 15 of 31 vs 8 of 19. Prompt rewording moved every condition by 6-12 percentage points, with no single comparison significant. · 2. Results and Discussion; Tables

  3. SuggestedOn a small hand-built benchmark, the deployed dense retrieval puts relevant passages at or near the top. The remaining misses are ranking slips rather than gaps in coverage.

    30 questions across seven domains, k = 10: overall mean reciprocal rank (MRR) 0.928; lowest per-domain MRR in simulation (0.833) and biomaterials (0.867); whenever the top hit missed, a relevant passage appeared second or third. Five fracture-domain literature probes returned on-topic passages at cosine 0.78 to 0.84. · 2. Results and Discussion; Tables

  4. ShownA small head trained on frozen BiomedCLIP features classifies upper-limb body region accurately. A nearest-neighbour cosine guard separates in-scope radiographs from out-of-scope inputs, but abnormality detection remains weak.

    MURA patient-disjoint validation: MLP 92.6% accuracy (macro-F1 0.918) vs linear probe 89.6% (0.886), seven-class chance 14%, every region above 85%; abnormality MLP 74.2% (macro-F1 0.735), abnormal recall 61%. Guard cosine: MURA 0.83 to 0.99 (median 0.95) vs noise/flat fields 0.47 to 0.50; threshold 0.827. · 2. Results and Discussion; 4.7 Vision: A Trained Classifier Grounding a…; Figure legends

  5. SuggestedSingle worked examples show user corrections persisting without any weight updates. A taught comparative rule reversed a false osteoporosis premise when the question was asked again, and image-embedding memory recalled a femur correction on a rotated copy of the scan.

    Reasoning: first answer passed all eight built-in rules and critic review; after the llama3.2:3b-compiled rule, it passed nine rules and the critic accepted after one revision. Vision: femur labelled humerus at 68%; a 20° rotated copy matched the stored correction at 96% similarity, yielding femur. · 2. Results and Discussion; Figure legends

Methods, data and reproducibility

A FastAPI/React application exposes five pipelines over shared SQLite stores: SPECTER2 cosine retrieval for Search and HuatuoGPT-o1-8B Chat with server-rebuilt citations; an agent-critic loop with eight deterministic physics rules, one-hop knowledge-graph edges, and user rules; BiomedCLIP-head-grounded llava:13b Vision; and D^2IM strain prediction.

  • OpenAlex metadata for bone-science works retrieved by a 133-query, 17-group keyword taxonomy, spanning 1970–2026 · n = 54,634 unique papers
  • Validated open-access full-text PDFs (three-tier URL resolution via publisher patterns, CrossRef, Unpaywall) plus open-access textbooks, English-filtered · n = 7,674 PDFs; 7,433 English papers + 16 textbooks
  • SPECTER2 passage index of sentence-aware chunks (about 400 tokens, 768-dimensional vectors) stored in SQLite · n = 248,629 chunks (246,646 papers; 1,983 textbooks)
  • Bone knowledge graph: hand-curated seed (about 200 concepts, 80 causal edges) extended by HuatuoGPT-o1-8B triple extraction and six-stage cleanup · n = 1,597 nodes; 1,699 edges
  • MURA Stanford musculoskeletal radiographs, seven upper-limb regions, patient-disjoint train/validation splits · human (patient radiographs) · n = 36,808 radiographs
  • Manually constructed retrieval benchmark with expected-answer keywords from the ground-truth literature · n = 30 questions, 7 domains
  • LLM-generated multiple-choice calculation questions authored from corpus passages, each linked to its source chunk and quoted sentence · n = 50 questions; 400 model calls
  • Publicly available D^2IM porcine vertebral micro-CT slices used to train the bundled Mechanics weights · pig (porcine)
Methods & tools
SPECTER2 (proximity and ad-hoc query adapters), HuatuoGPT-o1-8B (Chat, Reasoning agent and critic), llava:13b vision-language model, llama3.2:3b (topic guard, feedback-to-rule), BiomedCLIP image encoder, D^2IM displacement/strain CNN, OpenAlex API, CrossRef, Unpaywall, PyMuPDF, NLTK sentence tokenizer, BM25 (benchmark baseline), Ollama and Hugging Face weights, FastAPI, Uvicorn, React, SQLite, Cloudflare Tunnel
Software & environment
Ollama and Hugging Face weights for local inference; FastAPI/Uvicorn backend with React frontend; SQLite stores; TensorFlow lazy-imported and optional for D^2IM; model tags llava:13b and llama3.2:3b; no library versions stated.
Assay conditions & compute
Served from one NVIDIA Jetson AGX Orin behind a Cloudflare Tunnel; developed on an Apple M3 Pro MacBook Pro (11-core CPU, 14-core GPU, 18 GB unified memory); ≈1.5 GB of embeddings held in memory give sub-100ms retrieval.
Reproducibility gaps
The repository omits the corpus and knowledge-graph databases, which users must regenerate by running the documented pipeline; paywalled full texts are excluded; the Zenodo archive is mentioned without an identifier.
Limitations (6)
  • Full text covers only open-access PDFs reachable via OpenAlex and the resolution pipeline; paywalled papers exist only as metadata, skewing body-text coverage toward open-access venues and preprints. · stated by the authors
  • The knowledge graph is machine-extracted, fragmented, and not checked edge by edge for fractures, and chained paths may imply false conclusions, so it serves only as one-hop evidence for the critic. · stated by the authors
  • Both benchmarks are small and single-run: 30 hand-built retrieval questions with keyword relevance, and 50 LLM-generated calculation questions checked only for the source quantity, not by experts; keyword conditions were tuned on those same questions. · stated by the authors
  • Perception models are narrowly scoped: the Vision classifier knows only upper-limb radiographs (MURA), so CT, MRI, and micro-CT fall back to the bare VLM, its abnormality head (74.2%, recall 0.61) is preliminary, and D^2IM generalizes best to similar vertebral micro-CT slices. · stated by the authors
  • The 42% to 78% gain uses oracle passages; the deployed SPECTER2 dense retriever was not evaluated on the grounded benchmark, and the BM25 conditions tested did not differ significantly from closed-book answering. · noted in this summary
  • The critic, the eight physics rules, and Vision correction memory are illustrated only by single worked examples, with no quantitative rate of errors caught or false flags, and no comparison against other domain QA systems. · noted in this summary

Read this if you build retrieval-augmented LLM tools for bone or biomechanics research and want a locally hosted design with deterministic rule checks, user-editable corrections, and needle-presence evaluation.

Citations
0

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISMohammadi '25Kinjo '26Joy '26Mohamed '26Yu '26Taylor '26
53% semantic matchScientific Communication & Education
Helia Mohammadi, Fatemeh Almodaresi, Gregory F J Hogue+5 more·bioRxiv·Dec 12, 2025