A Guided AI Framework for Customizable and Efficient Harmonisation to the OMOP Common Data Model
Nishu Nehra, Rohit Swami, Dharani Dadi, Ritika Mishra, Umang Sharma, Pawan Verma, Manimala Sen, Nobal Kishor Dhruw, Abhishek JhaiD
Abstract
Getting clinical data from different sources to "talk" to each other within the OMOP Common Data Model (CDM) is arguably the most tedious part of multi-center research. While this integration is essential, the transformation process is frequently a manual grind, requiring a rare overlap of deep clinical knowledge and technical expertise. In this paper, we present a framework designed to alleviate some of the burden on the researcher by automating data harmonization through two distinct steps: structural schema mapping and terminological standardization. For the structural piece, we moved away from "black box" logic in favor of a stateful workflow managed by large language models (LLMs) and directed acyclic graphs. By profiling EHR data at the source, our system generates context-aware dictionaries that offer ranked mapping suggestionsalongside confidence scores. While our benchmarking showed a 97.5% agreement rate at the schema level and an 84% agreement rate at the value level when compared with human experts, the system appears most effective when treated as a "co-pilot" rather than a total replacement for human oversight. To handle value-level standardization, we implemented a hybrid search strategy that pairs the semantic depth of SapBERT embeddings with the literal precision of fuzzy string matching. By using FAISS for rapid similarity retrieval, the engine attempts to resolve messy or "noisy" clinical descriptionsto standard OMOP concepts. This approach seems particularly promising for handling the non- standardized labels that often plague smaller, local datasets. Ultimately, our results suggest that this guided approach can shift the timeline for OHDSI-compliant warehousing from weeks of manual curation to a more manageable and scalable pipeline, potentially lowering the barrier to entry for smaller research teams.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 0
- Assay
- Research Focus
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- Metadata Harmonization from Biological Datasets with Language Models
Alexander Verbitsky, Patrick Boutet, Mohammed Eslami·Bioinformatics AdvancesPeer-reviewed·Dec 26, 2024
- MetaHarmonizer: robust biomedical metadata harmonization and a contamination control for inflated LLM performance on public benchmarks
Changchang Li, Abhilash Dahl, … Sehyun Oh·bioRxiv·Jun 17, 2026
- Semantic harmonization of Alzheimer's disease datasets using AD-Mapper
Philipp Wegner, Helena Balabin, … the Alzheimer’s Disease Repository Without Borders Investigators·Journal of Alzheimer s DiseasePeer-reviewed·May 15, 2024
- Automated Harmonization and Large-Scale Integration of Heterogeneous Biomedical Sample Metadata Using Large Language Models
Koichi Higashi, Zenichi Nakagawa, … Hiroshi Mori·bioRxiv·Oct 26, 2024
- The Generalized Data Model for Clinical Research
Mark D. Danese, Marc Halperin, … Ryan Duryea·BMC Medical Informatics and Decision MakingPeer-reviewed·Jun 24, 2019
- Large-scale Manual Curation and Harmonization of Metadata from Metagenomic and Cancer Genomic Repositories: Challenges and Solutions
Kaelyn Long, Kai Gravel-Pucillo, … Sehyun Oh·DatabasePeer-reviewed·Jan 1, 2026