Hybrid Semantic-Lexical Retrieval for Source-to-CDE Mapping in the NCI caDSR

Abstract

Accurate mapping of local source data elements to standardized common data elements (CDEs) is critical for biomedical data integration but is challenged by short, abbreviated, and heterogeneous source descriptions. We constructed a benchmark of approximately 69,000 expert-linked source-to-CDE mappings from the National Cancer Institute\textquotesingle s caDSR and evaluated multiple bi-encoder models with two-stage fine-tuning. Here we developed a hybrid retrieval system combining semantic bi-encoder retrieval, CDE Match-inspired keyword retrieval, cross-encoder scoring, and supervised reranking. This system achieved Recall@5 ranging from 0.802 to 0.972 across internal and five distribution-shifted holdouts, outperforming the other evaluated methods on five of six datasets and approaching ceiling performance on the sixth dataset (GDC). These results demonstrate the effectiveness and robustness of integrating semantic and lexical evidence for source-to-CDE mapping, supporting improved interoperability and reuse of biomedical metadata across diverse clinical and research contexts.