A Study of Calibration as a Measurement of Trustworthiness of Large Language Models in Biomedical Research
Rodrigo de Oliveira, Matthew Garber, James M. Gwinnutt, Emaan Rashidi, Jwu-Hsuan (Shantina) Hwang, William Gilmour, Jay Nanavati, Khaldoun Zine El Abidine, Christina DeFilippo Mack
This study evaluates the calibration of 9 LLMs in BioNLP tasks using Flex-ECE and finds that post-hoc calibration significantly improves model reliability.
JAMIA Open · Jul 3, 2025Peer-reviewedBioinformatics