henhaobio
Sign in

bioRxivAnimal Behavior & CognitionPreprint

Grounding olfactory perception in language: Benchmarks and models for generating natural language odor descriptions

Cyrille MascartiD, Khue Tran, Khristina V. Samoilova, Liam Thomas Storan, Tingkai LiuiD, Alexei A. KoulakoviD

PDFDOICode
In-depth full-text digest available · Jump to digest ↓

Abstract

Recent advances in deep learning have enabled prediction of odorant perception from molecular structure, opening new avenues for odor classification. However, most existing models are limited to predicting percepts from fixed vocabularies and fail to capture the full richness of olfactory experience. Progress is further limited by the scarcity of large-scale olfactory datasets and the lack of standardized metrics for evaluating free-form natural-language odor descriptions. To address these challenges, we introduce Odor Description and Inference Evaluation Understudy (ODIEU), a benchmark which includes perceptual descriptions of over 10,000 molecules paired with a model-based metric for evaluating free-form odor text descriptions. The model-based metric uses Sentence-BERT (SBERT) models which are fine-tuned on olfactory descriptions to allow better evaluation of human-generated odor texts. Using the fine-tuned SBERT models, we show that free-form text odor descriptions contain additional perceptual information in their syntactic structure compared to semantic labels. We further introduce CIRANO (Chemical Information Recognition and Annotation Network for Odors), a transformer-based model that generates free-form odor descriptions directly from molecular structure, thus implementing the molecular structure-to-text (S2T) prediction. CIRANO achieves performance comparable to humans. Finally, we generate human-like descriptions from mouse olfactory bulb neural data using an invertible SBERT model, yielding neural-to-text (N2T) predictions highly aligned with human descriptions. Together, CIRANO and ODIEU establish a standardized framework for generating natural language olfactory descriptions and evaluating their alignment with human perception. Code is available at https://github.com/KoulakovLab/ODIEU

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (5 Mar 2026) · CC BY 4.0

ODIEU pairs human odor texts for 10,124 molecules with olfaction-tuned Sentence-BERT evaluators; CIRANO, a frozen GPT-2 with a molecular prefix, generates descriptions approaching human agreement.

Structure-to-odor models usually predict labels from fixed vocabularies and are judged by AUC, which may miss the richness of free-form perception, and no standard metric existed for scoring free-text odor descriptions. This work supplies a benchmark calibrated to human-to-human agreement, a structure-to-text generator, and a route from mouse bulb activity to language.

Key results

  1. ShownCIRANO, a frozen GPT-2 conditioned by a learned molecular prefix, produces odor descriptions closer to human text than the other structure-based predictors tested, though still below human-to-human concordance.

    Held-out cosine similarity (CS) to completed human descriptions (CHD), fine-tuned MiniLM evaluator: 0.51 (0.003) with DeepNose inputs, 0.499 (0.004) with Mordred, vs 0.58 (0.01) for CHD-CHD; STELLA evaluator: 0.45 (0.004) and 0.44 (0.004) vs 0.55 (0.02). Scores exceed DeepNose label-based (CDL) descriptions. · Training an LLM to generate S2T odor…; Discussion; Figure legends

  2. ShownFine-tuning Sentence-BERT encoders on olfactory text with a contrastive objective gives an evaluator that separates descriptions of the same molecule from random pairs. This evaluator forms the ODIEU metric, which is anchored to human-to-human agreement.

    Pretrained vs fine-tuned (AnglE loss, 5-fold cross-validation): EMD between same-molecule and shuffled CS distributions 0.14 vs 0.40; top-k retrieval at k=20 38% vs 64% (STELLA-400M-v5). Model-free BLEU-1: human-to-human Shh = 59 vs Smh = 46 for DeepNose label-based descriptions. · Evaluating human odor descriptions using model…; Evaluating human descriptions using sentence…; Figure legends

  3. SuggestedTwo humans' completed sentence descriptions agree with each other more than with sentences rebuilt from semantic labels for the same molecules. This suggests that syntax carries perceptual information that labels lack.

    Fine-tuned SBERT CS for CHD-CHD exceeded CHD vs completed human labels (CHL) (p<0.01, t-test); BLEU-1 was Shh = 59 between humans vs approximately 51 for CHD vs CHL, computed over 4,324 independent description pairs. · Evaluating human odor descriptions using model…; Application of ODIEU benchmark: the role of…

  4. SuggestedMouse olfactory bulb glomerular responses, linearly aligned to an invertible SBERT space, can be decoded into human-like odor sentences. These sentences score above chance but below human agreement.

    Q-vectors (D = 30) for N = 59 odors from two mice; Procrustes map with leave-one-out validation. CS of mouse-derived vs CHD: 0.43 (MiniLM) and 0.36 (STELLA) vs human-to-human 0.58 and 0.55 and chance 0.25 and 0.14; embedding CS above shuffled odorants (t-test, p<10^-3). · Predicting odor descriptions using mouse neural…; Figure legends

  5. ShownOdor spaces from neural networks (DeepNose, OpenPOM), from fine-tuned SBERT embeddings and from Isomap embeddings of semantic datasets share substantial structure and can be linearly predicted from one another.

    Sum of squared canonical correlations about 21 for DeepNose vs POM, and 13 and 15 for fine-tuned MiniLM vs DeepNose and POM. Held-out (20%) Pearson R: DeepNose to POM 0.85, MiniLM to STELLA 0.86, SBERT to GoodScents/Leffingwell Isomap spaces 0.62-0.72. · Using ODIEU to define the odor space; Discussion

Methods, data and reproducibility

Llama 3.3 rewrites odor phrases and labels from Leffingwell, Arctander and GoodScents into sentences; SBERT models fine-tuned with AnglE contrastive loss score generated text against human-to-human cosine similarity. CIRANO feeds DeepNose or Mordred features through a 4-layer MLP prefix into frozen GPT-2; mouse Q-vectors map to invertible SBERT via Procrustes.

  • ODIEU dataset: Leffingwell, Arctander and GoodScents free-text odor descriptions and semantic labels, with 3D coordinates, .mol files and SMILES · human (perceptual descriptions) · n = 10,124 molecules
  • Molecules described independently in more than one source dataset, used as the human-to-human reference · human · n = 1,762 in two / 854 in three datasets; 4,324 pairs
  • Llama 3.3 augmented texts: completed human descriptions (CHD) and labels (CHL), SMILES zero-shot and 50-shot outputs, and DeepNose-label sentences (CDL)
  • Previously recorded mouse olfactory bulb glomerular odor responses embedded in 30-dimensional Q-space · mouse · n = two mice; N = 59 odors overlapping ODIEU
  • Comparison odor spaces: DeepNose latent space, OpenPOM embeddings, and Isomap embeddings of GoodScents and Leffingwell labels
Methods & tools
Llama 3.3 70B-Instruct, ALL-MiniLM-L12-V1 (33.4M parameters), STELLA-400M-v5 (435M parameters), AnglE contrastive loss, GPT-2 (frozen backbone), DeepNose equivariant CNN, Mordred fingerprints (1,623 descriptors), OpenPOM, invertible SBERT (invSBERT), UMAP, MDS, Isomap, Canonical Correlation Analysis; Procrustes alignment, BLEU, ROUGE, METEOR, BERTScore
Reproducibility gaps
No separate data URL, accession, software versions or hardware are stated, so the dataset's release location is unspecified beyond the code repository. The original POM latent space is not public, so the OpenPOM replica was used instead.
Limitations (6)
  • CIRANO's similarity to human descriptions stays below human-to-human consensus under both fine-tuned evaluators. GPT-2's language ability was preserved, but with rare exceptions in the generated outputs. · stated by the authors
  • Only 59 odors overlapped with the bulb recordings from two mice, too few to train a dedicated neural-to-text generator. Mouse-derived sentences fall short of both human agreement and CIRANO. · stated by the authors
  • Averaging over only three datasets can inflate average-to-individual similarity because the average contains the compared individual. Excluding it lowers CS to 0.61 (MiniLM) and 0.58 (STELLA), and sentences decoded from averaged embeddings were not better than cross-individual ones. · stated by the authors
  • Individual cultural and genetic variability may keep odor descriptions from perfectly reflecting perception. Separately, because averaged embeddings slightly outperform single individuals, a model trained to predict average responses might exceed the human-consensus benchmark. · stated by the authors
  • Both the human reference sentences (CHD) and the label-derived sentences (CHL) come from Llama 3.3 rewriting, so the syntax effect and the human ceiling may partly reflect the augmentation prompt. Robustness to a different rewriting model is not reported. · noted in this summary
  • The abstract calls CIRANO comparable to humans, yet its CS stays below human concordance (0.51 vs 0.58 with MiniLM; 0.45 vs 0.55 with STELLA), and no significance test for that gap is reported. · noted in this summary

Read this if you model structure-odor relationships or olfactory coding and want a human-calibrated benchmark and baselines for generating free-text odor descriptions from molecules or neural data.

Citations
0

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISGutiérrez '18Dhurandhar '23Keller '17Tran '18McConachie '25Lee '23
80% semantic matchNeuroscience
E. Darío Gutiérrez, Amit Dhurandhar, Andreas Keller+2 more·bioRxiv·May 25, 2018· 1 citations