henhaobio
Sign in

bioRxivBioinformaticsPreprint

Can AI reproduce observed chemical diversity?

Mostapha BenhendaiD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Generating diverse molecules with desired chemical properties is important for drug discovery. The use of generative neural networks is promising for this task. To facilitate evaluation of generative models, this paper introduces a metric of internal chemical diversity, and raises the following challenge: can a nontrivial AI model reproduce observed internal diversity for desired molecules? To illustrate this metric, a mini-benchmark is performed with two generative models: a Reinforcement Learning model and the recently introduced ORGAN. The aim of this paper is to encourage research about internal diversity metrics.

In-depth summary

Claude Sonnet 4.6 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (30 Mar 2018) · CC BY 4.0

An average-pairwise Tanimoto internal diversity metric shows reinforcement learning and ORGAN generators lose diversity among desired molecules, ending far below reference-set diversity.

Generative-model papers often judge samples visually or with metrics not best suited to drug discovery, and ORGAN has been argued to be less repetitive than RL on visual grounds alone. This paper adds an internal diversity metric and a challenge: match the diversity of observed molecules that have a desired property.

Key results

  1. ShownInternal diversity is defined as the mean pairwise Tanimoto distance within a single generated set, using Morgan fingerprints. In a toy 2D example, a generator collapsed to one point scores external diversity 1 but internal diversity 0.

    Toy setting: training points on a unit circle, all generated samples at the origin. External diversity equals 1; internal diversity equals 0. · 2 The metric of internal chemical diversity; 2.1 Internal diversity; 2.2.1 Internal vs. external diversity metric [8…

  2. ShownDopamine D2 task: RL after 30 steps reaches overall internal diversity above the D2-active reference set (DRD2). Among molecules with Pa > 0.8, diversity falls to vanishingly small levels.

    32k samples per output; reference is 8324 D2-active ExCAPE-DB molecules. The Pa > 0.8 column is the key comparison. Text gives no raw table values for RL. · 4.1 Dopamine D2 activity; Tables

  3. ShownDopamine D2 task, ORGAN: with lambda 0.04 and 30 steps, the Pa > 0.8 subset is about 2 orders of magnitude more diverse than RL 30, yet 1 order below the DRD2 baseline. At 60 steps it falls back to RL-like levels. With lambda 0.5, D2 learning had not started.

    At lambda 0.5 after 60 steps, only 6 of 32k samples had Pa > 0.8, so diversity for that subset was undetectable. · 4.1 Dopamine D2 activity

  4. SuggestedDruglikeness task: ORGAN (lambda 0.8, 200 steps) lifts diversity among L > 0.8 molecules to detectable levels, unlike RL, but stays far below the ZINC reference. The author concludes both models fail to reproduce observed diversity here.

    6400 samples per output. ORGAN diversity for L > 0.8 is 2 orders of magnitude below ZINC for L > 0.8 and 3 orders below total ZINC diversity. · 4.2 Druglikeness; 5 Conclusion and additional future work

  5. ClaimedThe author states the ORGAN discriminator quickly becomes perfect, which he treats as a training problem and reads, together with the generator's failure to fool it, as a sign of generator weakness. A min(x, y)-type reward did not work in his experiments, which he attributes to this perfect-discriminator problem.

    Discriminator-assigned probability for generated samples stays below 0.5. No data or table for this is shown in the text. · 5.1 The perfect discriminator problem; 5.2 Imbalance in multi-objective RL

Methods, data and reproducibility

Defines internal diversity as mean pairwise Tanimoto distance over Morgan fingerprints. Applies it to LSTM SMILES generators trained by RL and by ORGAN (lambda-weighted discriminator reward), on a dopamine D2 activity reward and a druglikeness reward.

  • Random 15k-molecule subset of the ZINC database (35 million commercially available compounds), used for maximum-likelihood pre-training and as the druglikeness baseline · n = 15k molecules
  • D2-active reference set from ExCAPE-DB (DRD2), essentially the training set of the D2 activity classifier · n = 8324 molecules
  • Generated SMILES samples for the D2 experiment (MLE, RL 30, RL 60, ORGAN-0.04 at 30 and 60 steps, ORGAN-0.5 at 60 steps) · n = 32k samples per output
  • Generated SMILES samples for the druglikeness experiment (RL 200, ORGAN 200) · n = 6400 samples per output
Methods & tools
RDKit, Morgan fingerprints, Tanimoto distance, LSTM recurrent neural network, ORGAN, SeqGAN, REINVENT D2 activity predictive model, ZINC, ExCAPE-DB
Limitations (4)
  • The conclusions hold only for small training sets, small architectures and the specific hyperparameters tested; larger datasets, more and larger models and wider hyperparameter sweeps are left for future work. · stated by the authors
  • The metric relies on Morgan fingerprints, which are manually designed features and may not always reflect true chemical distance. · stated by the authors
  • Each configuration appears to be a single run with no repeats or variability estimates reported, so the modest RL versus ORGAN differences are hard to judge. · noted in this summary
  • The metric, challenge and benchmark are all designed by the same author, and the diversity reference comes only from the ZINC and ExCAPE-DB sets used here, not from broader observed chemistry. · noted in this summary

Read this if you evaluate molecule generators and want a simple internal diversity metric, or a small benchmark showing diversity loss among property-satisfying molecules.

Citations
59

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISDaina '25Kumar '26Oestreich '25Reboul '25Chen '25Fan '26
60% semantic matchBioinformatics
Antoine Daina, Vincent Zoete·bioRxiv·Dec 7, 2025· 1 citations