henhaobio
Sign in

bioRxivBioinformaticsPreprint

Two Comparators May Be All We Need

Steven M. MuskaliD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

A compound in a cell meets a spectrum of proteins drawn from many families at once, while screening most often interrogates one target at a time. Two questions asked many times in a rank ordering workflow ultimately guide decisions on what gets made and what gets counter-screened, and both are comparative: which of two targets does a compound prefer, and which of two compounds does a target prefer. We built one model for each: the target comparison over a roster of 2,279 human proteins in 34 protein families, the compound comparison over 2,079 in 32. Each model is given two chemical structures and a sequence, or two sequences and a chemical structure, and returns the predicted preference and its prediction strength. No conformational analysis, protein structure, binding site or docked pose is used. The target comparator reached 0.75 accuracy on 8,689 held-out comparisons across families, increasing to 0.94 when the comparisons are filtered by prediction strength. Within one family it reached 0.78 on 32,738 held-out comparisons, increasing to 0.95. The compound comparator reached 0.71 on 65,725 held-out comparisons, increasing to 0.96. Neither compound in any of those comparisons appeared anywhere in training. Accuracy tracks how far apart the two measured potencies actually are. Where the two readings lie within half a log unit of each other the comparators are right 0.57 to 0.64 of the time, and where they differ by more than two log units, 0.90 to 0.92. The compound comparator was also scored one family at a time, where its accuracy ranged from 0.59 to 0.80 across the 31 of its 32 families that had held-out comparisons. Within the chemistry and targets represented in their training data, these models rank compounds and targets. Both models can be explored and downloaded at familyfoundationmodel.com.

In-depth summary

Claude Sonnet 4.6 · xhigh effort · 7 Nov 2026 · from the full text of version 3 (8 Oct 2026) · CC BY 4.0

Random forests on sequence and 2D chemistry pick the favored protein for a molecule, or the favored molecule for a protein, at 0.71 to 0.78 accuracy on unseen compounds.

Earlier models score a single protein-compound pair, so comparing two targets usually means subtracting two independent predictions. This work feeds the protein in as an input and extends the roster beyond one family, to 2,279 proteins in 34 families for the target comparator.

Key results

  1. ShownThe target comparator names the preferred target correctly 0.75 of the time across families and 0.78 within a family, on compounds never seen in fitting. Within-family accuracy varies by family, and kinase is the hardest large family.

    Across families: 0.75 on 8,689 comparisons from 2,195 unseen compounds. Within a family: 0.78 on 32,738 comparisons from 13,343 unseen compounds across 30 families. Range 0.69 to 0.94 over 17 families with 200 or more held-out comparisons; kinase 0.73 on 4,979. · 3.1 Ranking two targets; Abstract

  2. ShownThe compound comparator picks the preferred compound at a target 0.71 of the time on a strict compound-disjoint holdout. Accuracy varies by family, and kinase falls below the pooled figure.

    0.71 on 65,725 comparisons where neither compound was in fitting (626,130 straddling comparisons discarded). Per family 0.59 to 0.80 across 31 families with held-out comparisons; kinase 0.69 on 9,585. · 3.2 Ranking two compounds; 3.4 What governs a call; 2.4 Split and evaluation

  3. ShownPrediction strength works as a confidence filter: accuracy rises and coverage falls as the strength cutoff rises, in all three comparisons.

    At strength 0.80: target comparator 0.94 on 2,980 of 8,689 across-family comparisons and 0.95 on 12,529 of 32,738 within-family; compound comparator 0.96 on 3,995 of 65,725 (coverage 0.06). · 3.3 Prediction strength; 4.3 Using it to prioritize

  4. ShownAccuracy depends strongly on how far apart the two measured potencies are; near-ties are close to coin-flip territory for both comparators.

    Within half a log unit: 0.59 (2,264 across-family target comparisons), 0.64 (11,228 within-family), 0.57 (20,200 compound). Beyond two log units: 0.91, 0.92 and 0.90 respectively. · 3.4 What governs a call; Abstract; 4.2 What limits it

  5. ShownAcross-family target comparison is capped by the public record: very few compounds are measured in more than one family. Family pairing also matters, and the aminergic transporter versus GPCR pairing is harder.

    Of 785,745 compounds, 749,812 are measured in one family and 4.6% cross a boundary. Across-family accuracy: kinase vs voltage-gated ion channel 0.94 on 301; electrochemical transporter vs Family A GPCR 0.68 on 1,016. · 3.5 Cross-family coverage sets the across-family…; 3.4 What governs a call; 4.2 What limits it

Methods, data and reproducibility

Random forests (300 trees) take a feature row of ESM2-t12-35M mean sequence embeddings plus Morgan count fingerprints and descriptors, arranged so the order encodes the question, with each comparison entered in both orders. Evaluation uses compound-disjoint splits from ChEMBL 37 pairs.

  • ChEMBL 37 human single-protein measurements (Ki, IC50, Kd, EC50, Kb; exact readings; PubChem BioAssay rows excluded): 1,263,626 measurements, 2,705 targets, 794,638 compounds, 34 level-2 families · human · n = 1,263,626 measurements
  • Target comparisons formed from the pool: 89,888 across families and 333,893 within a family; held-out sets of 8,689 and 32,738 · human · n = 423,781 comparisons
  • Compound comparisons formed from the pool (two compounds at one target, same endpoint); held-out set of 65,725 after discarding 626,130 straddling comparisons · human · n = 2,519,433 comparisons
Methods & tools
ChEMBL 37, ESM2-t12-35M, RDKit, scikit-learn, UniProt, Morgan count fingerprints, random forest
Software & environment
Pinned: Python 3.10.15, scikit-learn 1.7.2, numpy 2.2.6, RDKit 2025.09.5, torch 2.9.1, transformers 4.57.3. install.sh admits an install only if 30 reference predictions reproduce within 1e-6.
Assay conditions & compute
CPU only, no GPU. Fitting took 239 s (target comparator) and 1,257 s (compound comparator). Resident memory about 3.4 GB and 5.4 GB; the compound comparator reaches 7.8 GB while unpickling.
Reproducibility gaps
Forests load only with the released scikit-learn version. Code is Apache 2.0 and weights CC BY-SA 4.0, as stated in the text.
Limitations (6)
  • Few compounds have been measured against targets from more than one family (4.6%), which caps the across-family target comparison; extra within-family data cannot lift that ceiling. · stated by the authors
  • Near-ties are hard (0.57 to 0.64 accuracy within half a log unit), and decisions that depend on separating them require measurement. Raising the strength threshold cuts coverage; at 0.80 the compound comparator answers a coverage of 0.06. · stated by the authors
  • Results hold only for represented chemistry, families and targets. Withholding compounds is weaker than withholding whole targets, and targets outside the roster are refused. Models give relative preference, not potency. · stated by the authors
  • Nothing measured here links cross-family breadth to toxicity; breadth in a public database partly reflects how many assay panels a compound passed through. · stated by the authors
  • Accuracy is the only metric reported, and no head-to-head baseline (such as subtracting two separate affinity predictions) is shown; sibling-model comparisons use splits the paper says are not comparable. · noted in this summary
  • Single-author evaluation by the founder and CEO of the company that funded and develops the models; holdout design and benchmarking were done by the model developer. · noted in this summary

Read this if you prioritize counter-screens or compounds to make and want a structure-free, locally runnable ranker with accuracy reported by family, prediction strength and potency gap.

Citations
0

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISPeriwal '20Muskal '26Singh '22Williams '24Chang-Gonza '26Karelina '23
61% semantic matchBioinformatics
Vinita Periwal, Stefan Oliver Bassler, Sergej Andrejev+3 more·bioRxiv·Nov 6, 2020· 1 citations