Negative sampling shapes benchmark discrimination and experimental retrieval in synthetic-lethality prediction

Abstract

Benchmarks for synthetic-lethality prediction compare known interactions with sampled negatives. Does performance under degree matching predict which gene pairs a model can prioritize? Across five gene-disjoint folds, matching test negatives by degree in the full labelled graph brought four methods near chance. SLMGAE states selected by benchmark test F1 had a mean area under the receiver operating characteristic curve of 0.501 on matched negatives. The same states retrieved 21 experimental hits among 198 SPIDR selections, versus 8.00 expected by chance. Comparable enrichment was not established in three other screens. Changing training negatives increased HAP1 retrieval from 3.6 to 15.8 hits for logistic regression; swapping 28 endpoint coefficients transferred most of this difference. Rewiring positive pairs while keeping the original matched negatives reduced HAP1 retrieval for logistic regression and gradient boosting. This preserved individual gene degrees but also changed their joint distribution, limiting attribution to partner identity. A density-ratio estimate closed 55-65% of the logistic-regression shortlist gap across four previously evaluated pools, with limited transfer to nonlinear models. Frozen predictions met the shortlist criterion for one of two new graph relations. Benchmark discrimination and experimental retrieval therefore warrant separate evaluation at the intended screening budget.