rewirebio.iobenchmarks
Contested result

Condel and PolyPhen-2: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • two-study-conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PolyPhen-2 above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above PolyPhen-2 in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank Condel above PolyPhen-2 in 7 of their 12 comparisons and PolyPhen-2 higher in 4; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks PolyPhen-2 higher on its own (4 of 6).

Papers grouped by which system they rank higher: Condel or PolyPhen-2Rank Condel higher: Livesey and Marsh 2025. Rank PolyPhen-2 higher: Clinical likelihood ratios and balanced…. Mixed: none.Rank Condel higherMixedRank PolyPhen-2 higherLivesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)Clinical likelihoodratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes
  • Rank Condel higher (1): Livesey and Marsh 2025
  • Rank PolyPhen-2 higher (1): Clinical likelihood ratios and balanced…
  • Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores Condel or PolyPhen-2 higherClinical likelihood ratios and balanced…: Condel higher in 1, PolyPhen-2 higher in 4, 1 tie. Livesey and Marsh 2025: Condel higher in 6, PolyPhen-2 higher in 0, 0 ties.Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 Condel higher11 tie14 PolyPhen-2 higher4Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)6 Condel higher6
Condel higherTiePolyPhen-2 higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Papers compared (2)
PaperRoleRanks Condel higherRanks PolyPhen-2 higherTies
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and 1 more record · DOIAgrees141
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOIAgrees600
Comparisons in Clinical likelihood ratios and balanced… (6)
ProtocolDatasetMetricCondelPolyPhen-2Higher
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingBenignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set), unitless7.427.6PolyPhen-2
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingFalse negatives (Functionally deleterious BRCA1 variants called tolerated), count92PolyPhen-2
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingLikelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set), unitless1.181.16Condel
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairBenignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set), unitless4.167.28PolyPhen-2
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairFalse negatives (Functionally deleterious BRCA2 variants called tolerated), count11Tie
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairLikelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set), unitless1.051.11PolyPhen-2
Comparisons in Livesey and Marsh 2025 (6)
ProtocolDatasetMetricCondelPolyPhen-2Higher
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets), percent47.542.4Condel
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets), percent9.766.76Condel
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only), percent45.241.5Condel
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only), percent4943.1Condel
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only), percent47.542.4Condel
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent47.844.1Condel

What disagrees

studies
2
Compared
Condel, PolyPhen-2
Study that disagrees
Other studies
Rule
two-study-conflict; unknown
Model identity
Matched across studies partly through links not yet reviewed

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-7ba0b807f99da183 · All contested results · How findings are made and followed up