rewirebio.iobenchmarks
Contested result

LRT and MetaLR: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Different tasks across studies

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaLR in 15 of 17 comparisons; other studies rank MetaLR above LRT in 12 of 13.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank MetaLR above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaLR in 15 of its 20 comparisons and MetaLR higher in 2; 3 are ties.

Papers grouped by which system they rank higher: MetaLR or LRTRank MetaLR higher: Livesey and Marsh 2025, Clinical likelihood ratios and balanced…. Rank LRT higher: Comprehensive assessment… (disagrees). Mixed: none.Rank MetaLR higherMixedRank LRT higherLivesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)Clinical likelihoodratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genesComprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations
  • Rank MetaLR higher (2): Livesey and Marsh 2025, Clinical likelihood ratios and balanced…
  • Rank LRT higher (1): Comprehensive assessment… (disagrees)
  • Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores MetaLR or LRT higherComprehensive assessment… (disagrees): MetaLR higher in 2, LRT higher in 15, 3 ties. Livesey and Marsh 2025: MetaLR higher in 7, LRT higher in 0, 0 ties. Clinical likelihood ratios and balanced…: MetaLR higher in 5, LRT higher in 1, 0 ties.Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations2 MetaLR higher23 tie315 LRT higher15Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MetaLR higher7Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes5 MetaLR higher51 LRT higher1
MetaLR higherTieLRT higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Papers compared (3)
PaperRoleRanks MetaLR higherRanks LRT higherTies
Comprehensive assessment of computational algorithms in predicting cancer driver mutations and 4 more records · DOIDisagrees2153
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOIAgrees700
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and 1 more record · DOIAgrees510
Comparisons in Comprehensive assessment… (20)
ProtocolDatasetMetricMetaLRLRTHigher
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Accuracy (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.570.66LRT
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Negative predictive value (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.590.86LRT
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Precision (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.560.6LRT
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Recall (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.670.93LRT
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Specificity (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.470.38MetaLR
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.570.65LRT
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.570.67LRT
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Precision (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.570.63LRT
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Recall (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.570.73LRT
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Specificity (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.570.57Tie
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Accuracy (cell viability drivers versus neutral, default categorical calls), fraction0.560.58LRT
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Negative predictive value (cell viability drivers versus neutral, default categorical calls), fraction0.550.71LRT
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Precision (cell viability drivers versus neutral, default categorical calls), fraction0.560.56Tie
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Recall (cell viability drivers versus neutral, default categorical calls), fraction0.530.93LRT
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Specificity (cell viability drivers versus neutral, default categorical calls), fraction0.590.2MetaLR
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Accuracy (cell viability drivers versus neutral, median-score threshold), fraction0.560.65LRT
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Negative predictive value (cell viability drivers versus neutral, median-score threshold), fraction0.560.68LRT
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Precision (cell viability drivers versus neutral, median-score threshold), fraction0.560.63LRT
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Recall (cell viability drivers versus neutral, median-score threshold), fraction0.560.74LRT
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Specificity (cell viability drivers versus neutral, median-score threshold), fraction0.560.56Tie
Comparisons in Livesey and Marsh 2025 (7)
ProtocolDatasetMetricMetaLRLRTHigher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missenseLivesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent49.118.8MetaLR
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets), percent45.713.6MetaLR
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets), percent8.131.22MetaLR
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only), percent42.115.4MetaLR
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only), percent48.312.3MetaLR
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only), percent45.713.6MetaLR
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent46.614.3MetaLR
Comparisons in Clinical likelihood ratios and balanced… (6)
ProtocolDatasetMetricMetaLRLRTHigher
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingBenignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set), unitless6.052.99MetaLR
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingFalse negatives (Functionally deleterious BRCA1 variants called tolerated), count1885MetaLR
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingLikelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set), unitless1.342.46LRT
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairBenignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set), unitless21.89.62MetaLR
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairFalse negatives (Functionally deleterious BRCA2 variants called tolerated), count12MetaLR
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairLikelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set), unitless1.491.38MetaLR

What disagrees

Other studies
rank MetaLR above LRT in 12 of 13 comparisons
This study
ranks LRT above MetaLR in 15 of 17 comparisons
Compared
LRT, MetaLR
Study that disagrees
Comprehensive assessment of computational algorithms in predicting cancer driver mutations, Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB), Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability), Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability), Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)
Other studies
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes, Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx), Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8)
Rule
Ordering reversal; different tasks across studies
Model identity
Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (34)
StudyToolProtocol or benchmarkMetricValue
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Accuracy (OncoKB benchmark (positive set not stated), default categorical calls)0.66
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Negative predictive value (OncoKB benchmark (positive set not stated), default categorical calls)0.86
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Precision (OncoKB benchmark (positive set not stated), default categorical calls)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Recall (OncoKB benchmark (positive set not stated), default categorical calls)0.93
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Specificity (OncoKB benchmark (positive set not stated), default categorical calls)0.38
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold)0.67
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Precision (OncoKB benchmark (positive set not stated), median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)LRT (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Recall (OncoKB benchmark (positive set not stated), median-score threshold)0.73
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Accuracy (cell viability drivers versus neutral, default categorical calls)0.58
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Negative predictive value (cell viability drivers versus neutral, default categorical calls)0.71
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Recall (cell viability drivers versus neutral, default categorical calls)0.93
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Specificity (cell viability drivers versus neutral, default categorical calls)0.2
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Accuracy (cell viability drivers versus neutral, median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Negative predictive value (cell viability drivers versus neutral, median-score threshold)0.68
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Precision (cell viability drivers versus neutral, median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)LRT (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Recall (cell viability drivers versus neutral, median-score threshold)0.74
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Accuracy (OncoKB benchmark (positive set not stated), default categorical calls)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Negative predictive value (OncoKB benchmark (positive set not stated), default categorical calls)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Precision (OncoKB benchmark (positive set not stated), default categorical calls)0.56
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Recall (OncoKB benchmark (positive set not stated), default categorical calls)0.67
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Specificity (OncoKB benchmark (positive set not stated), default categorical calls)0.47
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Precision (OncoKB benchmark (positive set not stated), median-score threshold)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MetaLR (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Recall (OncoKB benchmark (positive set not stated), median-score threshold)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Accuracy (cell viability drivers versus neutral, default categorical calls)0.56
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Negative predictive value (cell viability drivers versus neutral, default categorical calls)0.55
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Recall (cell viability drivers versus neutral, default categorical calls)0.53
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Specificity (cell viability drivers versus neutral, default categorical calls)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Accuracy (cell viability drivers versus neutral, median-score threshold)0.56
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Negative predictive value (cell viability drivers versus neutral, median-score threshold)0.56
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Precision (cell viability drivers versus neutral, median-score threshold)0.56
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MetaLR (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Recall (cell viability drivers versus neutral, median-score threshold)0.56
In the other studies (26)
StudyToolProtocol or benchmarkMetricValue
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)LRT (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)2.99
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)LRT (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)False negatives (Functionally deleterious BRCA1 variants called tolerated)85
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)LRT (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)2.46
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)LRT (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set)9.62
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)LRT (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)False negatives (Functionally deleterious BRCA2 variants called tolerated)2
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)LRT (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)1.38
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MetaLR (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)6.05
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MetaLR (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)False negatives (Functionally deleterious BRCA1 variants called tolerated)18
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MetaLR (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)1.34
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MetaLR (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set)21.8
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MetaLR (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)False negatives (Functionally deleterious BRCA2 variants called tolerated)1
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MetaLR (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)1.49
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missensePairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)18.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missensePairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)49.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets)13.6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets)1.22
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets)45.7
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets)8.13
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only)15.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only)42.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only)12.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only)48.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)14.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)46.6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)LRT (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only)13.6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MetaLR (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only)45.7

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-0156f7219727ea03 · All contested results · How findings are made and followed up