rewirebio.iobenchmarks
Contested result

MVP and VEST4: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above VEST4 in 3 of 4 comparisons; 2 of 2 other studies rank VEST4 above MVP.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

3 papers rank VEST4 above MVP in 17 of their 23 comparisons and MVP higher in 6. Clinical likelihood ratios and balanced… ranks MVP above VEST4 in 3 of its 5 comparisons and VEST4 higher in 1; 1 is a tie.

Papers grouped by which system they rank higher: VEST4 or MVPRank VEST4 higher: Livesey and Marsh 2025, An openly licensed benchmark…. Rank MVP higher: Clinical likelihood ratios and balanced… (disagrees). Mixed: Comprehensive assessment….Rank VEST4 higherMixedRank MVP higherLivesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)An openly licensedbenchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer driversComprehensiveassessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutationsClinical likelihoodratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes
  • Rank VEST4 higher (2): Livesey and Marsh 2025, An openly licensed benchmark…
  • Rank MVP higher (1): Clinical likelihood ratios and balanced… (disagrees)
  • Mixed (1): Comprehensive assessment…

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores VEST4 or MVP higherClinical likelihood ratios and balanced… (disagrees): VEST4 higher in 1, MVP higher in 3, 1 tie. Comprehensive assessment…: VEST4 higher in 5, MVP higher in 5, 0 ties. Livesey and Marsh 2025: VEST4 higher in 6, MVP higher in 1, 0 ties. An openly licensed benchmark…: VEST4 higher in 6, MVP higher in 0, 0 ties.Disagrees: Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 VEST4 higher11 tie13 MVP higher3Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations5 VEST4 higher55 MVP higher5Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)6 VEST4 higher61 MVP higher1An openly licensed benchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers6 VEST4 higher6
VEST4 higherTieMVP higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in Clinical likelihood ratios and balanced… (5)
ProtocolDatasetMetricVEST4MVPHigher
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingBenignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set), unitless55.268MVP
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingFalse negatives (Functionally deleterious BRCA1 variants called tolerated), count11Tie
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editingLikelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set), unitless1.171.22MVP
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairFalse negatives (Functionally deleterious BRCA2 variants called tolerated), count10MVP
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repairLikelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set), unitless1.131VEST4
Comparisons in Comprehensive assessment… (10)
ProtocolDatasetMetricVEST4MVPHigher
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.690.65VEST4
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.690.65VEST4
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Precision (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.690.65VEST4
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Recall (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.690.65VEST4
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Specificity (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.690.65VEST4
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Accuracy (cell viability drivers versus neutral, median-score threshold), fraction0.620.64MVP
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Negative predictive value (cell viability drivers versus neutral, median-score threshold), fraction0.620.64MVP
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Precision (cell viability drivers versus neutral, median-score threshold), fraction0.620.64MVP
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Recall (cell viability drivers versus neutral, median-score threshold), fraction0.620.64MVP
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Specificity (cell viability drivers versus neutral, median-score threshold), fraction0.620.64MVP
Comparisons in Livesey and Marsh 2025 (7)
ProtocolDatasetMetricVEST4MVPHigher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missenseLivesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent77.150.5VEST4
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets), percent5936.4VEST4
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets), percent2.335.36MVP
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only), percent59.931.3VEST4
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only), percent58.439.9VEST4
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only), percent5936.4VEST4
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent60.537.8VEST4
Comparisons in An openly licensed benchmark… (6)
ProtocolDatasetMetricVEST4MVPHigher
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAUROC (CGC missense, all genes), unitless0.8330.807VEST4
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAverage precision (CGC missense, all genes), unitless0.6480.57VEST4
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAUROC (CGC missense, oncogenes (gain of function)), unitless0.840.815VEST4
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAverage precision (CGC missense, oncogenes (gain of function)), unitless0.7130.657VEST4
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAUROC (CGC missense, tumour suppressors (loss of function)), unitless0.8850.803VEST4
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAverage precision (CGC missense, tumour suppressors (loss of function)), unitless0.6260.502VEST4

What disagrees

Other studies
rank VEST4 above MVP in 12 of 13 comparisons
This study
ranks MVP above VEST4 in 3 of 4 comparisons
Compared
MVP, VEST4
Study that disagrees
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes, Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)
Other studies
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers, OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role), Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8)
Rule
Ordering reversal; unknown
Model identity
Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (8)
StudyToolProtocol or benchmarkMetricValue
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MVP (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)68
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MVP (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)1.22
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MVP (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)False negatives (Functionally deleterious BRCA2 variants called tolerated)0
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)MVP (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)1
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)VEST3_b (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)55.2
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)VEST3_b (Cubuk et al. 2021)BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)1.17
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)VEST3_b (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)False negatives (Functionally deleterious BRCA2 variants called tolerated)1
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)VEST3_b (Cubuk et al. 2021)BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)1.13
In the other studies (26)
StudyToolProtocol or benchmarkMetricValue
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MVP rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, all genes)0.57
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MVP rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, all genes)0.807
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MVP rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, oncogenes (gain of function))0.657
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MVP rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, oncogenes (gain of function))0.815
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MVP rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, tumour suppressors (loss of function))0.502
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MVP rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, tumour suppressors (loss of function))0.803
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)VEST4 rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, all genes)0.648
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)VEST4 rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, all genes)0.833
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)VEST4 rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, oncogenes (gain of function))0.713
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)VEST4 rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, oncogenes (gain of function))0.84
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)VEST4 rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, tumour suppressors (loss of function))0.626
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)VEST4 rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, tumour suppressors (loss of function))0.885
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missensePairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)50.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missensePairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)77.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets)36.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets)5.36
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets)59
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets)2.33
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only)31.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only)59.9
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only)39.9
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only)58.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)37.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)60.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MVP (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only)36.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)VEST4 (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only)59

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-2b4c7a65187a308b · All contested results · How findings are made and followed up