Contested result

REVEL and BayesDel: An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above REVEL in 6 of 6 comparisons; other studies rank REVEL above BayesDel in 11 of 13.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

3 papers rank REVEL above BayesDel in 11 of their 14 comparisons and BayesDel higher in 2; 1 is a tie. Of those papers, ProteinGym AMFR assay-level Spearman… ranks BayesDel higher on its own (1 of 1). An openly licensed benchmark… ranks BayesDel above REVEL in all 6 of its comparisons.

Rank REVEL higher (2): Livesey and Marsh 2025, Clinical likelihood ratios and balanced…
Rank BayesDel higher (2): An openly licensed benchmark… (disagrees), ProteinGym AMFR assay-level Spearman…
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

REVEL higherTieBayesDel higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (4)
Paper	Role	Ranks REVEL higher	Ranks BayesDel higher	Ties
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers and 1 more record · DOI	Disagrees	0	6	0
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOI	Agrees	7	0	0
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and 1 more record · DOI	Agrees	4	1	1
ProteinGym AMFR assay-level Spearman comparison and 4 more records · Repository	Agrees	0	1	0

Comparisons in An openly licensed benchmark… (6)

Protocol	Dataset	Metric	REVEL	BayesDel	Higher
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	AUROC (CGC missense, all genes), unitless	0.816	0.864	BayesDel
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	Average precision (CGC missense, all genes), unitless	0.652	0.712	BayesDel
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	AUROC (CGC missense, oncogenes (gain of function)), unitless	0.843	0.878	BayesDel
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	Average precision (CGC missense, oncogenes (gain of function)), unitless	0.739	0.764	BayesDel
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	AUROC (CGC missense, tumour suppressors (loss of function)), unitless	0.859	0.892	BayesDel
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	Average precision (CGC missense, tumour suppressors (loss of function)), unitless	0.618	0.684	BayesDel

Comparisons in Livesey and Marsh 2025 (7)

Protocol	Dataset	Metric	REVEL	BayesDel	Higher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent	76	71.8	REVEL
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	66.1	57.1	REVEL
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	13.2	6	REVEL
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only), percent	65.6	56.8	REVEL
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only), percent	66.3	57.4	REVEL
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only), percent	66.1	57.1	REVEL
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent	67.5	58.2	REVEL

Comparisons in Clinical likelihood ratios and balanced… (6)

Protocol	Dataset	Metric	REVEL	BayesDel	Higher
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editing	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set), unitless	22.4	17.2	REVEL
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editing	False negatives (Functionally deleterious BRCA1 variants called tolerated), count	2	2	Tie
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editing	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set), unitless	4.54	1.26	REVEL
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set), unitless	22.3	16.1	REVEL
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair	False negatives (Functionally deleterious BRCA2 variants called tolerated), count	1	0	BayesDel
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set), unitless	2.88	1.51	REVEL

Comparisons in ProteinGym AMFR assay-level Spearman… (1)

Protocol	Dataset	Metric	REVEL	BayesDel	Higher
ProteinGym clinical substitutions, supervised predictors, average AUC (commit 144fe22)	ProteinGym clinical substitution benchmark at commit 144fe22	AUROC (ProteinGym clinical substitutions, supervised clinical predictors, Average_AUC over proteins), unitless	0.928	0.972	BayesDel

What disagrees

Other studies: rank REVEL above BayesDel in 11 of 13 comparisons
This study: ranks BayesDel above REVEL in 6 of 6 comparisons

Compared: REVEL, BayesDel
Study that disagrees: An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers, OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)
Other studies: 9 studies
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes, Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx), ProteinGym AMFR assay-level Spearman comparison, Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8), ProteinGym clinical supervised substitutions AUC summary at commit 144fe22, ProteinGym clinical zero-shot indels AUC summary at commit 144fe22, ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22, ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (12)
Study	Tool	Protocol or benchmark	Metric	Value
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	BayesDel_addAF rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, all genes)	0.712
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	BayesDel_addAF rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, all genes)	0.864
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	BayesDel_addAF rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, oncogenes (gain of function))	0.764
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	BayesDel_addAF rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, oncogenes (gain of function))	0.878
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	BayesDel_addAF rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, tumour suppressors (loss of function))	0.684
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	BayesDel_addAF rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, tumour suppressors (loss of function))	0.892
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	REVEL rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, all genes)	0.652
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	REVEL rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, all genes)	0.816
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	REVEL rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, oncogenes (gain of function))	0.739
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	REVEL rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, oncogenes (gain of function))	0.843
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	REVEL rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, tumour suppressors (loss of function))	0.618
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	REVEL rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, tumour suppressors (loss of function))	0.859

In the other studies (29)
Study	Tool	Protocol or benchmark	Metric	Value
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_MaxAF_genespecific (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)	17.2
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_MaxAF_genespecific (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	0
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_MaxAF_universal (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	0
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_MaxAF_universal (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)	1.51
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_NoMaxAF_genespecific (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	0
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_NoMaxAF_universal (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)	1.26
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	BayesDEL_NoMaxAF_universal (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set)	16.1
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	Revel_a (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	1
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	Revel_b (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)	4.54
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	Revel_b (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)	2.88
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	Revel_c (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)	22.4
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	Revel_c (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set)	22.3
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	Revel_c (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel_noAF (Livesey and Marsh 2025)	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	71.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	76
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel_noAF (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	57.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel_noAF (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)	6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	66.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)	13.2
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	56.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	65.6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel_noAF (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	57.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	66.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel_noAF (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	57.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	BayesDel_noAF (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	58.2
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	67.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	REVEL (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	66.1
ProteinGym clinical supervised substitutions AUC summary at commit 144fe22	BayesDel (addAF) [ProteinGym 144fe22 clinical summary]	ProteinGym clinical substitutions, supervised predictors, average AUC (commit 144fe22)	AUROC (ProteinGym clinical substitutions, supervised clinical predictors, Average_AUC over proteins)	0.972
ProteinGym clinical supervised substitutions AUC summary at commit 144fe22	REVEL [ProteinGym 144fe22 clinical summary]	ProteinGym clinical substitutions, supervised predictors, average AUC (commit 144fe22)	AUROC (ProteinGym clinical substitutions, supervised clinical predictors, Average_AUC over proteins)	0.928

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-66f24d743b1c30aa · All contested results · How findings are made and followed up