Protocol

Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays only

Rank score recalculated using only the DMS assays the authors class as direct (Table S5 rank_score_direct).

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

97 recorded evaluations, 97 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

97 evaluations · 97 results. Different protocols are not a single leaderboard.

Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays only · Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)

Sorted by Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)
CPT-1 (Livesey and Marsh 2025)Evaluation	93.5% (best in column)
AlphaMissense (Livesey and Marsh 2025)Evaluation	92.1%
ESCOTT (Livesey and Marsh 2025)Evaluation	91.8%
iGEMME (Livesey and Marsh 2025)Evaluation	89.6%
popEVE (Livesey and Marsh 2025)Evaluation	89.2%
GEMME (Livesey and Marsh 2025)Evaluation	88.7%
TranceptEVE (Livesey and Marsh 2025)Evaluation	88.5%
SaProt (Livesey and Marsh 2025)Evaluation	84.8%
EVE (Livesey and Marsh 2025)Evaluation	83%
SNPred (Livesey and Marsh 2025)Evaluation	79.1%
ESM-1v (Livesey and Marsh 2025)Evaluation	78.4%
MSA_Transformer (Livesey and Marsh 2025)Evaluation	77.7%
VARITY_R (Livesey and Marsh 2025)Evaluation	77.5%
VARITY_ER (Livesey and Marsh 2025)Evaluation	77.4%
MetaRNN (Livesey and Marsh 2025)Evaluation	74.8%
MutFormer (Livesey and Marsh 2025)Evaluation	74.7%
EVmutation_epistatic (Livesey and Marsh 2025)Evaluation	74.3%
ESM2 (Livesey and Marsh 2025)Evaluation	74.2%
mvPPT (Livesey and Marsh 2025)Evaluation	73.9%
DeepSequence (Livesey and Marsh 2025)Evaluation	73.5%
CARP (Livesey and Marsh 2025)Evaluation	73%
mutationTCN (Livesey and Marsh 2025)Evaluation	70.7%
VESPA (Livesey and Marsh 2025)Evaluation	69.5%
UNEECON (Livesey and Marsh 2025)Evaluation	68.1%
EVmutation_independent (Livesey and Marsh 2025)Evaluation	67.1%
ClinPred (Livesey and Marsh 2025)Evaluation	66.9%
PHACTboost (Livesey and Marsh 2025)Evaluation	66.8%
DEOGEN2 (Livesey and Marsh 2025)Evaluation	66.1%
REVEL (Livesey and Marsh 2025)Evaluation	65.6%
ESM-1b (Livesey and Marsh 2025)Evaluation	62.4%
gMVP (Livesey and Marsh 2025)Evaluation	61.7%
MutScore (Livesey and Marsh 2025)Evaluation	61.5%
ProGen2 (Livesey and Marsh 2025)Evaluation	61%
SPRI (Livesey and Marsh 2025)Evaluation	60.2%
VEST4 (Livesey and Marsh 2025)Evaluation	59.9%
SuSPect (Livesey and Marsh 2025)Evaluation	59.5%
MutationAssessor (Livesey and Marsh 2025)Evaluation	59.3%
MISTIC (Livesey and Marsh 2025)Evaluation	57.3%
MPC (Livesey and Marsh 2025)Evaluation	57.2%
BayesDel (Livesey and Marsh 2025)Evaluation	56.8%
BayesDel_noAF (Livesey and Marsh 2025)Evaluation	56.6%
SNAP2 (Livesey and Marsh 2025)Evaluation	56.5%
SNPs&GO (Livesey and Marsh 2025)Evaluation	55%
DeepSAV (Livesey and Marsh 2025)Evaluation	54.7%
MOIpred_benign (Livesey and Marsh 2025)Evaluation	54.6%
Eigen (Livesey and Marsh 2025)Evaluation	53.7%
PhD_SNP (Livesey and Marsh 2025)Evaluation	52.2%
Tranception (Livesey and Marsh 2025)Evaluation	51.9%
PHACT (Livesey and Marsh 2025)Evaluation	51.6%
PROVEAN (Livesey and Marsh 2025)Evaluation	51%
SIFT4G (Livesey and Marsh 2025)Evaluation	49.9%
Envision (Livesey and Marsh 2025)Evaluation	49.2%
VESPAl (Livesey and Marsh 2025)Evaluation	48.8%
RITA (Livesey and Marsh 2025)Evaluation	48.2%
DeMaSk (Livesey and Marsh 2025)Evaluation	47.9%
SIGMA (Livesey and Marsh 2025)Evaluation	47.7%
Rhapsody (Livesey and Marsh 2025)Evaluation	47.2%
MutPred (Livesey and Marsh 2025)Evaluation	46.5%
PolyPhen2_HumVar (Livesey and Marsh 2025)Evaluation	46.2%
LASSIE (Livesey and Marsh 2025)Evaluation	45.2%
CONDEL (Livesey and Marsh 2025)Evaluation	45.2%
SNPs&GO3D (Livesey and Marsh 2025)Evaluation	44.8%
PANTHER (Livesey and Marsh 2025)Evaluation	44.1%
Wavenet (Livesey and Marsh 2025)Evaluation	43.3%
MetaLR (Livesey and Marsh 2025)Evaluation	42.1%
PolyPhen2_HumDiv (Livesey and Marsh 2025)Evaluation	41.5%
Maverick_benign (Livesey and Marsh 2025)Evaluation	41.1%
SIFT (Livesey and Marsh 2025)Evaluation	39.9%
MetaSVM (Livesey and Marsh 2025)Evaluation	39.4%
CADD_1.7 (Livesey and Marsh 2025)Evaluation	39%
M-CAP (Livesey and Marsh 2025)Evaluation	38.3%
Unirep_evotune (Livesey and Marsh 2025)Evaluation	37.7%
CADD_1.6 (Livesey and Marsh 2025)Evaluation	34.4%
AlphScore (Livesey and Marsh 2025)Evaluation	32.3%
MVP (Livesey and Marsh 2025)Evaluation	31.3%
PonP2 (Livesey and Marsh 2025)Evaluation	30%
CAPICE (Livesey and Marsh 2025)Evaluation	29.1%
FathmmXF (Livesey and Marsh 2025)Evaluation	27.1%
InMeRF (Livesey and Marsh 2025)Evaluation	26%
sequence_unet (Livesey and Marsh 2025)Evaluation	20.3%
PhD-SNPg (Livesey and Marsh 2025)Evaluation	20.1%
phyloP (Livesey and Marsh 2025)Evaluation	17.7%
PrimateAI (Livesey and Marsh 2025)Evaluation	17.5%
LIST-S2 (Livesey and Marsh 2025)Evaluation	16.7%
fathmm-MKL (Livesey and Marsh 2025)Evaluation	15.8%
LRT (Livesey and Marsh 2025)Evaluation	15.4%
DANN (Livesey and Marsh 2025)Evaluation	14.5%
BLOSUM62 (Livesey and Marsh 2025)Evaluation	13.7%
FATHMM (Livesey and Marsh 2025)Evaluation	12.7%
phastCons (Livesey and Marsh 2025)Evaluation	9.13%
GERP++ (Livesey and Marsh 2025)Evaluation	9.13%
GenoCanyon (Livesey and Marsh 2025)Evaluation	8.65%
ProtGPT2 (Livesey and Marsh 2025)Evaluation	8.46%
SiPhy (Livesey and Marsh 2025)Evaluation	8.13%
Grantham (Livesey and Marsh 2025)Evaluation	7.32%
3Cnet (Livesey and Marsh 2025)Evaluation	3.51%
fitCons (Livesey and Marsh 2025)Evaluation	2.48%

All 97 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: 3Cnet (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	3.51% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source 3Cnet: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C93 (row '3Cnet', column 'rank_score_direct')
Configuration: AlphaMissense (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	92.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphaMissense: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C3 (row 'AlphaMissense', column 'rank_score_direct')
Configuration: AlphScore (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	32.3% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphScore: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C76 (row 'AlphScore', column 'rank_score_direct')
Configuration: BayesDel_noAF (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	56.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel_noAF: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C39 (row 'BayesDel_noAF', column 'rank_score_direct')
Configuration: BayesDel (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	56.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C40 (row 'BayesDel', column 'rank_score_direct')
Configuration: BLOSUM62 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	13.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BLOSUM62: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C87 (row 'BLOSUM62', column 'rank_score_direct')
Configuration: CADD_1.6 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	34.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.6: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C75 (row 'CADD_1.6', column 'rank_score_direct')
Configuration: CADD_1.7 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	39% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.7: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C68 (row 'CADD_1.7', column 'rank_score_direct')
Configuration: CAPICE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	29.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CAPICE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C77 (row 'CAPICE', column 'rank_score_direct')
Configuration: CARP (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	73% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CARP: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C15 (row 'CARP', column 'rank_score_direct')
Configuration: ClinPred (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	66.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ClinPred: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C32 (row 'ClinPred', column 'rank_score_direct')
Configuration: CONDEL (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	45.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CONDEL: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C56 (row 'CONDEL', column 'rank_score_direct')
Configuration: CPT-1 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	93.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CPT-1: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C2 (row 'CPT-1', column 'rank_score_direct')
Configuration: DANN (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	14.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DANN: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C90 (row 'DANN', column 'rank_score_direct')
Configuration: DeepSAV (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	54.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSAV: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C49 (row 'DeepSAV', column 'rank_score_direct')
Configuration: DeepSequence (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	73.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSequence: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C13 (row 'DeepSequence', column 'rank_score_direct')
Configuration: DeMaSk (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	47.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeMaSk: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C47 (row 'DeMaSk', column 'rank_score_direct')
Configuration: DEOGEN2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	66.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DEOGEN2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C43 (row 'DEOGEN2', column 'rank_score_direct')
Configuration: Eigen (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	53.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Eigen: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C51 (row 'Eigen', column 'rank_score_direct')
Configuration: Envision (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	49.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Envision: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C66 (row 'Envision', column 'rank_score_direct')
Configuration: ESCOTT (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	91.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESCOTT: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C6 (row 'ESCOTT', column 'rank_score_direct')
Configuration: ESM-1b (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	62.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1b: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C26 (row 'ESM-1b', column 'rank_score_direct')
Configuration: ESM-1v (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	78.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1v: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C8 (row 'ESM-1v', column 'rank_score_direct')
Configuration: ESM2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	74.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C23 (row 'ESM2', column 'rank_score_direct')
Configuration: EVE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	83% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C21 (row 'EVE', column 'rank_score_direct')

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 2 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

No execution recipe linked to this protocol. Recipe availability does not establish a completed evaluation.

External evaluations: 97

Literature evidence is not a Rewire measurement. Executed but unpublished runs and private review status are not included.

Null control

Proposed control: requires review

Select a task-valid null control after reviewing inputs and metric

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Conventional reference

Proposed control: requires review

Select an upstream conventional reference after reviewing the full protocol

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)

Coverage is derived from release 2026-10-22-1b7cf027e4f6. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

2 source records and release history

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Original source · PMC12016141 (published 2025-04-22; Genome Biol 26:104)
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8) · Original source · 13059_2025_3575_MOESM2_ESM.xlsx as published with PMC12016141

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: vep-lit-20261014-protocol-livesey2025-dms-direct

areas: dna-genomes; proteins-complexes
contexts: research
scope note: Same pairwise procedure as the main DMS ranking, restricted to direct assays: those that directly measure the ability of the target protein to carry out one or more functions (for example one- and two-hybrid assays and VAMP-seq).
task: Missense variant effect prediction ranked against DMS
metric: pairwise-win-rate
metric direction: higher
unit: percent
metric qualifier: average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only
metric implementation: scipy.stats.spearmanr (scipy 1.5.4, Python 3.6.8); absolute Spearman; the VEP with the higher |rho| on variants scored by the DMS and both VEPs earns the point
aggregation: Win rate of each VEP pair across proteins (wins divided by times the pair was tested), averaged over all other VEPs (Methods, 'Spearman's correlation and rank score')
version: Livesey and Marsh 2025, Supplementary Material 2 Table S5
publication status: peer_reviewed
source locator: Methods 'Spearman's correlation and rank score' and 'AUROC and AUPRC'; Supplementary Material 2 Legends (Table S5)
limitations: A rank score is relative to the other VEPs in the same table; values from different tables or editions are not comparable.; Per-pair, per-protein correlations and AUROCs behind the win rates are not printed; they are in the figshare deposit, which was not retrieved.; DMS assays measure a cellular or molecular phenotype chosen per protein, not pathogenicity.; Figure 2 error bars (standard deviation over 1000 bootstraps of the DMS sets) are not printed; Table S4 gives pairwise bootstrap counts, not extracted.; Five VEPs (Envision, CPT-1, DeMaSk, VARITY_R, VARITY_ER) have DMS sets they were trained on excluded, so they are compared on fewer proteins.
review: reviewer: claude; reviewer note: Claude review agent; no human review claimed. Independent Claude (Opus 5.5) review agent, separate from the extraction agent that built this batch. Automated review only. Transcription, not reproduction.; reviewed at: 2026-10-14T15:30:00Z; artifact sha256: 1735ee461ef192d9de67c914e9ad73d72d8aa255ea0e3479eb68cd8631d99fa2; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12016141/fullTextXML; method: ai-assisted-source-review; scope-review; method note: Descriptive fields, identity links and missing-metadata reasons checked against the cited sources.; note: Checked; see the dated review for rulings on identity and method type.

Related records

Suggest a correction

Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: 3Cnet (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	3.51% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source 3Cnet: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C93 (row '3Cnet', column 'rank_score_direct')
Configuration: AlphaMissense (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	92.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphaMissense: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C3 (row 'AlphaMissense', column 'rank_score_direct')
Configuration: AlphScore (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	32.3% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphScore: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C76 (row 'AlphScore', column 'rank_score_direct')
Configuration: BayesDel_noAF (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	56.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel_noAF: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C39 (row 'BayesDel_noAF', column 'rank_score_direct')
Configuration: BayesDel (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	56.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C40 (row 'BayesDel', column 'rank_score_direct')
Configuration: BLOSUM62 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	13.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BLOSUM62: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C87 (row 'BLOSUM62', column 'rank_score_direct')
Configuration: CADD_1.6 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	34.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.6: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C75 (row 'CADD_1.6', column 'rank_score_direct')
Configuration: CADD_1.7 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	39% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.7: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C68 (row 'CADD_1.7', column 'rank_score_direct')
Configuration: CAPICE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	29.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CAPICE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C77 (row 'CAPICE', column 'rank_score_direct')
Configuration: CARP (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	73% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CARP: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C15 (row 'CARP', column 'rank_score_direct')
Configuration: ClinPred (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	66.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ClinPred: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C32 (row 'ClinPred', column 'rank_score_direct')
Configuration: CONDEL (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	45.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CONDEL: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C56 (row 'CONDEL', column 'rank_score_direct')
Configuration: CPT-1 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	93.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CPT-1: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C2 (row 'CPT-1', column 'rank_score_direct')
Configuration: DANN (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	14.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DANN: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C90 (row 'DANN', column 'rank_score_direct')
Configuration: DeepSAV (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	54.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSAV: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C49 (row 'DeepSAV', column 'rank_score_direct')
Configuration: DeepSequence (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	73.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSequence: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C13 (row 'DeepSequence', column 'rank_score_direct')
Configuration: DeMaSk (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	47.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeMaSk: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C47 (row 'DeMaSk', column 'rank_score_direct')
Configuration: DEOGEN2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	66.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DEOGEN2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C43 (row 'DEOGEN2', column 'rank_score_direct')
Configuration: Eigen (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	53.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Eigen: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C51 (row 'Eigen', column 'rank_score_direct')
Configuration: Envision (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	49.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Envision: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C66 (row 'Envision', column 'rank_score_direct')
Configuration: ESCOTT (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	91.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESCOTT: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C6 (row 'ESCOTT', column 'rank_score_direct')
Configuration: ESM-1b (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	62.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1b: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C26 (row 'ESM-1b', column 'rank_score_direct')
Configuration: ESM-1v (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	78.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1v: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C8 (row 'ESM-1v', column 'rank_score_direct')
Configuration: ESM2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	74.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C23 (row 'ESM2', column 'rank_score_direct')
Configuration: EVE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	83% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell C21 (row 'EVE', column 'rank_score_direct')