Protocol

Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays only

Rank score recalculated using only the DMS assays the authors class as indirect (Table S5 rank_score_indirect).

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

97 recorded evaluations, 97 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

97 evaluations · 97 results. Different protocols are not a single leaderboard.

Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays only · Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)

Sorted by Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)
CPT-1 (Livesey and Marsh 2025)Evaluation	92.1% (best in column)
AlphaMissense (Livesey and Marsh 2025)Evaluation	89.5%
VARITY_R (Livesey and Marsh 2025)Evaluation	86.4%
popEVE (Livesey and Marsh 2025)Evaluation	86.1%
ESCOTT (Livesey and Marsh 2025)Evaluation	85.5%
VARITY_ER (Livesey and Marsh 2025)Evaluation	83.7%
ESM-1v (Livesey and Marsh 2025)Evaluation	83.6%
iGEMME (Livesey and Marsh 2025)Evaluation	83.4%
SaProt (Livesey and Marsh 2025)Evaluation	82.7%
GEMME (Livesey and Marsh 2025)Evaluation	82.1%
mvPPT (Livesey and Marsh 2025)Evaluation	82.1%
DeepSequence (Livesey and Marsh 2025)Evaluation	78.4%
TranceptEVE (Livesey and Marsh 2025)Evaluation	78.2%
CARP (Livesey and Marsh 2025)Evaluation	77.7%
SNPred (Livesey and Marsh 2025)Evaluation	76.4%
gMVP (Livesey and Marsh 2025)Evaluation	75.7%
MSA_Transformer (Livesey and Marsh 2025)Evaluation	74.7%
PHACTboost (Livesey and Marsh 2025)Evaluation	73.4%
VESPA (Livesey and Marsh 2025)Evaluation	72.6%
EVE (Livesey and Marsh 2025)Evaluation	72.5%
MetaRNN (Livesey and Marsh 2025)Evaluation	71.9%
ESM2 (Livesey and Marsh 2025)Evaluation	70.8%
MutFormer (Livesey and Marsh 2025)Evaluation	68.9%
EVmutation_epistatic (Livesey and Marsh 2025)Evaluation	68.3%
ESM-1b (Livesey and Marsh 2025)Evaluation	67%
REVEL (Livesey and Marsh 2025)Evaluation	66.3%
UNEECON (Livesey and Marsh 2025)Evaluation	65.7%
mutationTCN (Livesey and Marsh 2025)Evaluation	65.3%
MutScore (Livesey and Marsh 2025)Evaluation	64.8%
EVmutation_independent (Livesey and Marsh 2025)Evaluation	62.4%
ClinPred (Livesey and Marsh 2025)Evaluation	60.4%
SNAP2 (Livesey and Marsh 2025)Evaluation	59.9%
SNPs&GO (Livesey and Marsh 2025)Evaluation	59%
VEST4 (Livesey and Marsh 2025)Evaluation	58.4%
MutationAssessor (Livesey and Marsh 2025)Evaluation	58.3%
Rhapsody (Livesey and Marsh 2025)Evaluation	57.9%
SuSPect (Livesey and Marsh 2025)Evaluation	57.5%
BayesDel_noAF (Livesey and Marsh 2025)Evaluation	57.4%
BayesDel (Livesey and Marsh 2025)Evaluation	57%
PHACT (Livesey and Marsh 2025)Evaluation	57%
VESPAl (Livesey and Marsh 2025)Evaluation	56.6%
DEOGEN2 (Livesey and Marsh 2025)Evaluation	54.6%
PhD_SNP (Livesey and Marsh 2025)Evaluation	54%
MOIpred_benign (Livesey and Marsh 2025)Evaluation	53.7%
SIFT4G (Livesey and Marsh 2025)Evaluation	53.6%
DeMaSk (Livesey and Marsh 2025)Evaluation	53.2%
ProGen2 (Livesey and Marsh 2025)Evaluation	53.1%
DeepSAV (Livesey and Marsh 2025)Evaluation	52.9%
MISTIC (Livesey and Marsh 2025)Evaluation	52.3%
Eigen (Livesey and Marsh 2025)Evaluation	51.3%
MPC (Livesey and Marsh 2025)Evaluation	50.7%
SPRI (Livesey and Marsh 2025)Evaluation	50.3%
PolyPhen2_HumVar (Livesey and Marsh 2025)Evaluation	50%
SNPs&GO3D (Livesey and Marsh 2025)Evaluation	49.4%
CONDEL (Livesey and Marsh 2025)Evaluation	49%
MutPred (Livesey and Marsh 2025)Evaluation	49%
MetaLR (Livesey and Marsh 2025)Evaluation	48.3%
PROVEAN (Livesey and Marsh 2025)Evaluation	46%
RITA (Livesey and Marsh 2025)Evaluation	45.4%
SIGMA (Livesey and Marsh 2025)Evaluation	45.2%
PANTHER (Livesey and Marsh 2025)Evaluation	44.6%
PolyPhen2_HumDiv (Livesey and Marsh 2025)Evaluation	43.1%
Unirep_evotune (Livesey and Marsh 2025)Evaluation	42.9%
Wavenet (Livesey and Marsh 2025)Evaluation	42.4%
Envision (Livesey and Marsh 2025)Evaluation	42.4%
Maverick_benign (Livesey and Marsh 2025)Evaluation	41.3%
CADD_1.7 (Livesey and Marsh 2025)Evaluation	41.2%
M-CAP (Livesey and Marsh 2025)Evaluation	41.2%
LASSIE (Livesey and Marsh 2025)Evaluation	41%
MVP (Livesey and Marsh 2025)Evaluation	39.9%
Tranception (Livesey and Marsh 2025)Evaluation	39.3%
SIFT (Livesey and Marsh 2025)Evaluation	38.8%
MetaSVM (Livesey and Marsh 2025)Evaluation	38.5%
CADD_1.6 (Livesey and Marsh 2025)Evaluation	35.4%
AlphScore (Livesey and Marsh 2025)Evaluation	34.3%
CAPICE (Livesey and Marsh 2025)Evaluation	29.7%
InMeRF (Livesey and Marsh 2025)Evaluation	27.8%
PonP2 (Livesey and Marsh 2025)Evaluation	26.9%
LIST-S2 (Livesey and Marsh 2025)Evaluation	26.2%
FathmmXF (Livesey and Marsh 2025)Evaluation	22.6%
PrimateAI (Livesey and Marsh 2025)Evaluation	21.7%
sequence_unet (Livesey and Marsh 2025)Evaluation	20.3%
PhD-SNPg (Livesey and Marsh 2025)Evaluation	18.1%
fathmm-MKL (Livesey and Marsh 2025)Evaluation	16.3%
FATHMM (Livesey and Marsh 2025)Evaluation	16.1%
BLOSUM62 (Livesey and Marsh 2025)Evaluation	15.7%
ProtGPT2 (Livesey and Marsh 2025)Evaluation	14.3%
phyloP (Livesey and Marsh 2025)Evaluation	13.8%
DANN (Livesey and Marsh 2025)Evaluation	12.6%
LRT (Livesey and Marsh 2025)Evaluation	12.3%
Grantham (Livesey and Marsh 2025)Evaluation	9.79%
3Cnet (Livesey and Marsh 2025)Evaluation	9.57%
fitCons (Livesey and Marsh 2025)Evaluation	9.52%
GenoCanyon (Livesey and Marsh 2025)Evaluation	9.35%
GERP++ (Livesey and Marsh 2025)Evaluation	8.55%
SiPhy (Livesey and Marsh 2025)Evaluation	7.01%
phastCons (Livesey and Marsh 2025)Evaluation	5.5%

All 97 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: 3Cnet (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	9.57% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source 3Cnet: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B93 (row '3Cnet', column 'rank_score_indirect')
Configuration: AlphaMissense (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	89.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphaMissense: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B3 (row 'AlphaMissense', column 'rank_score_indirect')
Configuration: AlphScore (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	34.3% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphScore: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B76 (row 'AlphScore', column 'rank_score_indirect')
Configuration: BayesDel_noAF (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	57.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel_noAF: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B39 (row 'BayesDel_noAF', column 'rank_score_indirect')
Configuration: BayesDel (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	57% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B40 (row 'BayesDel', column 'rank_score_indirect')
Configuration: BLOSUM62 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	15.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BLOSUM62: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B87 (row 'BLOSUM62', column 'rank_score_indirect')
Configuration: CADD_1.6 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	35.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.6: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B75 (row 'CADD_1.6', column 'rank_score_indirect')
Configuration: CADD_1.7 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	41.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.7: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B68 (row 'CADD_1.7', column 'rank_score_indirect')
Configuration: CAPICE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	29.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CAPICE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B77 (row 'CAPICE', column 'rank_score_indirect')
Configuration: CARP (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	77.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CARP: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B15 (row 'CARP', column 'rank_score_indirect')
Configuration: ClinPred (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	60.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ClinPred: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B32 (row 'ClinPred', column 'rank_score_indirect')
Configuration: CONDEL (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	49% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CONDEL: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B56 (row 'CONDEL', column 'rank_score_indirect')
Configuration: CPT-1 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	92.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CPT-1: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B2 (row 'CPT-1', column 'rank_score_indirect')
Configuration: DANN (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	12.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DANN: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B90 (row 'DANN', column 'rank_score_indirect')
Configuration: DeepSAV (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	52.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSAV: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B49 (row 'DeepSAV', column 'rank_score_indirect')
Configuration: DeepSequence (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	78.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSequence: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B13 (row 'DeepSequence', column 'rank_score_indirect')
Configuration: DeMaSk (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	53.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeMaSk: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B47 (row 'DeMaSk', column 'rank_score_indirect')
Configuration: DEOGEN2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	54.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DEOGEN2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B43 (row 'DEOGEN2', column 'rank_score_indirect')
Configuration: Eigen (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	51.3% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Eigen: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B51 (row 'Eigen', column 'rank_score_indirect')
Configuration: Envision (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	42.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Envision: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B66 (row 'Envision', column 'rank_score_indirect')
Configuration: ESCOTT (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	85.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESCOTT: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B6 (row 'ESCOTT', column 'rank_score_indirect')
Configuration: ESM-1b (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	67% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1b: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B26 (row 'ESM-1b', column 'rank_score_indirect')
Configuration: ESM-1v (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	83.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1v: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B8 (row 'ESM-1v', column 'rank_score_indirect')
Configuration: ESM2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	70.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B23 (row 'ESM2', column 'rank_score_indirect')
Configuration: EVE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	72.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B21 (row 'EVE', column 'rank_score_indirect')

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 2 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

No execution recipe linked to this protocol. Recipe availability does not establish a completed evaluation.

External evaluations: 97

Literature evidence is not a Rewire measurement. Executed but unpublished runs and private review status are not included.

Null control

Proposed control: requires review

Select a task-valid null control after reviewing inputs and metric

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Conventional reference

Proposed control: requires review

Select an upstream conventional reference after reviewing the full protocol

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)

Coverage is derived from release 2026-10-22-1b7cf027e4f6. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

2 source records and release history

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Original source · PMC12016141 (published 2025-04-22; Genome Biol 26:104)
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8) · Original source · 13059_2025_3575_MOESM2_ESM.xlsx as published with PMC12016141

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: vep-lit-20261014-protocol-livesey2025-dms-indirect

areas: dna-genomes; proteins-complexes
contexts: research
scope note: Same pairwise procedure as the main DMS ranking, restricted to indirect assays, most commonly growth-rate experiments where the measured attribute is not directly controlled by the target protein.
task: Missense variant effect prediction ranked against DMS
metric: pairwise-win-rate
metric direction: higher
unit: percent
metric qualifier: average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only
metric implementation: scipy.stats.spearmanr (scipy 1.5.4, Python 3.6.8); absolute Spearman; the VEP with the higher |rho| on variants scored by the DMS and both VEPs earns the point
aggregation: Win rate of each VEP pair across proteins (wins divided by times the pair was tested), averaged over all other VEPs (Methods, 'Spearman's correlation and rank score')
version: Livesey and Marsh 2025, Supplementary Material 2 Table S5
publication status: peer_reviewed
source locator: Methods 'Spearman's correlation and rank score' and 'AUROC and AUPRC'; Supplementary Material 2 Legends (Table S5)
limitations: A rank score is relative to the other VEPs in the same table; values from different tables or editions are not comparable.; Per-pair, per-protein correlations and AUROCs behind the win rates are not printed; they are in the figshare deposit, which was not retrieved.; DMS assays measure a cellular or molecular phenotype chosen per protein, not pathogenicity.; Figure 2 error bars (standard deviation over 1000 bootstraps of the DMS sets) are not printed; Table S4 gives pairwise bootstrap counts, not extracted.; Five VEPs (Envision, CPT-1, DeMaSk, VARITY_R, VARITY_ER) have DMS sets they were trained on excluded, so they are compared on fewer proteins.
review: reviewer: claude; reviewer note: Claude review agent; no human review claimed. Independent Claude (Opus 5.5) review agent, separate from the extraction agent that built this batch. Automated review only. Transcription, not reproduction.; reviewed at: 2026-10-14T15:30:00Z; artifact sha256: 1735ee461ef192d9de67c914e9ad73d72d8aa255ea0e3479eb68cd8631d99fa2; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12016141/fullTextXML; method: ai-assisted-source-review; scope-review; method note: Descriptive fields, identity links and missing-metadata reasons checked against the cited sources.; note: Checked; see the dated review for rulings on identity and method type.

Related records

Suggest a correction

Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: 3Cnet (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	9.57% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source 3Cnet: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B93 (row '3Cnet', column 'rank_score_indirect')
Configuration: AlphaMissense (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	89.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphaMissense: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B3 (row 'AlphaMissense', column 'rank_score_indirect')
Configuration: AlphScore (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	34.3% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source AlphScore: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B76 (row 'AlphScore', column 'rank_score_indirect')
Configuration: BayesDel_noAF (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	57.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel_noAF: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B39 (row 'BayesDel_noAF', column 'rank_score_indirect')
Configuration: BayesDel (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	57% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BayesDel: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B40 (row 'BayesDel', column 'rank_score_indirect')
Configuration: BLOSUM62 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	15.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source BLOSUM62: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B87 (row 'BLOSUM62', column 'rank_score_indirect')
Configuration: CADD_1.6 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	35.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.6: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B75 (row 'CADD_1.6', column 'rank_score_indirect')
Configuration: CADD_1.7 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	41.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CADD_1.7: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B68 (row 'CADD_1.7', column 'rank_score_indirect')
Configuration: CAPICE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	29.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CAPICE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B77 (row 'CAPICE', column 'rank_score_indirect')
Configuration: CARP (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	77.7% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CARP: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B15 (row 'CARP', column 'rank_score_indirect')
Configuration: ClinPred (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	60.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ClinPred: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B32 (row 'ClinPred', column 'rank_score_indirect')
Configuration: CONDEL (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	49% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CONDEL: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B56 (row 'CONDEL', column 'rank_score_indirect')
Configuration: CPT-1 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	92.1% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source CPT-1: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B2 (row 'CPT-1', column 'rank_score_indirect')
Configuration: DANN (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	12.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DANN: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B90 (row 'DANN', column 'rank_score_indirect')
Configuration: DeepSAV (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	52.9% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSAV: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B49 (row 'DeepSAV', column 'rank_score_indirect')
Configuration: DeepSequence (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	78.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeepSequence: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B13 (row 'DeepSequence', column 'rank_score_indirect')
Configuration: DeMaSk (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	53.2% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DeMaSk: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B47 (row 'DeMaSk', column 'rank_score_indirect')
Configuration: DEOGEN2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	54.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source DEOGEN2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B43 (row 'DEOGEN2', column 'rank_score_indirect')
Configuration: Eigen (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	51.3% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Eigen: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B51 (row 'Eigen', column 'rank_score_indirect')
Configuration: Envision (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	42.4% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Envision: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B66 (row 'Envision', column 'rank_score_indirect')
Configuration: ESCOTT (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	85.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESCOTT: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B6 (row 'ESCOTT', column 'rank_score_indirect')
Configuration: ESM-1b (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	67% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1b: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B26 (row 'ESM-1b', column 'rank_score_indirect')
Configuration: ESM-1v (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	83.6% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM-1v: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B8 (row 'ESM-1v', column 'rank_score_indirect')
Configuration: ESM2 (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	70.8% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source ESM2: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B23 (row 'ESM2', column 'rank_score_indirect')
Configuration: EVE (Livesey and Marsh 2025)	Protocol: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Dataset: Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	72.5% Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only) percent · higher Uncertainty: Not reported by the source: Bootstrap standard deviations appear only as Figure 2 error bars; not printed Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE: Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only Aggregation: Not reported Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) · Supplementary Material 2, sheet 'Table S5', cell B21 (row 'EVE', column 'rank_score_indirect')