Contested result

SiPhy and GERP: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SiPhy above GERP in 4 of 4 comparisons; other studies rank GERP above SiPhy in 12 of 13.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank GERP above SiPhy in 12 of their 13 comparisons and SiPhy higher in 1. Brandes et al. 2023 ranks SiPhy above GERP in all 4 of its comparisons.

Rank GERP higher (2): Livesey and Marsh 2025, Clinical likelihood ratios and balanced…
Rank SiPhy higher (1): Brandes et al. 2023 (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

GERP higherTieSiPhy higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (3)
Paper	Role	Ranks GERP higher	Ranks SiPhy higher
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · DOI	Disagrees	0	4
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOI	Agrees	6	1
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and 1 more record · DOI	Agrees	6	0

Comparisons in Brandes et al. 2023 (4)

Protocol	Dataset	Metric	GERP	SiPhy	Higher
ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot), unitless	0.641	0.666	SiPhy
ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	AUROC (global ROC-AUC over all variants; zero-shot), unitless	0.704	0.738	SiPhy
HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot), unitless	0.645	0.671	SiPhy
HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	AUROC (global ROC-AUC over all variants; zero-shot), unitless	0.7	0.743	SiPhy

Comparisons in Livesey and Marsh 2025 (7)

Protocol	Dataset	Metric	GERP	SiPhy	Higher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent	12.4	16.6	SiPhy
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	8.8	7.48	GERP
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	0.667	0	GERP
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only), percent	9.13	8.13	GERP
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only), percent	8.55	7.01	GERP
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only), percent	8.8	7.48	GERP
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent	9.43	8.32	GERP

Comparisons in Clinical likelihood ratios and balanced… (6)

Protocol	Dataset	Metric	GERP	SiPhy	Higher
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editing	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set), unitless	7.59	0.722	GERP
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editing	False negatives (Functionally deleterious BRCA1 variants called tolerated), count	6	105	GERP
BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA1 functional truth set: 1641 missense variants classed deleterious or tolerated by HAP1 cell survival after saturation genome editing	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set), unitless	1.78	0.9	GERP
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set), unitless	5.98	4.68	GERP
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair	False negatives (Functionally deleterious BRCA2 variants called tolerated), count	0	2	GERP
BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set), unitless	1.26	1.13	GERP

What disagrees

Other studies: rank GERP above SiPhy in 12 of 13 comparisons
This study: ranks SiPhy above GERP in 4 of 4 comparisons

Compared: SiPhy, GERP
Study that disagrees: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods
Other studies: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes, Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx), Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8)
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (8)
Study	Tool	Protocol or benchmark	Metric	Value
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	GERP++ (Brandes et al. 2023 clinical benchmarks)	ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.641
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	GERP++ (Brandes et al. 2023 clinical benchmarks)	ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUROC (global ROC-AUC over all variants; zero-shot)	0.704
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	SiPhy (Brandes et al. 2023 clinical benchmarks)	ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.666
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	SiPhy (Brandes et al. 2023 clinical benchmarks)	ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUROC (global ROC-AUC over all variants; zero-shot)	0.738
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	GERP++ (Brandes et al. 2023 clinical benchmarks)	HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.645
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	GERP++ (Brandes et al. 2023 clinical benchmarks)	HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUROC (global ROC-AUC over all variants; zero-shot)	0.7
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	SiPhy (Brandes et al. 2023 clinical benchmarks)	HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.671
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	SiPhy (Brandes et al. 2023 clinical benchmarks)	HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)	AUROC (global ROC-AUC over all variants; zero-shot)	0.743

In the other studies (26)
Study	Tool	Protocol or benchmark	Metric	Value
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	GERP++_RS_a (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)	7.59
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	GERP++_RS_a (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA1 variants called tolerated)	6
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	GERP++_RS_a (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	0
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	GERP++_RS_b (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set)	5.98
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	GERP++_RS_c (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)	1.78
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	GERP++_RS_c (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)	1.26
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	SiPhy_29way_logOdds (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA1 functional truth set)	0.722
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	SiPhy_29way_logOdds (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA1 variants called tolerated)	105
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	SiPhy_29way_logOdds (Cubuk et al. 2021)	BRCA1 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA1 functional truth set)	0.9
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	SiPhy_29way_logOdds (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Benignity likelihood ratio (Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set)	4.68
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	SiPhy_29way_logOdds (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	False negatives (Functionally deleterious BRCA2 variants called tolerated)	2
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx)	SiPhy_29way_logOdds (Cubuk et al. 2021)	BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)	Likelihood ratio (Positive likelihood ratio of a deleterious call (TPR/FPR), BRCA2 functional truth set)	1.13
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	12.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	16.6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	8.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)	0.667
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	7.48
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)	0
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	9.13
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	8.13
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	8.55
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	7.01
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	9.43
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	8.32
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	GERP++ (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	8.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	SiPhy (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	7.48

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-f751e8d31b2c2117 · All contested results · How findings are made and followed up