Contested result

ESM-2 and CADD: Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

Low severity
Not yet reviewed
two-study-conflict
different-task

Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks CADD above ESM-2 in 14 of 14 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above CADD in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank ESM-2 above CADD in 6 of their 21 comparisons and CADD higher in 15. Of those papers, Genome modelling and design across all… ranks CADD higher on its own (14 of 14).

Rank ESM-2 higher (1): Livesey and Marsh 2025
Rank CADD higher (1): Genome modelling and design across all…
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

ESM-2 higherTieCADD higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks ESM-2 higher	Ranks CADD higher	Ties
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) and 1 more record · DOI	Agrees	0	14	0
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOI	Agrees	6	1	0

Comparisons in Genome modelling and design across all… (14)

Protocol	Dataset	Metric	ESM-2	CADD	Higher
Zero-shot ClinVar coding SNVs classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs), unitless	0.827	0.952	CADD
Zero-shot ClinVar coding SNVs classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs), unitless	0.677	0.927	CADD
Zero-shot ClinVar coding SNVs, far from splice site classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs, far from splice site), unitless	0.822	0.949	CADD
Zero-shot ClinVar coding SNVs, far from splice site classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs, far from splice site), unitless	0.678	0.925	CADD
Zero-shot ClinVar coding SNVs, highly conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs, highly conserved), unitless	0.895	0.961	CADD
Zero-shot ClinVar coding SNVs, highly conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs, highly conserved), unitless	0.657	0.883	CADD
Zero-shot ClinVar coding SNVs, moderately conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs, moderately conserved), unitless	0.62	0.818	CADD
Zero-shot ClinVar coding SNVs, moderately conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs, moderately conserved), unitless	0.706	0.905	CADD
Zero-shot ClinVar coding SNVs, near splice site classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs, near splice site), unitless	0.896	0.977	CADD
Zero-shot ClinVar coding SNVs, near splice site classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs, near splice site), unitless	0.68	0.934	CADD
Zero-shot ClinVar coding SNVs, not conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs, not conserved), unitless	0.493	0.623	CADD
Zero-shot ClinVar coding SNVs, not conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs, not conserved), unitless	0.718	0.88	CADD
Zero-shot ClinVar coding SNVs, poorly conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUPRC (zero-shot; ClinVar coding SNVs, poorly conserved), unitless	0.503	0.759	CADD
Zero-shot ClinVar coding SNVs, poorly conserved classification (Evo 2 paper, Supplementary Data 1)	ClinVar October 2024 release, filtered (Brixi et al. 2026)	AUROC (zero-shot; ClinVar coding SNVs, poorly conserved), unitless	0.696	0.912	CADD

Comparisons in Livesey and Marsh 2025 (7)

Protocol	Dataset	Metric	ESM-2	CADD	Higher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent	51.4	52.1	CADD
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	72.2	40.3	ESM-2
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	27.8	2.89	ESM-2
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only), percent	74.2	39	ESM-2
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only), percent	70.8	41.2	ESM-2
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only), percent	70.5	40.3	ESM-2
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent	71.1	42	ESM-2

What disagrees

studies: 2

Compared: ESM-2, CADD
Study that disagrees
Other studies
Rule: two-study-conflict; different-task
Model identity: Matched across studies partly through links not yet reviewed

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-6ca27ffce846d52c · All contested results · How findings are made and followed up