Contested result

MutPred and ESM-1b: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutPred in 6 of 7 comparisons; other studies rank MutPred above ESM-1b in 8 of 9.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

3 papers rank MutPred above ESM-1b in 8 of their 11 comparisons and ESM-1b higher in 1; 2 are ties. Of those papers, ProteinGym AMFR assay-level Spearman… ranks ESM-1b higher on its own (1 of 1). Livesey and Marsh 2025 ranks ESM-1b above MutPred in 6 of its 7 comparisons and MutPred higher in 1.

Rank MutPred higher (2): An openly licensed benchmark…, Brandes et al. 2023
Rank ESM-1b higher (2): Livesey and Marsh 2025 (disagrees), ProteinGym AMFR assay-level Spearman…
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

MutPred higherTieESM-1b higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (4)
Paper	Role	Ranks MutPred higher	Ranks ESM-1b higher	Ties
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOI	Disagrees	1	6	0
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers and 1 more record · DOI	Agrees	5	0	1
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · DOI	Agrees	3	0	1
ProteinGym AMFR assay-level Spearman comparison and 4 more records · Repository	Agrees	0	1	0

Comparisons in Livesey and Marsh 2025 (7)

Protocol	Dataset	Metric	MutPred	ESM-1b	Higher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent	65.1	54.3	MutPred
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	47.9	65	ESM-1b
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets), percent	14.4	14.7	ESM-1b
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only), percent	46.5	62.4	ESM-1b
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only), percent	49	67	ESM-1b
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only), percent	47.9	64.5	ESM-1b
Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Livesey and Marsh 2025 human DMS benchmark set (36 proteins)	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent	49.2	65.9	ESM-1b

Comparisons in An openly licensed benchmark… (6)

Protocol	Dataset	Metric	MutPred	ESM-1b	Higher
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	AUROC (CGC missense, all genes), unitless	0.846	0.842	Tie
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	Average precision (CGC missense, all genes), unitless	0.664	0.641	MutPred
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	AUROC (CGC missense, oncogenes (gain of function)), unitless	0.876	0.841	MutPred
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	Average precision (CGC missense, oncogenes (gain of function)), unitless	0.773	0.719	MutPred
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	AUROC (CGC missense, tumour suppressors (loss of function)), unitless	0.896	0.858	MutPred
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels	Average precision (CGC missense, tumour suppressors (loss of function)), unitless	0.661	0.594	MutPred

Comparisons in Brandes et al. 2023 (4)

Protocol	Dataset	Metric	MutPred	ESM-1b	Higher
ClinVar missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	ClinVar missense variants scored by both ESM1b and MutPred (Brandes et al. 2023)	AUPRC (balanced PRC-AUC over the variants both methods score: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot), unitless	0.907	0.892	MutPred
ClinVar missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	ClinVar missense variants scored by both ESM1b and MutPred (Brandes et al. 2023)	AUROC (global ROC-AUC over the variants both methods score; zero-shot), unitless	0.903	0.9	Tie
HGMD/gnomAD missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	HGMD/gnomAD missense variants scored by both ESM1b and MutPred (Brandes et al. 2023)	AUPRC (balanced PRC-AUC over the variants both methods score: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot), unitless	0.921	0.898	MutPred
HGMD/gnomAD missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	HGMD/gnomAD missense variants scored by both ESM1b and MutPred (Brandes et al. 2023)	AUROC (global ROC-AUC over the variants both methods score; zero-shot), unitless	0.924	0.899	MutPred

Comparisons in ProteinGym AMFR assay-level Spearman… (1)

Protocol	Dataset	Metric	MutPred	ESM-1b	Higher
ProteinGym clinical substitutions, zero-shot, average AUC (commit 144fe22)	ProteinGym clinical substitution benchmark at commit 144fe22	AUROC (ProteinGym clinical substitutions, zero-shot, Average_AUC over proteins), unitless	0.875	0.892	ESM-1b

What disagrees

Other studies: rank MutPred above ESM-1b in 8 of 9 comparisons
This study: ranks ESM-1b above MutPred in 6 of 7 comparisons

Compared: MutPred, ESM-1b
Study that disagrees: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8)
Other studies: 8 studies
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods, An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers, OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role), ProteinGym AMFR assay-level Spearman comparison, ProteinGym clinical supervised substitutions AUC summary at commit 144fe22, ProteinGym clinical zero-shot indels AUC summary at commit 144fe22, ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22, ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (14)
Study	Tool	Protocol or benchmark	Metric	Value
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	54.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense	Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)	65.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	65
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)	14.7
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, 36 human DMS sets)	47.9
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, all 36 DMS sets	Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), \|Spearman\| with DMS on shared variants, 36 human DMS sets)	14.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	62.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, direct assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, direct DMS assays only)	46.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	67
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, indirect assays only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, indirect DMS assays only)	49
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	64.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	ESM-1b (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	65.9
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions with gaps filled	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)	49.2
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)	MutPred (Livesey and Marsh 2025)	Livesey and Marsh 2025 DMS benchmark: pairwise \|Spearman\| win rate, SNV-accessible substitutions only	Pairwise win rate (average pairwise win rate against all other VEPs, \|Spearman\| with DMS on shared variants, SNV-accessible substitutions only)	47.9

In the other studies (18)
Study	Tool	Protocol or benchmark	Metric	Value
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	ESM1b (Brandes et al. 2023 clinical benchmarks)	ClinVar missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	AUPRC (balanced PRC-AUC over the variants both methods score: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.892
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	MutPred (Brandes et al. 2023 head-to-head comparison)	ClinVar missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	AUPRC (balanced PRC-AUC over the variants both methods score: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.907
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	ESM1b (Brandes et al. 2023 clinical benchmarks)	HGMD/gnomAD missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	AUPRC (balanced PRC-AUC over the variants both methods score: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.898
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	ESM1b (Brandes et al. 2023 clinical benchmarks)	HGMD/gnomAD missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	AUROC (global ROC-AUC over the variants both methods score; zero-shot)	0.899
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	MutPred (Brandes et al. 2023 head-to-head comparison)	HGMD/gnomAD missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	AUPRC (balanced PRC-AUC over the variants both methods score: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	0.921
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods; Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023)	MutPred (Brandes et al. 2023 head-to-head comparison)	HGMD/gnomAD missense pathogenicity, ESM1b head to head with one other method on the variants both score (Brandes et al. 2023 Fig. 2g,h, Supplementary Table 2)	AUROC (global ROC-AUC over the variants both methods score; zero-shot)	0.924
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	ESM1b rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, all genes)	0.641
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	ESM1b rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, oncogenes (gain of function))	0.719
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	ESM1b rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, oncogenes (gain of function))	0.841
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	ESM1b rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, tumour suppressors (loss of function))	0.594
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	ESM1b rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, tumour suppressors (loss of function))	0.858
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	MutPred2 rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, all genes)	0.664
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	MutPred2 rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, oncogenes (gain of function))	0.773
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	MutPred2 rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, oncogenes (gain of function))	0.876
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	MutPred2 rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	Average precision (CGC missense, tumour suppressors (loss of function))	0.661
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)	MutPred2 rank score, dbNSFP 5.3.1a (Lee 2026)	CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)	AUROC (CGC missense, tumour suppressors (loss of function))	0.896
ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22	ESM-1b [ProteinGym 144fe22b snapshot]	ProteinGym clinical substitutions, zero-shot, average AUC (commit 144fe22)	AUROC (ProteinGym clinical substitutions, zero-shot, Average_AUC over proteins)	0.892
ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22	MutPred [ProteinGym 144fe22 clinical summary]	ProteinGym clinical substitutions, zero-shot, average AUC (commit 144fe22)	AUROC (ProteinGym clinical substitutions, zero-shot, Average_AUC over proteins)	0.875

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-80d23e35d6f0e984 · All contested results · How findings are made and followed up