MPC and REVEL: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

Which studies agree

3 papers rank MPC above REVEL in 12 of their 17 comparisons and REVEL higher in 3; 2 are ties. Of those papers, ProteinGym AMFR assay-level Spearman… ranks REVEL higher on its own (1 of 1). Livesey and Marsh 2025 ranks REVEL above MPC in all 7 of its comparisons.

What disagrees

Other studies

rank MPC above REVEL in 12 of 15 comparisons

This study

ranks REVEL above MPC in 7 of 7 comparisons

Compared

MPC, REVEL

Study that disagrees

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8)

Other studies

12 studies

Comprehensive assessment of computational algorithms in predicting cancer driver mutations, Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB), Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability), Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability), Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB), An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers, OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role), ProteinGym AMFR assay-level Spearman comparison, ProteinGym clinical supervised substitutions AUC summary at commit 144fe22, ProteinGym clinical zero-shot indels AUC summary at commit 144fe22, ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22, ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22

Rule

Ordering reversal; different tasks across studies

Model identity

Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-8529b47e85e2ba50 · All contested results · How findings are made and followed up