rewirebio.iobenchmarks
Evidence and investigation

Contested results

Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.

These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.

A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.

How the check works, and how each follow-up is recorded: discrepancies between studies.

In this release

  • 192open
  • 6settled
  • 7resolved
  • 3medium severity
  • 202low severity

Release 2026-11-21-c8bea25e5abb · 205 findings in all · rules version 5

Findings

Status: All · Open · Settled · Resolved

Severity: All · Medium · Low

State: All · Not yet reviewed · Under investigation · Explained · Confirmed error · Not a real disagreement

Showing findings that involve results in Predict missense variant effects in a protein of interest, from the study that disagrees or from the agreeing group. All use cases

141 findings match, most severe first.

ESM3 relative to ESM-2: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design is outside the range of other studies

  • Low severity
  • Not yet reviewed
  • Gap outlier
  • Same task, different protocols

On ProteinGym mutation effects, ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design reports ESM3 at a median relative difference of +0.20 from ESM-2; other studies report -0.06 to -0.01.

In 2 papers, ESM3 scores between 1% and 6% below ESM-2. In ProteinGym, ESM3 scores 20% above ESM-2.

How far ESM3 scores from ESM-2 in each paperThe other papers range from −6% to −1%. ProteinGym (disagrees): +20%. HyCell CITATION.cff: −1%. pfmbench primary benchmark evidence: −6%.ProteinGym: +20%ProteinGym +20%HyCell CITATION.cff: −1%pfmbench primary benchmark evidence: −6%−6%= ESM-2+20%
Range of the other papers
Compared
ESM3, ESM-2

Not reviewed yet.

LRT and MetaLR: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaLR in 15 of 17 comparisons; 2 of 2 other studies rank MetaLR above LRT.

2 papers rank MetaLR above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaLR in 15 of its 20 comparisons and MetaLR higher in 2; 3 are ties.

Comparisons in which each paper scores MetaLR or LRT higher2 other papers: MetaLR higher in 12, LRT higher in 1, 0 ties. Comprehensive assessment… (disagrees): MetaLR higher in 2, LRT higher in 15, 3 ties.2 other papers12 MetaLR higher121 LRT higherDisagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations2 MetaLR higher23 tie315 LRT higher15
MetaLR higherTieLRT higher
Compared
LRT, MetaLR

Not reviewed yet.

CADD and MetaLR: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above CADD in 6 of 7 comparisons; 2 of 2 other studies rank CADD above MetaLR.

3 papers rank CADD above MetaLR in 19 of their 21 comparisons and MetaLR higher in 2. Livesey and Marsh 2025 ranks MetaLR above CADD in 6 of its 7 comparisons and CADD higher in 1.

Comparisons in which each paper scores CADD or MetaLR higher3 other papers: CADD higher in 19, MetaLR higher in 2, 0 ties. Livesey and Marsh 2025 (disagrees): CADD higher in 1, MetaLR higher in 6, 0 ties.3 other papers19 CADD higher192 MetaLR higher2Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 CADD higher16 MetaLR higher6
CADD higherTieMetaLR higher
Compared
CADD, MetaLR

Not reviewed yet.

CADD and FATHMM-XF: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM-XF above CADD in 10 of 10 comparisons; 2 of 2 other studies rank CADD above FATHMM-XF.

2 papers rank CADD above FATHMM-XF in all 12 comparisons that are not ties (1 is a tie). Comprehensive assessment… ranks FATHMM-XF above CADD in all 10 of its comparisons.

Comparisons in which each paper scores CADD or FATHMM-XF higher2 other papers: CADD higher in 12, FATHMM-XF higher in 0, 1 tie. Comprehensive assessment… (disagrees): CADD higher in 0, FATHMM-XF higher in 10, 0 ties.2 other papers12 CADD higher121 tieDisagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations10 FATHMM-XF higher10
CADD higherTieFATHMM-XF higher
Compared
CADD, FATHMM-XF

Not reviewed yet.

MVP and ESM-1b: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MVP above ESM-1b in 2 of 2 comparisons; 2 of 2 other studies rank ESM-1b above MVP.

2 papers rank ESM-1b above MVP in all 13 of their comparisons. Brandes et al. 2023 ranks MVP above ESM-1b in all 2 comparisons that are not ties (2 are ties).

Comparisons in which each paper scores ESM-1b or MVP higher2 other papers: ESM-1b higher in 13, MVP higher in 0, 0 ties. Brandes et al. 2023 (disagrees): ESM-1b higher in 0, MVP higher in 2, 2 ties.2 other papers13 ESM-1b higher13Disagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods2 tie22 MVP higher2
ESM-1b higherTieMVP higher
Compared
MVP, ESM-1b

Not reviewed yet.

DANN and GERP: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks GERP above DANN in 5 of 6 comparisons; 2 of 2 other studies rank DANN above GERP.

2 papers rank DANN above GERP in all 12 comparisons that are not ties (1 is a tie). Clinical likelihood ratios and balanced… ranks GERP above DANN in 5 of its 6 comparisons and DANN higher in 1.

Comparisons in which each paper scores DANN or GERP higher2 other papers: DANN higher in 12, GERP higher in 0, 1 tie. Clinical likelihood ratios and balanced… (disagrees): DANN higher in 1, GERP higher in 5, 0 ties.2 other papers12 DANN higher121 tieDisagrees: Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 DANN higher15 GERP higher5
DANN higherTieGERP higher
Compared
DANN, GERP

Not reviewed yet.

ClinPred and REVEL: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above ClinPred in 6 of 7 comparisons; 2 of 2 other studies rank ClinPred above REVEL.

3 papers rank ClinPred above REVEL in 10 of their 12 comparisons and REVEL higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks REVEL above ClinPred in 6 of its 7 comparisons and ClinPred higher in 1.

Comparisons in which each paper scores ClinPred or REVEL higher3 other papers: ClinPred higher in 10, REVEL higher in 1, 1 tie. Livesey and Marsh 2025 (disagrees): ClinPred higher in 1, REVEL higher in 6, 0 ties.3 other papers10 ClinPred higher101 tie1 REVEL higherDisagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 ClinPred higher16 REVEL higher6
ClinPred higherTieREVEL higher
Compared
ClinPred, REVEL

Not reviewed yet.

BayesDel and ESM-1b: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above BayesDel in 6 of 7 comparisons; 2 of 2 other studies rank BayesDel above ESM-1b.

2 papers rank BayesDel above ESM-1b in all 10 of their comparisons. Livesey and Marsh 2025 ranks ESM-1b above BayesDel in 6 of its 7 comparisons and BayesDel higher in 1.

Comparisons in which each paper scores BayesDel or ESM-1b higher2 other papers: BayesDel higher in 10, ESM-1b higher in 0, 0 ties. Livesey and Marsh 2025 (disagrees): BayesDel higher in 1, ESM-1b higher in 6, 0 ties.2 other papers10 BayesDel higher10Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 BayesDel higher16 ESM-1b higher6
BayesDel higherTieESM-1b higher
Compared
BayesDel, ESM-1b

Not reviewed yet.

MVP and VEST4: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above VEST4 in 3 of 4 comparisons; 2 of 2 other studies rank VEST4 above MVP.

3 papers rank VEST4 above MVP in 17 of their 23 comparisons and MVP higher in 6. Clinical likelihood ratios and balanced… ranks MVP above VEST4 in 3 of its 5 comparisons and VEST4 higher in 1; 1 is a tie.

Comparisons in which each paper scores VEST4 or MVP higher3 other papers: VEST4 higher in 17, MVP higher in 6, 0 ties. Clinical likelihood ratios and balanced… (disagrees): VEST4 higher in 1, MVP higher in 3, 1 tie.3 other papers17 VEST4 higher176 MVP higher6Disagrees: Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 VEST4 higher11 tie13 MVP higher3
VEST4 higherTieMVP higher
Compared
MVP, VEST4

Not reviewed yet.

CADD and MVP: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MVP above CADD in 10 of 10 comparisons; 2 of 2 other studies rank CADD above MVP.

3 papers rank CADD above MVP in 11 of their 18 comparisons and MVP higher in 4; 3 are ties. Of those papers, Clinical likelihood ratios and balanced… ranks MVP higher on its own (2 of 5). Comprehensive assessment… ranks MVP above CADD in all 10 of its comparisons.

Comparisons in which each paper scores CADD or MVP higher3 other papers: CADD higher in 11, MVP higher in 4, 3 ties. Comprehensive assessment… (disagrees): CADD higher in 0, MVP higher in 10, 0 ties.3 other papers11 CADD higher113 tie34 MVP higher4Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations10 MVP higher10
CADD higherTieMVP higher
Compared
CADD, MVP

Not reviewed yet.

MVP and REVEL: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above REVEL in 4 of 5 comparisons; 3 of 3 other studies rank REVEL above MVP.

3 papers rank REVEL above MVP in all 19 comparisons that are not ties (4 are ties). Clinical likelihood ratios and balanced… ranks MVP above REVEL in 4 of its 5 comparisons and REVEL higher in 1.

Comparisons in which each paper scores REVEL or MVP higher3 other papers: REVEL higher in 19, MVP higher in 0, 4 ties. Clinical likelihood ratios and balanced… (disagrees): REVEL higher in 1, MVP higher in 4, 0 ties.3 other papers19 REVEL higher194 tie4Disagrees: Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 REVEL higher14 MVP higher4
REVEL higherTieMVP higher
Compared
MVP, REVEL

Not reviewed yet.

Eigen and MutationAssessor: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Explained
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above Eigen in 6 of 7 comparisons; 3 of 3 other studies rank Eigen above MutationAssessor.

3 papers rank Eigen above MutationAssessor in 17 of their 22 comparisons and MutationAssessor higher in 1; 4 are ties. Livesey and Marsh 2025 ranks MutationAssessor above Eigen in 6 of its 7 comparisons and Eigen higher in 1.

Comparisons in which each paper scores Eigen or MutationAssessor higher3 other papers: Eigen higher in 17, MutationAssessor higher in 1, 4 ties. Livesey and Marsh 2025 (disagrees): Eigen higher in 1, MutationAssessor higher in 6, 0 ties.3 other papers17 Eigen higher174 tie41 MutationAssessor higherDisagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 Eigen higher16 MutationAssessor higher6
Eigen higherTieMutationAssessor higher
Compared
Eigen, MutationAssessor

1 follow-up action, latest 23 October 2026

ESM-1b and EVE: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Explained
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks ESM-1b above EVE in 6 of 6 comparisons; 2 of 2 other studies rank EVE above ESM-1b.

2 papers rank EVE above ESM-1b in 22 of their 26 comparisons and ESM-1b higher in 1; 3 are ties. Brandes et al. 2023 ranks ESM-1b above EVE in all 6 comparisons that are not ties (2 are ties).

Comparisons in which each paper scores EVE or ESM-1b higher2 other papers: EVE higher in 22, ESM-1b higher in 1, 3 ties. Brandes et al. 2023 (disagrees): EVE higher in 0, ESM-1b higher in 6, 2 ties.2 other papers22 EVE higher223 tie31 ESM-1b higherDisagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods2 tie26 ESM-1b higher6
EVE higherTieESM-1b higher
Compared
ESM-1b, EVE

3 follow-up actions, latest 21 November 2026

M-CAP and PrimateAI: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above PrimateAI in 7 of 7 comparisons; 2 of 2 other studies rank PrimateAI above M-CAP.

3 papers rank PrimateAI above M-CAP in 24 of their 30 comparisons and M-CAP higher in 6. Livesey and Marsh 2025 ranks M-CAP above PrimateAI in all 7 of its comparisons.

Comparisons in which each paper scores PrimateAI or M-CAP higher3 other papers: PrimateAI higher in 24, M-CAP higher in 6, 0 ties. Livesey and Marsh 2025 (disagrees): PrimateAI higher in 0, M-CAP higher in 7, 0 ties.3 other papers24 PrimateAI higher246 M-CAP higher6Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 M-CAP higher7
PrimateAI higherTieM-CAP higher
Compared
M-CAP, PrimateAI

Not reviewed yet.

FATHMM-MKL and MutationAssessor: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above FATHMM-MKL in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-MKL above MutationAssessor.

2 papers rank FATHMM-MKL above MutationAssessor in 19 of their 25 comparisons and MutationAssessor higher in 6. Livesey and Marsh 2025 ranks MutationAssessor above FATHMM-MKL in all 7 of its comparisons.

Comparisons in which each paper scores FATHMM-MKL or MutationAssessor higher2 other papers: FATHMM-MKL higher in 19, MutationAssessor higher in 6, 0 ties. Livesey and Marsh 2025 (disagrees): FATHMM-MKL higher in 0, MutationAssessor higher in 7, 0 ties.2 other papers19 FATHMM-MKL higher196 MutationAssessor higher6Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MutationAssessor higher7
FATHMM-MKL higherTieMutationAssessor higher
Compared
FATHMM-MKL, MutationAssessor

Not reviewed yet.

GenoCanyon and SIFT: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks GenoCanyon above SIFT in 6 of 8 comparisons; 3 of 3 other studies rank SIFT above GenoCanyon.

3 papers rank SIFT above GenoCanyon in 14 of their 16 comparisons and GenoCanyon higher in 1; 1 is a tie. Comprehensive assessment… ranks GenoCanyon above SIFT in 6 of its 10 comparisons and SIFT higher in 2; 2 are ties.

Comparisons in which each paper scores SIFT or GenoCanyon higher3 other papers: SIFT higher in 14, GenoCanyon higher in 1, 1 tie. Comprehensive assessment… (disagrees): SIFT higher in 2, GenoCanyon higher in 6, 2 ties.3 other papers14 SIFT higher141 tie1 GenoCanyon higherDisagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations2 SIFT higher22 tie26 GenoCanyon higher6
SIFT higherTieGenoCanyon higher
Compared
GenoCanyon, SIFT

Not reviewed yet.

MetaLR and ESM-1b: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaLR above ESM-1b in 2 of 2 comparisons; 2 of 2 other studies rank ESM-1b above MetaLR.

2 papers rank ESM-1b above MetaLR in all 13 of their comparisons. Brandes et al. 2023 ranks MetaLR above ESM-1b in all 2 comparisons that are not ties (2 are ties).

Comparisons in which each paper scores ESM-1b or MetaLR higher2 other papers: ESM-1b higher in 13, MetaLR higher in 0, 0 ties. Brandes et al. 2023 (disagrees): ESM-1b higher in 0, MetaLR higher in 2, 2 ties.2 other papers13 ESM-1b higher13Disagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods2 tie22 MetaLR higher2
ESM-1b higherTieMetaLR higher
Compared
MetaLR, ESM-1b

Not reviewed yet.

MetaSVM and ESM-1b: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaSVM above ESM-1b in 2 of 2 comparisons; 2 of 2 other studies rank ESM-1b above MetaSVM.

2 papers rank ESM-1b above MetaSVM in all 13 of their comparisons. Brandes et al. 2023 ranks MetaSVM above ESM-1b in all 2 comparisons that are not ties (2 are ties).

Comparisons in which each paper scores ESM-1b or MetaSVM higher2 other papers: ESM-1b higher in 13, MetaSVM higher in 0, 0 ties. Brandes et al. 2023 (disagrees): ESM-1b higher in 0, MetaSVM higher in 2, 2 ties.2 other papers13 ESM-1b higher13Disagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods2 tie22 MetaSVM higher2
ESM-1b higherTieMetaSVM higher
Compared
MetaSVM, ESM-1b

Not reviewed yet.

FATHMM-XF and M-CAP: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above M-CAP.

2 papers rank FATHMM-XF above M-CAP in 22 of their 26 comparisons and M-CAP higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks M-CAP above FATHMM-XF in all 7 of its comparisons.

Comparisons in which each paper scores FATHMM-XF or M-CAP higher2 other papers: FATHMM-XF higher in 22, M-CAP higher in 3, 1 tie. Livesey and Marsh 2025 (disagrees): FATHMM-XF higher in 0, M-CAP higher in 7, 0 ties.2 other papers22 FATHMM-XF higher221 tie3 M-CAP higher3Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 M-CAP higher7
FATHMM-XF higherTieM-CAP higher
Compared
FATHMM-XF, M-CAP

Not reviewed yet.

FATHMM-MKL and MetaLR: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above FATHMM-MKL in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-MKL above MetaLR.

2 papers rank FATHMM-MKL above MetaLR in 18 of their 25 comparisons and MetaLR higher in 5; 2 are ties. Livesey and Marsh 2025 ranks MetaLR above FATHMM-MKL in all 7 of its comparisons.

Comparisons in which each paper scores FATHMM-MKL or MetaLR higher2 other papers: FATHMM-MKL higher in 18, MetaLR higher in 5, 2 ties. Livesey and Marsh 2025 (disagrees): FATHMM-MKL higher in 0, MetaLR higher in 7, 0 ties.2 other papers18 FATHMM-MKL higher182 tie5 MetaLR higher5Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MetaLR higher7
FATHMM-MKL higherTieMetaLR higher
Compared
FATHMM-MKL, MetaLR

Not reviewed yet.

phyloP and SIFT: An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks phyloP above SIFT in 3 of 4 comparisons; 2 of 2 other studies rank SIFT above phyloP.

3 papers rank SIFT above phyloP in 14 of their 16 comparisons and phyloP higher in 2. An openly licensed benchmark… ranks phyloP above SIFT in 3 of its 6 comparisons and SIFT higher in 1; 2 are ties.

Comparisons in which each paper scores SIFT or phyloP higher3 other papers: SIFT higher in 14, phyloP higher in 2, 0 ties. An openly licensed benchmark… (disagrees): SIFT higher in 1, phyloP higher in 3, 2 ties.3 other papers14 SIFT higher142 phyloP higher2Disagrees: An openly licensed benchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers1 SIFT higher12 tie23 phyloP higher3
SIFT higherTiephyloP higher
Compared
phyloP, SIFT

Not reviewed yet.

CADD and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above CADD in 10 of 10 comparisons; 2 of 2 other studies rank CADD above PrimateAI.

3 papers rank CADD above PrimateAI in 15 of their 18 comparisons and PrimateAI higher in 2; 1 is a tie. Comprehensive assessment… ranks PrimateAI above CADD in all 10 of its comparisons.

Comparisons in which each paper scores CADD or PrimateAI higher3 other papers: CADD higher in 15, PrimateAI higher in 2, 1 tie. Comprehensive assessment… (disagrees): CADD higher in 0, PrimateAI higher in 10, 0 ties.3 other papers15 CADD higher151 tie2 PrimateAI higher2Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations10 PrimateAI higher10
CADD higherTiePrimateAI higher
Compared
CADD, PrimateAI

Not reviewed yet.

FATHMM-XF and MetaLR: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above MetaLR.

2 papers rank FATHMM-XF above MetaLR in 22 of their 26 comparisons and MetaLR higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks MetaLR above FATHMM-XF in all 7 of its comparisons.

Comparisons in which each paper scores FATHMM-XF or MetaLR higher2 other papers: FATHMM-XF higher in 22, MetaLR higher in 3, 1 tie. Livesey and Marsh 2025 (disagrees): FATHMM-XF higher in 0, MetaLR higher in 7, 0 ties.2 other papers22 FATHMM-XF higher221 tie3 MetaLR higher3Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MetaLR higher7
FATHMM-XF higherTieMetaLR higher
Compared
FATHMM-XF, MetaLR

Not reviewed yet.

MetaSVM and MutationAssessor: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above MutationAssessor in 4 of 5 comparisons; 2 of 2 other studies rank MutationAssessor above MetaSVM.

3 papers rank MutationAssessor above MetaSVM in 22 of their 33 comparisons and MetaSVM higher in 10; 1 is a tie. Clinical likelihood ratios and balanced… ranks MetaSVM above MutationAssessor in 4 of its 6 comparisons and MutationAssessor higher in 1; 1 is a tie.

Comparisons in which each paper scores MutationAssessor or MetaSVM higher3 other papers: MutationAssessor higher in 22, MetaSVM higher in 10, 1 tie. Clinical likelihood ratios and balanced… (disagrees): MutationAssessor higher in 1, MetaSVM higher in 4, 1 tie.3 other papers22 MutationAssessor higher221 tie10 MetaSVM higher10Disagrees: Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 MutationAssessor higher11 tie14 MetaSVM higher4
MutationAssessor higherTieMetaSVM higher
Compared
MetaSVM, MutationAssessor

Not reviewed yet.

MutPred and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MutPred in 9 of 10 comparisons; 2 of 2 other studies rank MutPred above PrimateAI.

4 papers rank MutPred above PrimateAI in 17 of their 20 comparisons and PrimateAI higher in 3. Comprehensive assessment… ranks PrimateAI above MutPred in 9 of its 10 comparisons and MutPred higher in 1.

Comparisons in which each paper scores MutPred or PrimateAI higher4 other papers: MutPred higher in 17, PrimateAI higher in 3, 0 ties. Comprehensive assessment… (disagrees): MutPred higher in 1, PrimateAI higher in 9, 0 ties.4 other papers17 MutPred higher173 PrimateAI higher3Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations1 MutPred higher19 PrimateAI higher9
MutPred higherTiePrimateAI higher
Compared
MutPred, PrimateAI

Not reviewed yet.

FATHMM-XF and MetaSVM: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above MetaSVM.

2 papers rank FATHMM-XF above MetaSVM in 22 of their 26 comparisons and MetaSVM higher in 4. Livesey and Marsh 2025 ranks MetaSVM above FATHMM-XF in all 7 of its comparisons.

Comparisons in which each paper scores FATHMM-XF or MetaSVM higher2 other papers: FATHMM-XF higher in 22, MetaSVM higher in 4, 0 ties. Livesey and Marsh 2025 (disagrees): FATHMM-XF higher in 0, MetaSVM higher in 7, 0 ties.2 other papers22 FATHMM-XF higher224 MetaSVM higher4Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MetaSVM higher7
FATHMM-XF higherTieMetaSVM higher
Compared
FATHMM-XF, MetaSVM

Not reviewed yet.

PrimateAI and SIFT: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above PrimateAI in 6 of 6 comparisons; 2 of 2 other studies rank PrimateAI above SIFT.

4 papers rank PrimateAI above SIFT in 25 of their 32 comparisons and SIFT higher in 7. Of those papers, ProteinGym ranks SIFT higher on its own (1 of 1). Livesey and Marsh 2025 ranks SIFT above PrimateAI in all 6 comparisons that are not ties (1 is a tie).

Comparisons in which each paper scores PrimateAI or SIFT higher4 other papers: PrimateAI higher in 25, SIFT higher in 7, 0 ties. Livesey and Marsh 2025 (disagrees): PrimateAI higher in 0, SIFT higher in 6, 1 tie.4 other papers25 PrimateAI higher257 SIFT higher7Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 tie16 SIFT higher6
PrimateAI higherTieSIFT higher
Compared
PrimateAI, SIFT

Not reviewed yet.

REVEL and BayesDel: An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above REVEL in 6 of 6 comparisons; 2 of 2 other studies rank REVEL above BayesDel.

3 papers rank REVEL above BayesDel in 11 of their 14 comparisons and BayesDel higher in 2; 1 is a tie. Of those papers, ProteinGym ranks BayesDel higher on its own (1 of 1). An openly licensed benchmark… ranks BayesDel above REVEL in all 6 of its comparisons.

Comparisons in which each paper scores REVEL or BayesDel higher3 other papers: REVEL higher in 11, BayesDel higher in 2, 1 tie. An openly licensed benchmark… (disagrees): REVEL higher in 0, BayesDel higher in 6, 0 ties.3 other papers11 REVEL higher111 tie2 BayesDel higher2Disagrees: An openly licensed benchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers6 BayesDel higher6
REVEL higherTieBayesDel higher
Compared
REVEL, BayesDel

Not reviewed yet.

LRT and MetaSVM: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaSVM in 16 of 20 comparisons; 2 of 2 other studies rank MetaSVM above LRT.

2 papers rank MetaSVM above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaSVM in 16 of its 20 comparisons and MetaSVM higher in 4.

Comparisons in which each paper scores MetaSVM or LRT higher2 other papers: MetaSVM higher in 12, LRT higher in 1, 0 ties. Comprehensive assessment… (disagrees): MetaSVM higher in 4, LRT higher in 16, 0 ties.2 other papers12 MetaSVM higher121 LRT higherDisagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations4 MetaSVM higher416 LRT higher16
MetaSVM higherTieLRT higher
Compared
LRT, MetaSVM

Not reviewed yet.

MutPred and ESM-1b: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutPred in 6 of 7 comparisons; 2 of 2 other studies rank MutPred above ESM-1b.

3 papers rank MutPred above ESM-1b in 8 of their 11 comparisons and ESM-1b higher in 1; 2 are ties. Of those papers, ProteinGym ranks ESM-1b higher on its own (1 of 1). Livesey and Marsh 2025 ranks ESM-1b above MutPred in 6 of its 7 comparisons and MutPred higher in 1.

Comparisons in which each paper scores MutPred or ESM-1b higher3 other papers: MutPred higher in 8, ESM-1b higher in 1, 2 ties. Livesey and Marsh 2025 (disagrees): MutPred higher in 1, ESM-1b higher in 6, 0 ties.3 other papers8 MutPred higher82 tie21 ESM-1b higher1Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 MutPred higher16 ESM-1b higher6
MutPred higherTieESM-1b higher
Compared
MutPred, ESM-1b

Not reviewed yet.

MPC and MutationAssessor: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above MPC in 6 of 7 comparisons; 2 of 2 other studies rank MPC above MutationAssessor.

2 papers rank MPC above MutationAssessor in all 16 of their comparisons. Livesey and Marsh 2025 ranks MutationAssessor above MPC in 6 of its 7 comparisons and MPC higher in 1.

Comparisons in which each paper scores MPC or MutationAssessor higher2 other papers: MPC higher in 16, MutationAssessor higher in 0, 0 ties. Livesey and Marsh 2025 (disagrees): MPC higher in 1, MutationAssessor higher in 6, 0 ties.2 other papers16 MPC higher16Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 MPC higher16 MutationAssessor higher6
MPC higherTieMutationAssessor higher
Compared
MPC, MutationAssessor

Not reviewed yet.

MetaSVM and PrimateAI: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above PrimateAI in 7 of 7 comparisons; 2 of 2 other studies rank PrimateAI above MetaSVM.

3 papers rank PrimateAI above MetaSVM in 25 of their 32 comparisons and MetaSVM higher in 5; 2 are ties. Of those papers, Clinical likelihood ratios and balanced… ranks MetaSVM higher on its own (4 of 6). Livesey and Marsh 2025 ranks MetaSVM above PrimateAI in all 7 of its comparisons.

Comparisons in which each paper scores PrimateAI or MetaSVM higher3 other papers: PrimateAI higher in 25, MetaSVM higher in 5, 2 ties. Livesey and Marsh 2025 (disagrees): PrimateAI higher in 0, MetaSVM higher in 7, 0 ties.3 other papers25 PrimateAI higher252 tie5 MetaSVM higher5Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MetaSVM higher7
PrimateAI higherTieMetaSVM higher
Compared
MetaSVM, PrimateAI

Not reviewed yet.

LRT and SIFT: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above SIFT in 18 of 20 comparisons; 2 of 2 other studies rank SIFT above LRT.

4 papers rank SIFT above LRT in 14 of their 17 comparisons and LRT higher in 2; 1 is a tie. Comprehensive assessment… ranks LRT above SIFT in 18 of its 20 comparisons and SIFT higher in 2.

Comparisons in which each paper scores SIFT or LRT higher4 other papers: SIFT higher in 14, LRT higher in 2, 1 tie. Comprehensive assessment… (disagrees): SIFT higher in 2, LRT higher in 18, 0 ties.4 other papers14 SIFT higher141 tie2 LRT higher2Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations2 SIFT higher218 LRT higher18
SIFT higherTieLRT higher
Compared
LRT, SIFT

Not reviewed yet.

MVP and PolyPhen-2: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above MVP in 6 of 6 comparisons; 3 of 3 other studies rank MVP above PolyPhen-2.

3 papers rank MVP above PolyPhen-2 in 18 of their 21 comparisons and PolyPhen-2 higher in 2; 1 is a tie. Livesey and Marsh 2025 ranks PolyPhen-2 above MVP in all 6 of its comparisons.

Comparisons in which each paper scores MVP or PolyPhen-2 higher3 other papers: MVP higher in 18, PolyPhen-2 higher in 2, 1 tie. Livesey and Marsh 2025 (disagrees): MVP higher in 0, PolyPhen-2 higher in 6, 0 ties.3 other papers18 MVP higher181 tie2 PolyPhen-2 higher2Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)6 PolyPhen-2 higher6
MVP higherTiePolyPhen-2 higher
Compared
MVP, PolyPhen-2

Not reviewed yet.

DANN and MetaLR: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks DANN above MetaLR in 10 of 10 comparisons; 2 of 2 other studies rank MetaLR above DANN.

3 papers rank MetaLR above DANN in 15 of their 19 comparisons and DANN higher in 3; 1 is a tie. Comprehensive assessment… ranks DANN above MetaLR in all 10 of its comparisons.

Comparisons in which each paper scores MetaLR or DANN higher3 other papers: MetaLR higher in 15, DANN higher in 3, 1 tie. Comprehensive assessment… (disagrees): MetaLR higher in 0, DANN higher in 10, 0 ties.3 other papers15 MetaLR higher151 tie3 DANN higher3Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations10 DANN higher10
MetaLR higherTieDANN higher
Compared
DANN, MetaLR

Not reviewed yet.

GenoCanyon and phyloP: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks GenoCanyon above phyloP in 3 of 3 comparisons; 2 of 2 other studies rank phyloP above GenoCanyon.

2 papers rank phyloP above GenoCanyon in 11 of their 13 comparisons and GenoCanyon higher in 1; 1 is a tie. Brandes et al. 2023 ranks GenoCanyon above phyloP in all 3 comparisons that are not ties (1 is a tie).

Comparisons in which each paper scores phyloP or GenoCanyon higher2 other papers: phyloP higher in 11, GenoCanyon higher in 1, 1 tie. Brandes et al. 2023 (disagrees): phyloP higher in 0, GenoCanyon higher in 3, 1 tie.2 other papers11 phyloP higher111 tie1 GenoCanyon higherDisagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods1 tie13 GenoCanyon higher3
phyloP higherTieGenoCanyon higher
Compared
GenoCanyon, phyloP

Not reviewed yet.

VEST4 and ESM-1b: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above VEST4 in 6 of 7 comparisons; 2 of 2 other studies rank VEST4 above ESM-1b.

2 papers rank VEST4 above ESM-1b in 7 of their 10 comparisons and ESM-1b higher in 1; 2 are ties. Livesey and Marsh 2025 ranks ESM-1b above VEST4 in 6 of its 7 comparisons and VEST4 higher in 1.

Comparisons in which each paper scores VEST4 or ESM-1b higher2 other papers: VEST4 higher in 7, ESM-1b higher in 1, 2 ties. Livesey and Marsh 2025 (disagrees): VEST4 higher in 1, ESM-1b higher in 6, 0 ties.2 other papers7 VEST4 higher72 tie21 ESM-1b higher1Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 VEST4 higher16 ESM-1b higher6
VEST4 higherTieESM-1b higher
Compared
VEST4, ESM-1b

Not reviewed yet.

MutationAssessor and PROVEAN: An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PROVEAN above MutationAssessor in 6 of 6 comparisons; 2 of 2 other studies rank MutationAssessor above PROVEAN.

5 papers rank MutationAssessor above PROVEAN in 21 of their 38 comparisons and PROVEAN higher in 15; 2 are ties. Of those papers, ProteinGym ranks PROVEAN higher on its own (1 of 1). An openly licensed benchmark… ranks PROVEAN above MutationAssessor in all 6 of its comparisons.

Comparisons in which each paper scores MutationAssessor or PROVEAN higher5 other papers: MutationAssessor higher in 21, PROVEAN higher in 15, 2 ties. An openly licensed benchmark… (disagrees): MutationAssessor higher in 0, PROVEAN higher in 6, 0 ties.5 other papers21 MutationAssessor higher212 tie15 PROVEAN higher15Disagrees: An openly licensed benchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers6 PROVEAN higher6
MutationAssessor higherTiePROVEAN higher
Compared
MutationAssessor, PROVEAN

Not reviewed yet.

FATHMM-XF and MutationAssessor: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above MutationAssessor.

2 papers rank FATHMM-XF above MutationAssessor in 21 of their 26 comparisons and MutationAssessor higher in 5. Livesey and Marsh 2025 ranks MutationAssessor above FATHMM-XF in all 7 of its comparisons.

Comparisons in which each paper scores FATHMM-XF or MutationAssessor higher2 other papers: FATHMM-XF higher in 21, MutationAssessor higher in 5, 0 ties. Livesey and Marsh 2025 (disagrees): FATHMM-XF higher in 0, MutationAssessor higher in 7, 0 ties.2 other papers21 FATHMM-XF higher215 MutationAssessor higher5Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MutationAssessor higher7
FATHMM-XF higherTieMutationAssessor higher
Compared
FATHMM-XF, MutationAssessor

Not reviewed yet.

GenoCanyon and MetaSVM: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks GenoCanyon above MetaSVM in 6 of 8 comparisons; 2 of 2 other studies rank MetaSVM above GenoCanyon.

2 papers rank MetaSVM above GenoCanyon in 12 of their 13 comparisons and GenoCanyon higher in 1. Comprehensive assessment… ranks GenoCanyon above MetaSVM in 6 of its 10 comparisons and MetaSVM higher in 2; 2 are ties.

Comparisons in which each paper scores MetaSVM or GenoCanyon higher2 other papers: MetaSVM higher in 12, GenoCanyon higher in 1, 0 ties. Comprehensive assessment… (disagrees): MetaSVM higher in 2, GenoCanyon higher in 6, 2 ties.2 other papers12 MetaSVM higher121 GenoCanyon higherDisagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations2 MetaSVM higher22 tie26 GenoCanyon higher6
MetaSVM higherTieGenoCanyon higher
Compared
GenoCanyon, MetaSVM

Not reviewed yet.

CADD and PolyPhen-2: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above CADD in 6 of 6 comparisons; 2 of 2 other studies rank CADD above PolyPhen-2.

4 papers rank CADD above PolyPhen-2 in 19 of their 22 comparisons and PolyPhen-2 higher in 2; 1 is a tie. Livesey and Marsh 2025 ranks PolyPhen-2 above CADD in all 6 of its comparisons.

Comparisons in which each paper scores CADD or PolyPhen-2 higher4 other papers: CADD higher in 19, PolyPhen-2 higher in 2, 1 tie. Livesey and Marsh 2025 (disagrees): CADD higher in 0, PolyPhen-2 higher in 6, 0 ties.4 other papers19 CADD higher191 tie2 PolyPhen-2 higher2Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)6 PolyPhen-2 higher6
CADD higherTiePolyPhen-2 higher
Compared
CADD, PolyPhen-2

Not reviewed yet.

M-CAP and MVP: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above MVP in 6 of 7 comparisons; 2 of 2 other studies rank MVP above M-CAP.

3 papers rank MVP above M-CAP in 16 of their 20 comparisons and M-CAP higher in 1; 3 are ties. Livesey and Marsh 2025 ranks M-CAP above MVP in 6 of its 7 comparisons and MVP higher in 1.

Comparisons in which each paper scores MVP or M-CAP higher3 other papers: MVP higher in 16, M-CAP higher in 1, 3 ties. Livesey and Marsh 2025 (disagrees): MVP higher in 1, M-CAP higher in 6, 0 ties.3 other papers16 MVP higher163 tie31 M-CAP higherDisagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 MVP higher16 M-CAP higher6
MVP higherTieM-CAP higher
Compared
M-CAP, MVP

Not reviewed yet.

MVP and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MVP in 10 of 10 comparisons; 2 of 2 other studies rank MVP above PrimateAI.

3 papers rank MVP above PrimateAI in 13 of their 18 comparisons and PrimateAI higher in 3; 2 are ties. Comprehensive assessment… ranks PrimateAI above MVP in all 10 of its comparisons.

Comparisons in which each paper scores MVP or PrimateAI higher3 other papers: MVP higher in 13, PrimateAI higher in 3, 2 ties. Comprehensive assessment… (disagrees): MVP higher in 0, PrimateAI higher in 10, 0 ties.3 other papers13 MVP higher132 tie23 PrimateAI higher3Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations10 PrimateAI higher10
MVP higherTiePrimateAI higher
Compared
MVP, PrimateAI

Not reviewed yet.

PolyPhen-2 and PrimateAI: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above PrimateAI in 6 of 6 comparisons; 2 of 2 other studies rank PrimateAI above PolyPhen-2.

3 papers rank PrimateAI above PolyPhen-2 in 23 of their 32 comparisons and PolyPhen-2 higher in 8; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks PolyPhen-2 higher on its own (4 of 6). Livesey and Marsh 2025 ranks PolyPhen-2 above PrimateAI in all 6 of its comparisons.

Comparisons in which each paper scores PrimateAI or PolyPhen-2 higher3 other papers: PrimateAI higher in 23, PolyPhen-2 higher in 8, 1 tie. Livesey and Marsh 2025 (disagrees): PrimateAI higher in 0, PolyPhen-2 higher in 6, 0 ties.3 other papers23 PrimateAI higher231 tie8 PolyPhen-2 higher8Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)6 PolyPhen-2 higher6
PrimateAI higherTiePolyPhen-2 higher
Compared
PolyPhen-2, PrimateAI

Not reviewed yet.

DANN and M-CAP: Comprehensive assessment of computational algorithms in predicting cancer driver mutations reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks DANN above M-CAP in 5 of 5 comparisons; 2 of 2 other studies rank M-CAP above DANN.

3 papers rank M-CAP above DANN in 13 of their 17 comparisons and DANN higher in 3; 1 is a tie. Comprehensive assessment… ranks DANN above M-CAP in all 5 comparisons that are not ties (5 are ties).

Comparisons in which each paper scores M-CAP or DANN higher3 other papers: M-CAP higher in 13, DANN higher in 3, 1 tie. Comprehensive assessment… (disagrees): M-CAP higher in 0, DANN higher in 5, 5 ties.3 other papers13 M-CAP higher131 tie3 DANN higher3Disagrees: Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations5 tie55 DANN higher5
M-CAP higherTieDANN higher
Compared
DANN, M-CAP

Not reviewed yet.

SiPhy and phyloP: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SiPhy above phyloP in 3 of 4 comparisons; 2 of 2 other studies rank phyloP above SiPhy.

2 papers rank phyloP above SiPhy in all 13 of their comparisons. Brandes et al. 2023 ranks SiPhy above phyloP in 3 of its 4 comparisons and phyloP higher in 1.

Comparisons in which each paper scores phyloP or SiPhy higher2 other papers: phyloP higher in 13, SiPhy higher in 0, 0 ties. Brandes et al. 2023 (disagrees): phyloP higher in 1, SiPhy higher in 3, 0 ties.2 other papers13 phyloP higher13Disagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods1 phyloP higher13 SiPhy higher3
phyloP higherTieSiPhy higher
Compared
SiPhy, phyloP

Not reviewed yet.

FATHMM-XF and SIFT: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above FATHMM-XF in 6 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above SIFT.

2 papers rank FATHMM-XF above SIFT in 25 of their 26 comparisons and SIFT higher in 1. Livesey and Marsh 2025 ranks SIFT above FATHMM-XF in 6 of its 7 comparisons and FATHMM-XF higher in 1.

Comparisons in which each paper scores FATHMM-XF or SIFT higher2 other papers: FATHMM-XF higher in 25, SIFT higher in 1, 0 ties. Livesey and Marsh 2025 (disagrees): FATHMM-XF higher in 1, SIFT higher in 6, 0 ties.2 other papers25 FATHMM-XF higher251 SIFT higherDisagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 FATHMM-XF higher16 SIFT higher6
FATHMM-XF higherTieSIFT higher
Compared
FATHMM-XF, SIFT

Not reviewed yet.

CADD and Eigen: An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above Eigen in 6 of 6 comparisons; 2 of 2 other studies rank Eigen above CADD.

3 papers rank Eigen above CADD in 19 of their 22 comparisons and CADD higher in 3. An openly licensed benchmark… ranks CADD above Eigen in all 6 of its comparisons.

Comparisons in which each paper scores Eigen or CADD higher3 other papers: Eigen higher in 19, CADD higher in 3, 0 ties. An openly licensed benchmark… (disagrees): Eigen higher in 0, CADD higher in 6, 0 ties.3 other papers19 Eigen higher193 CADD higher3Disagrees: An openly licensed benchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers6 CADD higher6
Eigen higherTieCADD higher
Compared
CADD, Eigen

Not reviewed yet.

FATHMM-XF and PolyPhen-2: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above FATHMM-XF in 6 of 6 comparisons; 2 of 2 other studies rank FATHMM-XF above PolyPhen-2.

2 papers rank FATHMM-XF above PolyPhen-2 in 20 of their 26 comparisons and PolyPhen-2 higher in 1; 5 are ties. Livesey and Marsh 2025 ranks PolyPhen-2 above FATHMM-XF in all 6 of its comparisons.

Comparisons in which each paper scores FATHMM-XF or PolyPhen-2 higher2 other papers: FATHMM-XF higher in 20, PolyPhen-2 higher in 1, 5 ties. Livesey and Marsh 2025 (disagrees): FATHMM-XF higher in 0, PolyPhen-2 higher in 6, 0 ties.2 other papers20 FATHMM-XF higher205 tie51 PolyPhen-2 higherDisagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)6 PolyPhen-2 higher6
FATHMM-XF higherTiePolyPhen-2 higher
Compared
FATHMM-XF, PolyPhen-2

Not reviewed yet.

SiPhy and GERP: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SiPhy above GERP in 4 of 4 comparisons; 2 of 2 other studies rank GERP above SiPhy.

2 papers rank GERP above SiPhy in 11 of their 13 comparisons and SiPhy higher in 1; 1 is a tie. Brandes et al. 2023 ranks SiPhy above GERP in all 4 of its comparisons.

Comparisons in which each paper scores GERP or SiPhy higher2 other papers: GERP higher in 11, SiPhy higher in 1, 1 tie. Brandes et al. 2023 (disagrees): GERP higher in 0, SiPhy higher in 4, 0 ties.2 other papers11 GERP higher111 tie1 SiPhy higherDisagrees: Brandes et al. 2023Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods4 SiPhy higher4
GERP higherTieSiPhy higher
Compared
SiPhy, GERP

Not reviewed yet.

MetaLR and MVP: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above MVP in 6 of 7 comparisons; 2 of 2 other studies rank MVP above MetaLR.

3 papers rank MVP above MetaLR in 18 of their 21 comparisons and MetaLR higher in 3. Livesey and Marsh 2025 ranks MetaLR above MVP in 6 of its 7 comparisons and MVP higher in 1.

Comparisons in which each paper scores MVP or MetaLR higher3 other papers: MVP higher in 18, MetaLR higher in 3, 0 ties. Livesey and Marsh 2025 (disagrees): MVP higher in 1, MetaLR higher in 6, 0 ties.3 other papers18 MVP higher183 MetaLR higher3Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)1 MVP higher16 MetaLR higher6
MVP higherTieMetaLR higher
Compared
MetaLR, MVP

Not reviewed yet.

PrimateAI and PROVEAN: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PrimateAI above PROVEAN in 5 of 6 comparisons; 2 of 2 other studies rank PROVEAN above PrimateAI.

4 papers rank PROVEAN above PrimateAI in 20 of their 34 comparisons and PrimateAI higher in 12; 2 are ties. Clinical likelihood ratios and balanced… ranks PrimateAI above PROVEAN in 5 of its 6 comparisons and PROVEAN higher in 1.

Comparisons in which each paper scores PROVEAN or PrimateAI higher4 other papers: PROVEAN higher in 20, PrimateAI higher in 12, 2 ties. Clinical likelihood ratios and balanced… (disagrees): PROVEAN higher in 1, PrimateAI higher in 5, 0 ties.4 other papers20 PROVEAN higher202 tie12 PrimateAI higher12Disagrees: Clinical likelihood ratios and balanced…Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes1 PROVEAN higher15 PrimateAI higher5
PROVEAN higherTiePrimateAI higher
Compared
PrimateAI, PROVEAN

Not reviewed yet.

MetaLR and MetaSVM: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above MetaLR in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above MetaSVM in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MetaLR above MetaSVM in 15 of their 39 comparisons and MetaSVM higher in 19; 5 are ties. Of those papers, Comprehensive assessment…, Clinical likelihood ratios and balanced… rank MetaSVM higher on their own (12 of 20, 4 of 6).

Comparisons in which each paper scores MetaLR or MetaSVM higher4 other papers: MetaLR higher in 15, MetaSVM higher in 19, 5 ties.4 other papers15 MetaLR higher155 tie519 MetaSVM higher19
MetaLR higherTieMetaSVM higher
Compared
MetaLR, MetaSVM

Not reviewed yet.

ESM-1v and ESM-2: Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) ranks ESM-2 above ESM-1v in 3 of 3 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above ESM-2 in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank ESM-1v above ESM-2 in 14 of their 31 comparisons and ESM-2 higher in 11; 6 are ties. Of those papers, Genomic heterogeneity inflates… ranks ESM-2 higher on its own (3 of 5).

Comparisons in which each paper scores ESM-1v or ESM-2 higher3 other papers: ESM-1v higher in 14, ESM-2 higher in 11, 6 ties.3 other papers14 ESM-1v higher146 tie611 ESM-2 higher11
ESM-1v higherTieESM-2 higher
Compared
ESM-1v, ESM-2

Not reviewed yet.

DEOGEN2 and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above DEOGEN2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks DEOGEN2 above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank DEOGEN2 above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores DEOGEN2 or MutFormer higher2 other papers: DEOGEN2 higher in 6, MutFormer higher in 7, 0 ties.2 other papers6 DEOGEN2 higher67 MutFormer higher7
DEOGEN2 higherTieMutFormer higher
Compared
DEOGEN2, MutFormer

Not reviewed yet.

CARP and VESPAl: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks VESPAl above CARP in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above VESPAl in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank CARP above VESPAl in 10 of their 25 comparisons and VESPAl higher in 15. Of those papers, ProteinGym ranks VESPAl higher on its own (15 of 19).

Comparisons in which each paper scores CARP or VESPAl higher2 other papers: CARP higher in 10, VESPAl higher in 15, 0 ties.2 other papers10 CARP higher1015 VESPAl higher15
CARP higherTieVESPAl higher
Compared
CARP, VESPAl

Not reviewed yet.

SuSPect and MetaLR: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaLR above SuSPect in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SuSPect above MetaLR in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank SuSPect above MetaLR in 7 of their 13 comparisons and MetaLR higher in 5; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks MetaLR higher on its own (4 of 6).

Comparisons in which each paper scores SuSPect or MetaLR higher2 other papers: SuSPect higher in 7, MetaLR higher in 5, 1 tie.2 other papers7 SuSPect higher71 tie5 MetaLR higher5
SuSPect higherTieMetaLR higher
Compared
SuSPect, MetaLR

Not reviewed yet.

MetaLR and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MetaLR in 19 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MetaLR above PrimateAI in 14 of their 39 comparisons and PrimateAI higher in 25. Of those papers, Comprehensive assessment…, An openly licensed benchmark… rank PrimateAI higher on their own (19 of 20, 4 of 6).

Comparisons in which each paper scores MetaLR or PrimateAI higher4 other papers: MetaLR higher in 14, PrimateAI higher in 25, 0 ties.4 other papers14 MetaLR higher1425 PrimateAI higher25
MetaLR higherTiePrimateAI higher
Compared
MetaLR, PrimateAI

Not reviewed yet.

MetaRNN and VARITY: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VARITY above MetaRNN in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MetaRNN above VARITY in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank MetaRNN above VARITY in 5 of their 14 comparisons and VARITY higher in 7; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks VARITY higher on its own (6 of 7).

Comparisons in which each paper scores MetaRNN or VARITY higher3 other papers: MetaRNN higher in 5, VARITY higher in 7, 2 ties.3 other papers5 MetaRNN higher52 tie27 VARITY higher7
MetaRNN higherTieVARITY higher
Compared
MetaRNN, VARITY

Not reviewed yet.

MPC and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MPC in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MPC above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MPC above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores MPC or MutFormer higher2 other papers: MPC higher in 6, MutFormer higher in 7, 0 ties.2 other papers6 MPC higher67 MutFormer higher7
MPC higherTieMutFormer higher
Compared
MPC, MutFormer

Not reviewed yet.

ESM-1b and Tranception: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ESM-1b in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-1b above Tranception in 11 of their 26 comparisons and Tranception higher in 15. Of those papers, ProteinGym ranks Tranception higher on its own (15 of 19).

Comparisons in which each paper scores ESM-1b or Tranception higher2 other papers: ESM-1b higher in 11, Tranception higher in 15, 0 ties.2 other papers11 ESM-1b higher1115 Tranception higher15
ESM-1b higherTieTranception higher
Compared
ESM-1b, Tranception

Not reviewed yet.

SNAP2 and FATHMM-MKL: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above SNAP2 in 3 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SNAP2 above FATHMM-MKL in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank SNAP2 above FATHMM-MKL in 8 of their 12 comparisons and FATHMM-MKL higher in 3; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks FATHMM-MKL higher on its own (3 of 5).

Comparisons in which each paper scores SNAP2 or FATHMM-MKL higher2 other papers: SNAP2 higher in 8, FATHMM-MKL higher in 3, 1 tie.2 other papers8 SNAP2 higher81 tie3 FATHMM-MKL higher3
SNAP2 higherTieFATHMM-MKL higher
Compared
SNAP2, FATHMM-MKL

Not reviewed yet.

MutFormer and SIFT: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above SIFT in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks SIFT above MutFormer in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above SIFT in 8 of their 13 comparisons and SIFT higher in 3; 2 are ties. Of those papers, An openly licensed benchmark… ranks SIFT higher on its own (3 of 6).

Comparisons in which each paper scores MutFormer or SIFT higher2 other papers: MutFormer higher in 8, SIFT higher in 3, 2 ties.2 other papers8 MutFormer higher82 tie23 SIFT higher3
MutFormer higherTieSIFT higher
Compared
MutFormer, SIFT

Not reviewed yet.

ESM-IF1 and ProteinMPNN: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and proteinbench primary benchmark evidence disagree

  • Low severity
  • Explained
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks ESM-IF1 above ProteinMPNN in 19 of 19 comparisons; proteinbench primary benchmark evidence ranks ProteinMPNN above ESM-IF1 in 10 of 10 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-IF1 above ProteinMPNN in 19 of their 29 comparisons and ProteinMPNN higher in 10. Of those papers, proteinbench primary benchmark evidence ranks ProteinMPNN higher on its own (10 of 10).

Comparisons in which each paper scores ESM-IF1 or ProteinMPNN higher2 other papers: ESM-IF1 higher in 19, ProteinMPNN higher in 10, 0 ties.2 other papers19 ESM-IF1 higher1910 ProteinMPNN higher10
ESM-IF1 higherTieProteinMPNN higher
Compared
ESM-IF1, ProteinMPNN

2 follow-up actions, latest 24 October 2026

CARP and VESPA: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks VESPA above CARP in 18 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above VESPA in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank CARP above VESPA in 7 of their 25 comparisons and VESPA higher in 18. Of those papers, ProteinGym ranks VESPA higher on its own (18 of 19).

Comparisons in which each paper scores CARP or VESPA higher2 other papers: CARP higher in 7, VESPA higher in 18, 0 ties.2 other papers7 CARP higher718 VESPA higher18
CARP higherTieVESPA higher
Compared
CARP, VESPA

Not reviewed yet.

Condel and MetaSVM: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above MetaSVM in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Condel above MetaSVM in 8 of their 13 comparisons and MetaSVM higher in 4; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks MetaSVM higher on its own (4 of 6).

Comparisons in which each paper scores Condel or MetaSVM higher2 other papers: Condel higher in 8, MetaSVM higher in 4, 1 tie.2 other papers8 Condel higher81 tie4 MetaSVM higher4
Condel higherTieMetaSVM higher
Compared
Condel, MetaSVM

Not reviewed yet.

ESM-2 and Tranception: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ESM-2 in 16 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-2 above Tranception in 9 of their 26 comparisons and Tranception higher in 16; 1 is a tie. Of those papers, ProteinGym ranks Tranception higher on its own (16 of 19).

Comparisons in which each paper scores ESM-2 or Tranception higher2 other papers: ESM-2 higher in 9, Tranception higher in 16, 1 tie.2 other papers9 ESM-2 higher91 tie16 Tranception higher16
ESM-2 higherTieTranception higher
Compared
ESM-2, Tranception

Not reviewed yet.

ESM-1v and DeepSequence: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks DeepSequence above ESM-1v in 13 of 17 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above DeepSequence in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-1v above DeepSequence in 10 of their 26 comparisons and DeepSequence higher in 14; 2 are ties. Of those papers, ProteinGym ranks DeepSequence higher on its own (13 of 19).

Comparisons in which each paper scores ESM-1v or DeepSequence higher2 other papers: ESM-1v higher in 10, DeepSequence higher in 14, 2 ties.2 other papers10 ESM-1v higher102 tie14 DeepSequence higher14
ESM-1v higherTieDeepSequence higher
Compared
ESM-1v, DeepSequence

Not reviewed yet.

Condel and SIFT: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks SIFT above Condel in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above SIFT in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Condel above SIFT in 7 of their 12 comparisons and SIFT higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks SIFT higher on its own (5 of 5).

Comparisons in which each paper scores Condel or SIFT higher2 other papers: Condel higher in 7, SIFT higher in 5, 0 ties.2 other papers7 Condel higher75 SIFT higher5
Condel higherTieSIFT higher
Compared
Condel, SIFT

Not reviewed yet.

LRT and PolyPhen-2: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above PolyPhen-2 in 14 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above LRT in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank LRT above PolyPhen-2 in 17 of their 32 comparisons and PolyPhen-2 higher in 13; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks PolyPhen-2 higher on its own (6 of 6).

Comparisons in which each paper scores LRT or PolyPhen-2 higher3 other papers: LRT higher in 17, PolyPhen-2 higher in 13, 2 ties.3 other papers17 LRT higher172 tie13 PolyPhen-2 higher13
LRT higherTiePolyPhen-2 higher
Compared
LRT, PolyPhen-2

Not reviewed yet.

CARP and ESM-1b: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks ESM-1b above CARP in 14 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above ESM-1b in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank CARP above ESM-1b in 10 of their 25 comparisons and ESM-1b higher in 14; 1 is a tie. Of those papers, ProteinGym ranks ESM-1b higher on its own (14 of 19).

Comparisons in which each paper scores CARP or ESM-1b higher2 other papers: CARP higher in 10, ESM-1b higher in 14, 1 tie.2 other papers10 CARP higher101 tie14 ESM-1b higher14
CARP higherTieESM-1b higher
Compared
CARP, ESM-1b

Not reviewed yet.

CARP and Tranception: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above CARP in 19 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above Tranception in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank CARP above Tranception in 6 of their 25 comparisons and Tranception higher in 19. Of those papers, ProteinGym ranks Tranception higher on its own (19 of 19).

Comparisons in which each paper scores CARP or Tranception higher2 other papers: CARP higher in 6, Tranception higher in 19, 0 ties.2 other papers6 CARP higher619 Tranception higher19
CARP higherTieTranception higher
Compared
CARP, Tranception

Not reviewed yet.

MetaRNN and MutScore: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaRNN above MutScore in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MetaRNN in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MetaRNN above MutScore in 8 of their 13 comparisons and MutScore higher in 3; 2 are ties. Of those papers, An openly licensed benchmark… ranks MutScore higher on its own (3 of 6).

Comparisons in which each paper scores MetaRNN or MutScore higher2 other papers: MetaRNN higher in 8, MutScore higher in 3, 2 ties.2 other papers8 MetaRNN higher82 tie23 MutScore higher3
MetaRNN higherTieMutScore higher
Compared
MetaRNN, MutScore

Not reviewed yet.

MutFormer and PolyPhen-2: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PolyPhen-2 in 6 of 6 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PolyPhen-2 above MutFormer in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above PolyPhen-2 in 7 of their 12 comparisons and PolyPhen-2 higher in 5. Of those papers, An openly licensed benchmark… ranks PolyPhen-2 higher on its own (5 of 6).

Comparisons in which each paper scores MutFormer or PolyPhen-2 higher2 other papers: MutFormer higher in 7, PolyPhen-2 higher in 5, 0 ties.2 other papers7 MutFormer higher75 PolyPhen-2 higher5
MutFormer higherTiePolyPhen-2 higher
Compared
MutFormer, PolyPhen-2

Not reviewed yet.

ClinPred and MetaRNN: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaRNN above ClinPred in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above MetaRNN in 2 of 2 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank ClinPred above MetaRNN in 2 of their 14 comparisons and MetaRNN higher in 7; 5 are ties. Of those papers, Livesey and Marsh 2025 ranks MetaRNN higher on its own (7 of 7).

Comparisons in which each paper scores ClinPred or MetaRNN higher3 other papers: ClinPred higher in 2, MetaRNN higher in 7, 5 ties.3 other papers2 ClinPred higher25 tie57 MetaRNN higher7
ClinPred higherTieMetaRNN higher
Compared
ClinPred, MetaRNN

Not reviewed yet.

LRT and phyloP: Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • Same task, different protocols

Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks LRT above phyloP in 4 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks phyloP above LRT in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank LRT above phyloP in 7 of their 17 comparisons and phyloP higher in 10. Of those papers, Livesey and Marsh 2025, Clinical likelihood ratios and balanced… rank phyloP higher on their own (6 of 7, 4 of 6).

Comparisons in which each paper scores LRT or phyloP higher3 other papers: LRT higher in 7, phyloP higher in 10, 0 ties.3 other papers7 LRT higher710 phyloP higher10
LRT higherTiephyloP higher
Compared
LRT, phyloP

Not reviewed yet.

MutScore and VARITY: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VARITY above MutScore in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above VARITY in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutScore above VARITY in 5 of their 13 comparisons and VARITY higher in 6; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks VARITY higher on its own (6 of 7).

Comparisons in which each paper scores MutScore or VARITY higher2 other papers: MutScore higher in 5, VARITY higher in 6, 2 ties.2 other papers5 MutScore higher52 tie26 VARITY higher6
MutScore higherTieVARITY higher
Compared
MutScore, VARITY

Not reviewed yet.

DeepSequence and VESPA: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks VESPA above DeepSequence in 13 of 17 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks DeepSequence above VESPA in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank DeepSequence above VESPA in 9 of their 26 comparisons and VESPA higher in 14; 3 are ties. Of those papers, ProteinGym ranks VESPA higher on its own (13 of 19).

Comparisons in which each paper scores DeepSequence or VESPA higher2 other papers: DeepSequence higher in 9, VESPA higher in 14, 3 ties.2 other papers9 DeepSequence higher93 tie314 VESPA higher14
DeepSequence higherTieVESPA higher
Compared
DeepSequence, VESPA

Not reviewed yet.

SuSPect and MetaSVM: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above SuSPect in 5 of 6 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SuSPect above MetaSVM in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank SuSPect above MetaSVM in 8 of their 13 comparisons and MetaSVM higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks MetaSVM higher on its own (5 of 6).

Comparisons in which each paper scores SuSPect or MetaSVM higher2 other papers: SuSPect higher in 8, MetaSVM higher in 5, 0 ties.2 other papers8 SuSPect higher85 MetaSVM higher5
SuSPect higherTieMetaSVM higher
Compared
SuSPect, MetaSVM

Not reviewed yet.

MutFormer and REVEL: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above REVEL in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks REVEL above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above REVEL in 6 of their 13 comparisons and REVEL higher in 7. Of those papers, An openly licensed benchmark… ranks REVEL higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or REVEL higher2 other papers: MutFormer higher in 6, REVEL higher in 7, 0 ties.2 other papers6 MutFormer higher67 REVEL higher7
MutFormer higherTieREVEL higher
Compared
MutFormer, REVEL

Not reviewed yet.

MutScore and PHACTboost: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PHACTboost above MutScore in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above PHACTboost in 5 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutScore above PHACTboost in 5 of their 13 comparisons and PHACTboost higher in 7; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks PHACTboost higher on its own (7 of 7).

Comparisons in which each paper scores MutScore or PHACTboost higher2 other papers: MutScore higher in 5, PHACTboost higher in 7, 1 tie.2 other papers5 MutScore higher51 tie7 PHACTboost higher7
MutScore higherTiePHACTboost higher
Compared
MutScore, PHACTboost

Not reviewed yet.

MutFormer and PROVEAN: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PROVEAN in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PROVEAN above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above PROVEAN in 7 of their 13 comparisons and PROVEAN higher in 6. Of those papers, An openly licensed benchmark… ranks PROVEAN higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or PROVEAN higher2 other papers: MutFormer higher in 7, PROVEAN higher in 6, 0 ties.2 other papers7 MutFormer higher76 PROVEAN higher6
MutFormer higherTiePROVEAN higher
Compared
MutFormer, PROVEAN

Not reviewed yet.

ESM-2 and CADD: Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks CADD above ESM-2 in 14 of 14 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above CADD in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-2 above CADD in 6 of their 21 comparisons and CADD higher in 15. Of those papers, Genome modelling and design across all… ranks CADD higher on its own (14 of 14).

Comparisons in which each paper scores ESM-2 or CADD higher2 other papers: ESM-2 higher in 6, CADD higher in 15, 0 ties.2 other papers6 ESM-2 higher615 CADD higher15
ESM-2 higherTieCADD higher
Compared
ESM-2, CADD

Not reviewed yet.

MutFormer and ESM-1b: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above ESM-1b in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ESM-1b above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above ESM-1b in 7 of their 13 comparisons and ESM-1b higher in 6. Of those papers, An openly licensed benchmark… ranks ESM-1b higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or ESM-1b higher2 other papers: MutFormer higher in 7, ESM-1b higher in 6, 0 ties.2 other papers7 MutFormer higher76 ESM-1b higher6
MutFormer higherTieESM-1b higher
Compared
MutFormer, ESM-1b

Not reviewed yet.

Eigen and PROVEAN: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Eigen above PROVEAN in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PROVEAN above Eigen in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank Eigen above PROVEAN in 16 of their 29 comparisons and PROVEAN higher in 12; 1 is a tie. Of those papers, An openly licensed benchmark… ranks PROVEAN higher on its own (4 of 6).

Comparisons in which each paper scores Eigen or PROVEAN higher4 other papers: Eigen higher in 16, PROVEAN higher in 12, 1 tie.4 other papers16 Eigen higher161 tie12 PROVEAN higher12
Eigen higherTiePROVEAN higher
Compared
Eigen, PROVEAN

Not reviewed yet.

M-CAP and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above M-CAP in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks M-CAP above MutFormer in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank M-CAP above MutFormer in 4 of their 13 comparisons and MutFormer higher in 8; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores M-CAP or MutFormer higher2 other papers: M-CAP higher in 4, MutFormer higher in 8, 1 tie.2 other papers4 M-CAP higher41 tie8 MutFormer higher8
M-CAP higherTieMutFormer higher
Compared
M-CAP, MutFormer

Not reviewed yet.

Condel and MVP: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above MVP in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Condel above MVP in 7 of their 12 comparisons and MVP higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks MVP higher on its own (4 of 5).

Comparisons in which each paper scores Condel or MVP higher2 other papers: Condel higher in 7, MVP higher in 5, 0 ties.2 other papers7 Condel higher75 MVP higher5
Condel higherTieMVP higher
Compared
Condel, MVP

Not reviewed yet.

FATHMM-XF and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above FATHMM-XF in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks FATHMM-XF above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank FATHMM-XF above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores FATHMM-XF or MutFormer higher2 other papers: FATHMM-XF higher in 6, MutFormer higher in 7, 0 ties.2 other papers6 FATHMM-XF higher67 MutFormer higher7
FATHMM-XF higherTieMutFormer higher
Compared
FATHMM-XF, MutFormer

Not reviewed yet.

Condel and PolyPhen-2: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PolyPhen-2 above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above PolyPhen-2 in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Condel above PolyPhen-2 in 7 of their 12 comparisons and PolyPhen-2 higher in 4; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks PolyPhen-2 higher on its own (4 of 6).

Comparisons in which each paper scores Condel or PolyPhen-2 higher2 other papers: Condel higher in 7, PolyPhen-2 higher in 4, 1 tie.2 other papers7 Condel higher71 tie4 PolyPhen-2 higher4
Condel higherTiePolyPhen-2 higher
Compared
Condel, PolyPhen-2

Not reviewed yet.

FATHMM and GenoCanyon: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks GenoCanyon above FATHMM in 8 of 10 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks FATHMM above GenoCanyon in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank FATHMM above GenoCanyon in 12 of their 23 comparisons and GenoCanyon higher in 11. Of those papers, Comprehensive assessment… ranks GenoCanyon higher on its own (8 of 10).

Comparisons in which each paper scores FATHMM or GenoCanyon higher3 other papers: FATHMM higher in 12, GenoCanyon higher in 11, 0 ties.3 other papers12 FATHMM higher1211 GenoCanyon higher11
FATHMM higherTieGenoCanyon higher
Compared
FATHMM, GenoCanyon

Not reviewed yet.

MutFormer and BayesDel: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above BayesDel in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above BayesDel in 6 of their 13 comparisons and BayesDel higher in 7. Of those papers, An openly licensed benchmark… ranks BayesDel higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or BayesDel higher2 other papers: MutFormer higher in 6, BayesDel higher in 7, 0 ties.2 other papers6 MutFormer higher67 BayesDel higher7
MutFormer higherTieBayesDel higher
Compared
MutFormer, BayesDel

Not reviewed yet.

CADD and PROVEAN: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PROVEAN above CADD in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above PROVEAN in 3 of 3 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank CADD above PROVEAN in 9 of their 28 comparisons and PROVEAN higher in 13; 6 are ties. Of those papers, Comprehensive assessment…, Livesey and Marsh 2025 rank PROVEAN higher on their own (5 of 10, 6 of 7).

Comparisons in which each paper scores CADD or PROVEAN higher4 other papers: CADD higher in 9, PROVEAN higher in 13, 6 ties.4 other papers9 CADD higher96 tie613 PROVEAN higher13
CADD higherTiePROVEAN higher
Compared
CADD, PROVEAN

Not reviewed yet.

PHACTboost and BayesDel: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PHACTboost above BayesDel in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above PHACTboost in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank PHACTboost above BayesDel in 8 of their 13 comparisons and BayesDel higher in 4; 1 is a tie. Of those papers, An openly licensed benchmark… ranks BayesDel higher on its own (4 of 6).

Comparisons in which each paper scores PHACTboost or BayesDel higher2 other papers: PHACTboost higher in 8, BayesDel higher in 4, 1 tie.2 other papers8 PHACTboost higher81 tie4 BayesDel higher4
PHACTboost higherTieBayesDel higher
Compared
PHACTboost, BayesDel

Not reviewed yet.

ClinPred and VARITY: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VARITY above ClinPred in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above VARITY in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank ClinPred above VARITY in 5 of their 14 comparisons and VARITY higher in 7; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks VARITY higher on its own (7 of 7).

Comparisons in which each paper scores ClinPred or VARITY higher3 other papers: ClinPred higher in 5, VARITY higher in 7, 2 ties.3 other papers5 ClinPred higher52 tie27 VARITY higher7
ClinPred higherTieVARITY higher
Compared
ClinPred, VARITY

Not reviewed yet.

FATHMM-MKL and M-CAP: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM-MKL above M-CAP in 16 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above FATHMM-MKL in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank FATHMM-MKL above M-CAP in 18 of their 31 comparisons and M-CAP higher in 12; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks M-CAP higher on its own (7 of 7).

Comparisons in which each paper scores FATHMM-MKL or M-CAP higher3 other papers: FATHMM-MKL higher in 18, M-CAP higher in 12, 1 tie.3 other papers18 FATHMM-MKL higher181 tie12 M-CAP higher12
FATHMM-MKL higherTieM-CAP higher
Compared
FATHMM-MKL, M-CAP

Not reviewed yet.

MutFormer and MVP: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MVP in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MVP above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above MVP in 7 of their 13 comparisons and MVP higher in 6. Of those papers, An openly licensed benchmark… ranks MVP higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or MVP higher2 other papers: MutFormer higher in 7, MVP higher in 6, 0 ties.2 other papers7 MutFormer higher76 MVP higher6
MutFormer higherTieMVP higher
Compared
MutFormer, MVP

Not reviewed yet.

Eigen and FATHMM-XF: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Eigen above FATHMM-XF in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks FATHMM-XF above Eigen in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank Eigen above FATHMM-XF in 13 of their 23 comparisons and FATHMM-XF higher in 9; 1 is a tie. Of those papers, An openly licensed benchmark… ranks FATHMM-XF higher on its own (4 of 6).

Comparisons in which each paper scores Eigen or FATHMM-XF higher3 other papers: Eigen higher in 13, FATHMM-XF higher in 9, 1 tie.3 other papers13 Eigen higher131 tie9 FATHMM-XF higher9
Eigen higherTieFATHMM-XF higher
Compared
Eigen, FATHMM-XF

Not reviewed yet.

ClinPred and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above ClinPred in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ClinPred above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores ClinPred or MutFormer higher2 other papers: ClinPred higher in 6, MutFormer higher in 7, 0 ties.2 other papers6 ClinPred higher67 MutFormer higher7
ClinPred higherTieMutFormer higher
Compared
ClinPred, MutFormer

Not reviewed yet.

ClinPred and PHACTboost: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PHACTboost above ClinPred in 6 of 6 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above PHACTboost in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ClinPred above PHACTboost in 6 of their 13 comparisons and PHACTboost higher in 6; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks PHACTboost higher on its own (6 of 7).

Comparisons in which each paper scores ClinPred or PHACTboost higher2 other papers: ClinPred higher in 6, PHACTboost higher in 6, 1 tie.2 other papers6 ClinPred higher61 tie6 PHACTboost higher6
ClinPred higherTiePHACTboost higher
Compared
ClinPred, PHACTboost

Not reviewed yet.

CADD and MutationAssessor: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above CADD in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above MutationAssessor in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank CADD above MutationAssessor in 15 of their 28 comparisons and MutationAssessor higher in 13. Of those papers, Livesey and Marsh 2025 ranks MutationAssessor higher on its own (6 of 7).

Comparisons in which each paper scores CADD or MutationAssessor higher4 other papers: CADD higher in 15, MutationAssessor higher in 13, 0 ties.4 other papers15 CADD higher1513 MutationAssessor higher13
CADD higherTieMutationAssessor higher
Compared
CADD, MutationAssessor

Not reviewed yet.

MutationAssessor and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutationAssessor in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutationAssessor above MutFormer in 5 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutationAssessor above MutFormer in 5 of their 13 comparisons and MutFormer higher in 7; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores MutationAssessor or MutFormer higher2 other papers: MutationAssessor higher in 5, MutFormer higher in 7, 1 tie.2 other papers5 MutationAssessor higher51 tie7 MutFormer higher7
MutationAssessor higherTieMutFormer higher
Compared
MutationAssessor, MutFormer

Not reviewed yet.

DEOGEN2 and MPC: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MPC above DEOGEN2 in 10 of 10 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks DEOGEN2 above MPC in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank DEOGEN2 above MPC in 10 of their 24 comparisons and MPC higher in 13; 1 is a tie. Of those papers, Comprehensive assessment… ranks MPC higher on its own (10 of 10).

Comparisons in which each paper scores DEOGEN2 or MPC higher4 other papers: DEOGEN2 higher in 10, MPC higher in 13, 1 tie.4 other papers10 DEOGEN2 higher101 tie13 MPC higher13
DEOGEN2 higherTieMPC higher
Compared
DEOGEN2, MPC

Not reviewed yet.

Condel and FATHMM-MKL: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above Condel in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above FATHMM-MKL in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Condel above FATHMM-MKL in 7 of their 12 comparisons and FATHMM-MKL higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks FATHMM-MKL higher on its own (5 of 5).

Comparisons in which each paper scores Condel or FATHMM-MKL higher2 other papers: Condel higher in 7, FATHMM-MKL higher in 5, 0 ties.2 other papers7 Condel higher75 FATHMM-MKL higher5
Condel higherTieFATHMM-MKL higher
Compared
Condel, FATHMM-MKL

Not reviewed yet.

MetaSVM and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MetaSVM in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MetaSVM above MutFormer in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MetaSVM above MutFormer in 4 of their 13 comparisons and MutFormer higher in 8; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores MetaSVM or MutFormer higher2 other papers: MetaSVM higher in 4, MutFormer higher in 8, 1 tie.2 other papers4 MetaSVM higher41 tie8 MutFormer higher8
MetaSVM higherTieMutFormer higher
Compared
MetaSVM, MutFormer

Not reviewed yet.

FATHMM and LRT: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above FATHMM in 18 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks FATHMM above LRT in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank FATHMM above LRT in 10 of their 33 comparisons and LRT higher in 23. Of those papers, Comprehensive assessment…, Clinical likelihood ratios and balanced… rank LRT higher on their own (18 of 20, 4 of 6).

Comparisons in which each paper scores FATHMM or LRT higher3 other papers: FATHMM higher in 10, LRT higher in 23, 0 ties.3 other papers10 FATHMM higher1023 LRT higher23
FATHMM higherTieLRT higher
Compared
FATHMM, LRT

Not reviewed yet.

LRT and M-CAP: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above M-CAP in 16 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above LRT in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank LRT above M-CAP in 18 of their 31 comparisons and M-CAP higher in 13. Of those papers, Livesey and Marsh 2025 ranks M-CAP higher on its own (7 of 7).

Comparisons in which each paper scores LRT or M-CAP higher3 other papers: LRT higher in 18, M-CAP higher in 13, 0 ties.3 other papers18 LRT higher1813 M-CAP higher13
LRT higherTieM-CAP higher
Compared
LRT, M-CAP

Not reviewed yet.

MutFormer and MutPred: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutPred in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutPred above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above MutPred in 6 of their 13 comparisons and MutPred higher in 7. Of those papers, An openly licensed benchmark… ranks MutPred higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or MutPred higher2 other papers: MutFormer higher in 6, MutPred higher in 7, 0 ties.2 other papers6 MutFormer higher67 MutPred higher7
MutFormer higherTieMutPred higher
Compared
MutFormer, MutPred

Not reviewed yet.

FATHMM-XF and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above FATHMM-XF in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks FATHMM-XF above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank FATHMM-XF above PrimateAI in 14 of their 33 comparisons and PrimateAI higher in 17; 2 are ties. Of those papers, Comprehensive assessment… ranks PrimateAI higher on its own (15 of 20).

Comparisons in which each paper scores FATHMM-XF or PrimateAI higher3 other papers: FATHMM-XF higher in 14, PrimateAI higher in 17, 2 ties.3 other papers14 FATHMM-XF higher142 tie17 PrimateAI higher17
FATHMM-XF higherTiePrimateAI higher
Compared
FATHMM-XF, PrimateAI

Not reviewed yet.

MutPred and VEST4: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VEST4 above MutPred in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutPred above VEST4 in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MutPred above VEST4 in 14 of their 29 comparisons and VEST4 higher in 14; 1 is a tie. Of those papers, Livesey and Marsh 2025, Clinical likelihood ratios and balanced… rank VEST4 higher on their own (6 of 7, 4 of 6).

Comparisons in which each paper scores MutPred or VEST4 higher4 other papers: MutPred higher in 14, VEST4 higher in 14, 1 tie.4 other papers14 MutPred higher141 tie14 VEST4 higher14
MutPred higherTieVEST4 higher
Compared
MutPred, VEST4

Not reviewed yet.

MutFormer and phyloP: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above phyloP in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks phyloP above MutFormer in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above phyloP in 8 of their 13 comparisons and phyloP higher in 4; 1 is a tie. Of those papers, An openly licensed benchmark… ranks phyloP higher on its own (4 of 6).

Comparisons in which each paper scores MutFormer or phyloP higher2 other papers: MutFormer higher in 8, phyloP higher in 4, 1 tie.2 other papers8 MutFormer higher81 tie4 phyloP higher4
MutFormer higherTiephyloP higher
Compared
MutFormer, phyloP

Not reviewed yet.

LRT and MutPred: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks LRT above MutPred in 5 of 6 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutPred above LRT in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank LRT above MutPred in 12 of their 24 comparisons and MutPred higher in 12. Of those papers, Livesey and Marsh 2025, ProteinGym rank MutPred higher on their own (7 of 7, 1 of 1).

Comparisons in which each paper scores LRT or MutPred higher4 other papers: LRT higher in 12, MutPred higher in 12, 0 ties.4 other papers12 LRT higher1212 MutPred higher12
LRT higherTieMutPred higher
Compared
LRT, MutPred

Not reviewed yet.

FATHMM-MKL and BayesDel: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above BayesDel in 3 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks BayesDel above FATHMM-MKL in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank FATHMM-MKL above BayesDel in 3 of their 12 comparisons and BayesDel higher in 8; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks BayesDel higher on its own (7 of 7).

Comparisons in which each paper scores FATHMM-MKL or BayesDel higher2 other papers: FATHMM-MKL higher in 3, BayesDel higher in 8, 1 tie.2 other papers3 FATHMM-MKL higher31 tie8 BayesDel higher8
FATHMM-MKL higherTieBayesDel higher
Compared
FATHMM-MKL, BayesDel

Not reviewed yet.

MutScore and REVEL: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MutScore in 6 of 6 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above REVEL in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutScore above REVEL in 6 of their 13 comparisons and REVEL higher in 6; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks REVEL higher on its own (6 of 7).

Comparisons in which each paper scores MutScore or REVEL higher2 other papers: MutScore higher in 6, REVEL higher in 6, 1 tie.2 other papers6 MutScore higher61 tie6 REVEL higher6
MutScore higherTieREVEL higher
Compared
MutScore, REVEL

Not reviewed yet.

MutFormer and PrimateAI: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PrimateAI in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PrimateAI above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above PrimateAI in 7 of their 13 comparisons and PrimateAI higher in 6. Of those papers, An openly licensed benchmark… ranks PrimateAI higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or PrimateAI higher2 other papers: MutFormer higher in 7, PrimateAI higher in 6, 0 ties.2 other papers7 MutFormer higher76 PrimateAI higher6
MutFormer higherTiePrimateAI higher
Compared
MutFormer, PrimateAI

Not reviewed yet.

ESM-1v and MSA Transformer: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks MSA Transformer above ESM-1v in 16 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above MSA Transformer in 5 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-1v above MSA Transformer in 8 of their 25 comparisons and MSA Transformer higher in 16; 1 is a tie. Of those papers, ProteinGym ranks MSA Transformer higher on its own (16 of 19).

Comparisons in which each paper scores ESM-1v or MSA Transformer higher2 other papers: ESM-1v higher in 8, MSA Transformer higher in 16, 1 tie.2 other papers8 ESM-1v higher81 tie16 MSA Transformer higher16
ESM-1v higherTieMSA Transformer higher
Compared
ESM-1v, MSA Transformer

Not reviewed yet.

MPC and REVEL: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MPC above REVEL in 10 of 10 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MPC in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MPC above REVEL in 12 of their 24 comparisons and REVEL higher in 10; 2 are ties. Of those papers, Livesey and Marsh 2025, ProteinGym rank REVEL higher on their own (7 of 7, 1 of 1).

Comparisons in which each paper scores MPC or REVEL higher4 other papers: MPC higher in 12, REVEL higher in 10, 2 ties.4 other papers12 MPC higher122 tie10 REVEL higher10
MPC higherTieREVEL higher
Compared
MPC, REVEL

Not reviewed yet.

VEST4 and BayesDel: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VEST4 above BayesDel in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above VEST4 in 5 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank VEST4 above BayesDel in 8 of their 20 comparisons and BayesDel higher in 10; 2 are ties. Of those papers, An openly licensed benchmark…, ProteinGym rank BayesDel higher on their own (5 of 6, 1 of 1).

Comparisons in which each paper scores VEST4 or BayesDel higher4 other papers: VEST4 higher in 8, BayesDel higher in 10, 2 ties.4 other papers8 VEST4 higher82 tie210 BayesDel higher10
VEST4 higherTieBayesDel higher
Compared
VEST4, BayesDel

Not reviewed yet.

EVmutation and Tranception: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above EVmutation in 15 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks EVmutation above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank EVmutation above Tranception in 10 of their 26 comparisons and Tranception higher in 15; 1 is a tie. Of those papers, ProteinGym ranks Tranception higher on its own (15 of 19).

Comparisons in which each paper scores EVmutation or Tranception higher2 other papers: EVmutation higher in 10, Tranception higher in 15, 1 tie.2 other papers10 EVmutation higher101 tie15 Tranception higher15
EVmutation higherTieTranception higher
Compared
EVmutation, Tranception

Not reviewed yet.

Tranception and VESPAl: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above VESPAl in 17 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VESPAl above Tranception in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Tranception above VESPAl in 18 of their 26 comparisons and VESPAl higher in 7; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks VESPAl higher on its own (6 of 7).

Comparisons in which each paper scores Tranception or VESPAl higher2 other papers: Tranception higher in 18, VESPAl higher in 7, 1 tie.2 other papers18 Tranception higher181 tie7 VESPAl higher7
Tranception higherTieVESPAl higher
Compared
Tranception, VESPAl

Not reviewed yet.

ESM-1v and ESM-1b: Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) ranks ESM-1b above ESM-1v in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above ESM-1b in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank ESM-1v above ESM-1b in 18 of their 31 comparisons and ESM-1b higher in 12; 1 is a tie. Of those papers, Genomic heterogeneity inflates… ranks ESM-1b higher on its own (5 of 5).

Comparisons in which each paper scores ESM-1v or ESM-1b higher3 other papers: ESM-1v higher in 18, ESM-1b higher in 12, 1 tie.3 other papers18 ESM-1v higher181 tie12 ESM-1b higher12
ESM-1v higherTieESM-1b higher
Compared
ESM-1v, ESM-1b

Not reviewed yet.

ESM-1v and Tranception: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ESM-1v in 14 of 17 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM-1v above Tranception in 10 of their 26 comparisons and Tranception higher in 14; 2 are ties. Of those papers, ProteinGym ranks Tranception higher on its own (14 of 19).

Comparisons in which each paper scores ESM-1v or Tranception higher2 other papers: ESM-1v higher in 10, Tranception higher in 14, 2 ties.2 other papers10 ESM-1v higher102 tie14 Tranception higher14
ESM-1v higherTieTranception higher
Compared
ESM-1v, Tranception

Not reviewed yet.

ESM-2 and ESM-1b: Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks ESM-1b above ESM-2 in 26 of 28 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above ESM-1b in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

5 papers rank ESM-2 above ESM-1b in 31 of their 87 comparisons and ESM-1b higher in 47; 9 are ties. Of those papers, Genome modelling and design across all… ranks ESM-1b higher on its own (26 of 28).

Comparisons in which each paper scores ESM-2 or ESM-1b higher5 other papers: ESM-2 higher in 31, ESM-1b higher in 47, 9 ties.5 other papers31 ESM-2 higher319 tie947 ESM-1b higher47
ESM-2 higherTieESM-1b higher
Compared
ESM-2, ESM-1b

Not reviewed yet.

Eigen and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above Eigen in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks Eigen above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Eigen above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores Eigen or MutFormer higher2 other papers: Eigen higher in 6, MutFormer higher in 7, 0 ties.2 other papers6 Eigen higher67 MutFormer higher7
Eigen higherTieMutFormer higher
Compared
Eigen, MutFormer

Not reviewed yet.

ProGen2 and Tranception: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ProGen2 in 19 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ProGen2 above Tranception in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ProGen2 above Tranception in 6 of their 26 comparisons and Tranception higher in 19; 1 is a tie. Of those papers, ProteinGym ranks Tranception higher on its own (19 of 20).

Comparisons in which each paper scores ProGen2 or Tranception higher2 other papers: ProGen2 higher in 6, Tranception higher in 19, 1 tie.2 other papers6 ProGen2 higher61 tie19 Tranception higher19
ProGen2 higherTieTranception higher
Compared
ProGen2, Tranception

Not reviewed yet.

EVmutation and CARP: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks EVmutation above CARP in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above EVmutation in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank EVmutation above CARP in 16 of their 25 comparisons and CARP higher in 9. Of those papers, Livesey and Marsh 2025 ranks CARP higher on its own (5 of 6).

Comparisons in which each paper scores EVmutation or CARP higher2 other papers: EVmutation higher in 16, CARP higher in 9, 0 ties.2 other papers16 EVmutation higher169 CARP higher9
EVmutation higherTieCARP higher
Compared
EVmutation, CARP

Not reviewed yet.

MutFormer and VEST4: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above VEST4 in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks VEST4 above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above VEST4 in 6 of their 13 comparisons and VEST4 higher in 7. Of those papers, An openly licensed benchmark… ranks VEST4 higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or VEST4 higher2 other papers: MutFormer higher in 6, VEST4 higher in 7, 0 ties.2 other papers6 MutFormer higher67 VEST4 higher7
MutFormer higherTieVEST4 higher
Compared
MutFormer, VEST4

Not reviewed yet.

MutationAssessor and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MutationAssessor in 19 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

5 papers rank MutationAssessor above PrimateAI in 15 of their 40 comparisons and PrimateAI higher in 24; 1 is a tie. Of those papers, Comprehensive assessment… ranks PrimateAI higher on its own (19 of 20).

Comparisons in which each paper scores MutationAssessor or PrimateAI higher5 other papers: MutationAssessor higher in 15, PrimateAI higher in 24, 1 tie.5 other papers15 MutationAssessor higher151 tie24 PrimateAI higher24
MutationAssessor higherTiePrimateAI higher
Compared
MutationAssessor, PrimateAI

Not reviewed yet.

LIST-S2 and MetaLR: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above LIST-S2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above MetaLR in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank LIST-S2 above MetaLR in 4 of their 13 comparisons and MetaLR higher in 8; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks MetaLR higher on its own (7 of 7).

Comparisons in which each paper scores LIST-S2 or MetaLR higher2 other papers: LIST-S2 higher in 4, MetaLR higher in 8, 1 tie.2 other papers4 LIST-S2 higher41 tie8 MetaLR higher8
LIST-S2 higherTieMetaLR higher
Compared
LIST-S2, MetaLR

Not reviewed yet.

ESM3 and ProteinMPNN: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and proteinbench primary benchmark evidence disagree

  • Low severity
  • Explained
  • Two-study conflict
  • different-task

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks ESM3 above ProteinMPNN in 18 of 19 comparisons; proteinbench primary benchmark evidence ranks ProteinMPNN above ESM3 in 10 of 10 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank ESM3 above ProteinMPNN in 18 of their 29 comparisons and ProteinMPNN higher in 11. Of those papers, proteinbench primary benchmark evidence ranks ProteinMPNN higher on its own (10 of 10).

Comparisons in which each paper scores ESM3 or ProteinMPNN higher2 other papers: ESM3 higher in 18, ProteinMPNN higher in 11, 0 ties.2 other papers18 ESM3 higher1811 ProteinMPNN higher11
ESM3 higherTieProteinMPNN higher
Compared
ESM3, ProteinMPNN

2 follow-up actions, latest 24 October 2026

Eigen and PrimateAI: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above Eigen in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Eigen above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank Eigen above PrimateAI in 13 of their 29 comparisons and PrimateAI higher in 9; 7 are ties. Of those papers, Comprehensive assessment… ranks PrimateAI higher on its own (5 of 10).

Comparisons in which each paper scores Eigen or PrimateAI higher4 other papers: Eigen higher in 13, PrimateAI higher in 9, 7 ties.4 other papers13 Eigen higher137 tie79 PrimateAI higher9
Eigen higherTiePrimateAI higher
Compared
Eigen, PrimateAI

Not reviewed yet.

MutScore and ESM-1b: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutScore in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above ESM-1b in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutScore above ESM-1b in 7 of their 13 comparisons and ESM-1b higher in 6. Of those papers, Livesey and Marsh 2025 ranks ESM-1b higher on its own (6 of 7).

Comparisons in which each paper scores MutScore or ESM-1b higher2 other papers: MutScore higher in 7, ESM-1b higher in 6, 0 ties.2 other papers7 MutScore higher76 ESM-1b higher6
MutScore higherTieESM-1b higher
Compared
MutScore, ESM-1b

Not reviewed yet.

MutPred and REVEL: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MutPred in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutPred above REVEL in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MutPred above REVEL in 15 of their 29 comparisons and REVEL higher in 14. Of those papers, Livesey and Marsh 2025, Clinical likelihood ratios and balanced… rank REVEL higher on their own (6 of 7, 4 of 6).

Comparisons in which each paper scores MutPred or REVEL higher4 other papers: MutPred higher in 15, REVEL higher in 14, 0 ties.4 other papers15 MutPred higher1514 REVEL higher14
MutPred higherTieREVEL higher
Compared
MutPred, REVEL

Not reviewed yet.

MetaLR and PolyPhen-2: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PolyPhen-2 above MetaLR in 17 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above PolyPhen-2 in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MetaLR above PolyPhen-2 in 13 of their 38 comparisons and PolyPhen-2 higher in 22; 3 are ties. Of those papers, Comprehensive assessment… ranks PolyPhen-2 higher on its own (17 of 20).

Comparisons in which each paper scores MetaLR or PolyPhen-2 higher4 other papers: MetaLR higher in 13, PolyPhen-2 higher in 22, 3 ties.4 other papers13 MetaLR higher133 tie22 PolyPhen-2 higher22
MetaLR higherTiePolyPhen-2 higher
Compared
MetaLR, PolyPhen-2

Not reviewed yet.

LIST-S2 and M-CAP: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above LIST-S2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above M-CAP in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank LIST-S2 above M-CAP in 5 of their 13 comparisons and M-CAP higher in 8. Of those papers, Livesey and Marsh 2025 ranks M-CAP higher on its own (7 of 7).

Comparisons in which each paper scores LIST-S2 or M-CAP higher2 other papers: LIST-S2 higher in 5, M-CAP higher in 8, 0 ties.2 other papers5 LIST-S2 higher58 M-CAP higher8
LIST-S2 higherTieM-CAP higher
Compared
LIST-S2, M-CAP

Not reviewed yet.

PANTHER and SIFT: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks SIFT above PANTHER in 3 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PANTHER above SIFT in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank PANTHER above SIFT in 6 of their 11 comparisons and SIFT higher in 4; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks SIFT higher on its own (3 of 5).

Comparisons in which each paper scores PANTHER or SIFT higher2 other papers: PANTHER higher in 6, SIFT higher in 4, 1 tie.2 other papers6 PANTHER higher61 tie14 SIFT higher4
PANTHER higherTieSIFT higher
Compared
PANTHER, SIFT

Not reviewed yet.

Condel and CADD: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks CADD above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above CADD in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Condel above CADD in 7 of their 12 comparisons and CADD higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks CADD higher on its own (4 of 5).

Comparisons in which each paper scores Condel or CADD higher2 other papers: Condel higher in 7, CADD higher in 5, 0 ties.2 other papers7 Condel higher75 CADD higher5
Condel higherTieCADD higher
Compared
Condel, CADD

Not reviewed yet.

MutFormer and MutScore: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutScore in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank MutFormer above MutScore in 6 of their 13 comparisons and MutScore higher in 7. Of those papers, An openly licensed benchmark… ranks MutScore higher on its own (6 of 6).

Comparisons in which each paper scores MutFormer or MutScore higher2 other papers: MutFormer higher in 6, MutScore higher in 7, 0 ties.2 other papers6 MutFormer higher67 MutScore higher7
MutFormer higherTieMutScore higher
Compared
MutFormer, MutScore

Not reviewed yet.

REVEL and VEST4: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above VEST4 in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks VEST4 above REVEL in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

5 papers rank REVEL above VEST4 in 14 of their 30 comparisons and VEST4 higher in 12; 4 are ties. Of those papers, An openly licensed benchmark… ranks VEST4 higher on its own (3 of 6).

Comparisons in which each paper scores REVEL or VEST4 higher5 other papers: REVEL higher in 14, VEST4 higher in 12, 4 ties.5 other papers14 REVEL higher144 tie412 VEST4 higher12
REVEL higherTieVEST4 higher
Compared
REVEL, VEST4

Not reviewed yet.

LIST-S2 and MetaSVM: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above LIST-S2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above MetaSVM in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank LIST-S2 above MetaSVM in 3 of their 13 comparisons and MetaSVM higher in 8; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks MetaSVM higher on its own (7 of 7).

Comparisons in which each paper scores LIST-S2 or MetaSVM higher2 other papers: LIST-S2 higher in 3, MetaSVM higher in 8, 2 ties.2 other papers3 LIST-S2 higher32 tie28 MetaSVM higher8
LIST-S2 higherTieMetaSVM higher
Compared
LIST-S2, MetaSVM

Not reviewed yet.

MutationAssessor and MVP: Comprehensive assessment of computational algorithms in predicting cancer driver mutations and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MVP above MutationAssessor in 10 of 10 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above MVP in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.

4 papers rank MutationAssessor above MVP in 11 of their 28 comparisons and MVP higher in 17. Of those papers, Comprehensive assessment…, An openly licensed benchmark… rank MVP higher on their own (10 of 10, 4 of 6).

Comparisons in which each paper scores MutationAssessor or MVP higher4 other papers: MutationAssessor higher in 11, MVP higher in 17, 0 ties.4 other papers11 MutationAssessor higher1117 MVP higher17
MutationAssessor higherTieMVP higher
Compared
MutationAssessor, MVP

Not reviewed yet.

CADD and MutFormer: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • unknown

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above CADD in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank CADD above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).

Comparisons in which each paper scores CADD or MutFormer higher2 other papers: CADD higher in 6, MutFormer higher in 7, 0 ties.2 other papers6 CADD higher67 MutFormer higher7
CADD higherTieMutFormer higher
Compared
CADD, MutFormer

Not reviewed yet.