Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above Tranception in 7 of 7 comparisons; other studies rank Tranception above ESM-2 in 14 of 17.
ProteinGym AMFR assay-level Spearman… ranks Tranception above ESM-2 in 14 of its 19 comparisons and ESM-2 higher in 3; 2 are ties. Livesey and Marsh 2025 ranks ESM-2 above Tranception in all 7 of its comparisons.
Evaluation of phenotype-driven gene prioritization methods for Mendelian diseases ranks AMELIE above PhenIX in 11 of 14 comparisons; other studies rank PhenIX above AMELIE in 12 of 14.
Refined preferences of prioritizers… ranks PhenIX above AMELIE in 12 of its 14 comparisons and AMELIE higher in 2. Evaluation of phenotype-driven gene… ranks AMELIE above PhenIX in 11 of its 14 comparisons and PhenIX higher in 3.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above BayesDel in 6 of 7 comparisons; other studies rank BayesDel above MutFormer in 6 of 6.
An openly licensed benchmark… ranks BayesDel above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above BayesDel in 6 of its 7 comparisons and BayesDel higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above M-CAP in 5 of 6 comparisons; other studies rank M-CAP above LIST-S2 in 7 of 7.
Livesey and Marsh 2025 ranks M-CAP above LIST-S2 in all 7 of its comparisons. An openly licensed benchmark… ranks LIST-S2 above M-CAP in 5 of its 6 comparisons and M-CAP higher in 1.
Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset ranks ACDC-NN above MAESTRO in 7 of 9 comparisons; other studies rank MAESTRO above ACDC-NN in 5 of 5.
Transfer learning to leverage larger… ranks MAESTRO above ACDC-NN in all 5 comparisons that are not ties (1 is a tie). Predicting protein stability changes… ranks ACDC-NN above MAESTRO in 7 of its 10 comparisons and MAESTRO higher in 2; 1 is a tie.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above DeepSequence in 13 of 17 comparisons; other studies rank DeepSequence above Tranception in 7 of 7.
Livesey and Marsh 2025 ranks DeepSequence above Tranception in all 7 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above DeepSequence in 13 of its 19 comparisons and DeepSequence higher in 4; 2 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above MetaSVM in 7 of 7 comparisons; other studies rank MetaSVM above Condel in 4 of 5.
Clinical likelihood ratios and balanced… ranks MetaSVM above Condel in 4 of its 6 comparisons and Condel higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks Condel above MetaSVM in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks EVmutation above Tranception in 7 of 7 comparisons; other studies rank Tranception above EVmutation in 15 of 18.
ProteinGym AMFR assay-level Spearman… ranks Tranception above EVmutation in 15 of its 19 comparisons and EVmutation higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks EVmutation above Tranception in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above MutationAssessor in 21 of 26.
2 papers rank FATHMM-XF above MutationAssessor in 21 of their 26 comparisons and MutationAssessor higher in 5. Livesey and Marsh 2025 ranks MutationAssessor above FATHMM-XF in all 7 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above CADD in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above CADD in all 7 of its comparisons. An openly licensed benchmark… ranks CADD above MutFormer in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above PolyPhen-2 in 5 of 6 comparisons; other studies rank PolyPhen-2 above SIFT in 24 of 30.
3 papers rank PolyPhen-2 above SIFT in 24 of their 31 comparisons and SIFT higher in 6; 1 is a tie. Livesey and Marsh 2025 ranks SIFT above PolyPhen-2 in 5 of its 6 comparisons and PolyPhen-2 higher in 1.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM above Eigen in 10 of 10 comparisons; other studies rank Eigen above FATHMM in 12 of 13.
2 papers rank Eigen above FATHMM in 12 of their 13 comparisons and FATHMM higher in 1. Comprehensive assessment… ranks FATHMM above Eigen in all 10 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutScore in 6 of 7 comparisons; other studies rank MutScore above MutFormer in 6 of 6.
An openly licensed benchmark… ranks MutScore above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above MutScore in 6 of its 7 comparisons and MutScore higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutScore in 6 of 7 comparisons; other studies rank MutScore above ESM-1b in 6 of 6.
An openly licensed benchmark… ranks MutScore above ESM-1b in all 6 of its comparisons. Livesey and Marsh 2025 ranks ESM-1b above MutScore in 6 of its 7 comparisons and MutScore higher in 1.
ProteinGym AMFR assay-level Spearman comparison ranks EVmutation above CARP in 15 of 19 comparisons; other studies rank CARP above EVmutation in 5 of 6.
Livesey and Marsh 2025 ranks CARP above EVmutation in 5 of its 6 comparisons and EVmutation higher in 1. ProteinGym AMFR assay-level Spearman… ranks EVmutation above CARP in 15 of its 19 comparisons and CARP higher in 4.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MetaSVM in 7 of 7 comparisons; other studies rank MetaSVM above MutFormer in 4 of 5.
An openly licensed benchmark… ranks MetaSVM above MutFormer in 4 of its 6 comparisons and MutFormer higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks MutFormer above MetaSVM in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks GenoCanyon above MetaSVM in 6 of 8 comparisons; other studies rank MetaSVM above GenoCanyon in 12 of 13.
2 papers rank MetaSVM above GenoCanyon in 12 of their 13 comparisons and GenoCanyon higher in 1. Comprehensive assessment… ranks GenoCanyon above MetaSVM in 6 of its 10 comparisons and MetaSVM higher in 2; 2 are ties.
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13) ranks scFoundation above scGPT in 3 of 3 comparisons; other studies rank scGPT above scFoundation in 16 of 18.
2 papers rank scGPT above scFoundation in 16 of their 22 comparisons and scFoundation higher in 2; 4 are ties. Wu et al. 2025 ranks scFoundation above scGPT in all 3 comparisons that are not ties (1 is a tie).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PHACTboost above ClinPred in 6 of 6 comparisons; other studies rank ClinPred above PHACTboost in 6 of 6.
An openly licensed benchmark… ranks ClinPred above PHACTboost in all 6 of its comparisons. Livesey and Marsh 2025 ranks PHACTboost above ClinPred in all 6 comparisons that are not ties (1 is a tie).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above VEST4 in 6 of 7 comparisons; other studies rank VEST4 above MutFormer in 6 of 6.
An openly licensed benchmark… ranks VEST4 above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above VEST4 in 6 of its 7 comparisons and VEST4 higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MPC in 7 of 7 comparisons; other studies rank MPC above MutFormer in 6 of 6.
An openly licensed benchmark… ranks MPC above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above MPC in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above CADD in 6 of 6 comparisons; other studies rank CADD above PolyPhen-2 in 11 of 13.
4 papers rank CADD above PolyPhen-2 in 11 of their 22 comparisons and PolyPhen-2 higher in 2; 9 are ties. Livesey and Marsh 2025 ranks PolyPhen-2 above CADD in all 6 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above Condel in 4 of 5 comparisons; other studies rank Condel above MetaSVM in 7 of 7.
Livesey and Marsh 2025 ranks Condel above MetaSVM in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks MetaSVM above Condel in 4 of its 6 comparisons and Condel higher in 1; 1 is a tie.
ProteinGym AMFR assay-level Spearman comparison ranks VESPAl above CARP in 15 of 19 comparisons; other studies rank CARP above VESPAl in 6 of 6.
Livesey and Marsh 2025 ranks CARP above VESPAl in all 6 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks VESPAl above CARP in 15 of its 19 comparisons and CARP higher in 4.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above MVP in 6 of 7 comparisons; other studies rank MVP above M-CAP in 16 of 17.
3 papers rank MVP above M-CAP in 16 of their 20 comparisons and M-CAP higher in 1; 3 are ties. Livesey and Marsh 2025 ranks M-CAP above MVP in 6 of its 7 comparisons and MVP higher in 1.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks phastCons above GERP in 3 of 3 comparisons; other studies rank GERP above phastCons in 13 of 16.
3 papers rank GERP above phastCons in 13 of their 19 comparisons and phastCons higher in 3; 3 are ties. Brandes et al. 2023 ranks phastCons above GERP in all 3 comparisons that are not ties (1 is a tie).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PolyPhen-2 above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above PolyPhen-2 in 6 of 6.
Livesey and Marsh 2025 ranks MutFormer above PolyPhen-2 in all 6 of its comparisons. An openly licensed benchmark… ranks PolyPhen-2 above MutFormer in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above PHACTboost in 6 of 6 comparisons; other studies rank PHACTboost above ClinPred in 6 of 6.
Livesey and Marsh 2025 ranks PHACTboost above ClinPred in all 6 comparisons that are not ties (1 is a tie). An openly licensed benchmark… ranks ClinPred above PHACTboost in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above M-CAP in 7 of 7 comparisons; other studies rank M-CAP above MutFormer in 4 of 5.
An openly licensed benchmark… ranks M-CAP above MutFormer in 4 of its 6 comparisons and MutFormer higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks MutFormer above M-CAP in all 7 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks ESM-1b above MutationTaster in 4 of 4 comparisons; other studies rank MutationTaster above ESM-1b in 5 of 5.
An openly licensed benchmark… ranks MutationTaster above ESM-1b in all 5 comparisons that are not ties (1 is a tie). Brandes et al. 2023 ranks ESM-1b above MutationTaster in all 4 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above ESM-1b in 6 of 6 comparisons; other studies rank ESM-1b above MutScore in 6 of 7.
Livesey and Marsh 2025 ranks ESM-1b above MutScore in 6 of its 7 comparisons and MutScore higher in 1. An openly licensed benchmark… ranks MutScore above ESM-1b in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above PhD-SNPg in 7 of 7 comparisons; other studies rank PhD-SNPg above Condel in 5 of 6.
Clinical likelihood ratios and balanced… ranks PhD-SNPg above Condel in 5 of its 6 comparisons and Condel higher in 1. Livesey and Marsh 2025 ranks Condel above PhD-SNPg in all 7 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks MSA Transformer above ESM-1v in 16 of 19 comparisons; other studies rank ESM-1v above MSA Transformer in 5 of 5.
Livesey and Marsh 2025 ranks ESM-1v above MSA Transformer in all 5 comparisons that are not ties (1 is a tie). ProteinGym AMFR assay-level Spearman… ranks MSA Transformer above ESM-1v in 16 of its 19 comparisons and ESM-1v higher in 3.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MVP in 10 of 10 comparisons; other studies rank MVP above PrimateAI in 13 of 16.
3 papers rank MVP above PrimateAI in 13 of their 18 comparisons and PrimateAI higher in 3; 2 are ties. Comprehensive assessment… ranks PrimateAI above MVP in all 10 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks ESM-1b above CARP in 14 of 18 comparisons; other studies rank CARP above ESM-1b in 6 of 6.
Livesey and Marsh 2025 ranks CARP above ESM-1b in all 6 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks ESM-1b above CARP in 14 of its 19 comparisons and CARP higher in 4; 1 is a tie.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above Eigen in 7 of 7 comparisons; other studies rank Eigen above MutFormer in 6 of 6.
An openly licensed benchmark… ranks Eigen above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above Eigen in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MPC above REVEL in 10 of 10 comparisons; other studies rank REVEL above MPC in 10 of 12.
3 papers rank REVEL above MPC in 10 of their 14 comparisons and MPC higher in 2; 2 are ties. Comprehensive assessment… ranks MPC above REVEL in all 10 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above PhD-SNPg in 7 of 7 comparisons; other studies rank PhD-SNPg above MutationAssessor in 5 of 6.
Clinical likelihood ratios and balanced… ranks PhD-SNPg above MutationAssessor in 5 of its 6 comparisons and MutationAssessor higher in 1. Livesey and Marsh 2025 ranks MutationAssessor above PhD-SNPg in all 7 of its comparisons.
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNA-MSM above SpliceBERT in 8 of 8 comparisons; other studies rank SpliceBERT above RNA-MSM in 7 of 7.
Benchmarking pre-trained genomic… ranks SpliceBERT above RNA-MSM in all 7 comparisons that are not ties (8 are ties). Zero-shot benchmarking of RNA language… ranks RNA-MSM above SpliceBERT in all 8 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above ClinPred in 7 of 7 comparisons; other studies rank ClinPred above MutFormer in 6 of 6.
An openly licensed benchmark… ranks ClinPred above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above ClinPred in all 7 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MVP above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above MVP in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above MVP in all 7 of its comparisons. An openly licensed benchmark… ranks MVP above MutFormer in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SuSPect above MetaSVM in 7 of 7 comparisons; other studies rank MetaSVM above SuSPect in 5 of 6.
Clinical likelihood ratios and balanced… ranks MetaSVM above SuSPect in 5 of its 6 comparisons and SuSPect higher in 1. Livesey and Marsh 2025 ranks SuSPect above MetaSVM in all 7 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PhD-SNPg above MutationAssessor in 5 of 6 comparisons; other studies rank MutationAssessor above PhD-SNPg in 7 of 7.
Livesey and Marsh 2025 ranks MutationAssessor above PhD-SNPg in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks PhD-SNPg above MutationAssessor in 5 of its 6 comparisons and MutationAssessor higher in 1.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks DANN above M-CAP in 5 of 5 comparisons; other studies rank M-CAP above DANN in 12 of 15.
3 papers rank M-CAP above DANN in 12 of their 17 comparisons and DANN higher in 3; 2 are ties. Of those papers, Clinical likelihood ratios and balanced… ranks DANN higher on its own (2 of 4). Comprehensive assessment… ranks DANN above M-CAP in all 5 comparisons that are not ties (5 are ties).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above PHACTboost in 5 of 5 comparisons; other studies rank PHACTboost above MutScore in 7 of 7.
Livesey and Marsh 2025 ranks PHACTboost above MutScore in all 7 of its comparisons. An openly licensed benchmark… ranks MutScore above PHACTboost in all 5 comparisons that are not ties (1 is a tie).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM-MKL above M-CAP in 16 of 20 comparisons; other studies rank M-CAP above FATHMM-MKL in 8 of 10.
2 papers rank M-CAP above FATHMM-MKL in 8 of their 11 comparisons and FATHMM-MKL higher in 2; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks FATHMM-MKL higher on its own (2 of 4). Comprehensive assessment… ranks FATHMM-MKL above M-CAP in 16 of its 20 comparisons and M-CAP higher in 4.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PrimateAI above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above PrimateAI in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above PrimateAI in all 7 of its comparisons. An openly licensed benchmark… ranks PrimateAI above MutFormer in all 6 of its comparisons.
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) ranks DNABERT-2 above Evo in 35 of 45 comparisons; other studies rank Evo above DNABERT-2 in 6 of 6.
Zero-shot benchmarking of RNA language… ranks Evo above DNABERT-2 in all 6 of its comparisons. GENEB leaderboard ranks DNABERT-2 above Evo in 35 of its 45 comparisons and Evo higher in 10.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks DNABERT above UTR-LM in 8 of 8 comparisons; other studies rank UTR-LM above DNABERT in 23 of 25.
Zero-shot benchmarking of RNA language… ranks UTR-LM above DNABERT in 23 of its 30 comparisons and DNABERT higher in 2; 5 are ties. Benchmarking pre-trained genomic… ranks DNABERT above UTR-LM in all 8 comparisons that are not ties (4 are ties).
Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset ranks ThermoNet above MUpro in 9 of 9 comparisons; other studies rank MUpro above ThermoNet in 5 of 6.
Transfer learning to leverage larger… ranks MUpro above ThermoNet in 5 of its 6 comparisons and ThermoNet higher in 1. Predicting protein stability changes… ranks ThermoNet above MUpro in all 9 comparisons that are not ties (1 is a tie).