Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above MPC in 6 of 7 comparisons; other studies rank MPC above MutationAssessor in 16 of 16.
2 papers rank MPC above MutationAssessor in all 16 of their comparisons. Livesey and Marsh 2025 ranks MutationAssessor above MPC in 6 of its 7 comparisons and MPC higher in 1.
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks Sei above Borzoi in 4 of 4 comparisons; other studies rank Borzoi above Sei in 10 of 10.
2 papers rank Borzoi above Sei in all 10 comparisons that are not ties (1 is a tie). Comparative Analysis of Deep Learning… ranks Sei above Borzoi in all 4 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above FATHMM-XF in 15 of 19 comparisons; other studies rank FATHMM-XF above PrimateAI in 10 of 12.
2 papers rank FATHMM-XF above PrimateAI in 10 of their 13 comparisons and PrimateAI higher in 2; 1 is a tie. Comprehensive assessment… ranks PrimateAI above FATHMM-XF in 15 of its 20 comparisons and FATHMM-XF higher in 4; 1 is a tie.
ProteinGym AMFR assay-level Spearman comparison ranks ProGen2 above CARP in 16 of 19 comparisons; other studies rank CARP above ProGen2 in 6 of 6.
Livesey and Marsh 2025 ranks CARP above ProGen2 in all 6 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks ProGen2 above CARP in 16 of its 19 comparisons and CARP higher in 3.
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) ranks LucaOne above Evo in 40 of 45 comparisons; other studies rank Evo above LucaOne in 6 of 6.
Zero-shot benchmarking of RNA language… ranks Evo above LucaOne in all 6 of its comparisons. GENEB leaderboard ranks LucaOne above Evo in 40 of its 45 comparisons and Evo higher in 5.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks VEST4 above ClinPred in 4 of 4 comparisons; other studies rank ClinPred above VEST4 in 13 of 14.
3 papers rank ClinPred above VEST4 in 13 of their 14 comparisons and VEST4 higher in 1. Clinical likelihood ratios and balanced… ranks VEST4 above ClinPred in all 4 comparisons that are not ties (1 is a tie).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above PrimateAI in 7 of 7 comparisons; other studies rank PrimateAI above MetaSVM in 25 of 30.
3 papers rank PrimateAI above MetaSVM in 25 of their 32 comparisons and MetaSVM higher in 5; 2 are ties. Of those papers, Clinical likelihood ratios and balanced… ranks MetaSVM higher on its own (4 of 6). Livesey and Marsh 2025 ranks MetaSVM above PrimateAI in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MPC in 7 of 7 comparisons; other studies rank MPC above REVEL in 12 of 15.
3 papers rank MPC above REVEL in 12 of their 17 comparisons and REVEL higher in 3; 2 are ties. Of those papers, ProteinGym AMFR assay-level Spearman… ranks REVEL higher on its own (1 of 1). Livesey and Marsh 2025 ranks REVEL above MPC in all 7 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above PHACTboost in 4 of 5 comparisons; other studies rank PHACTboost above BayesDel in 7 of 7.
Livesey and Marsh 2025 ranks PHACTboost above BayesDel in all 7 of its comparisons. An openly licensed benchmark… ranks BayesDel above PHACTboost in 4 of its 6 comparisons and PHACTboost higher in 1; 1 is a tie.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above phyloP in 7 of 7 comparisons; other studies rank phyloP above MutFormer in 4 of 5.
An openly licensed benchmark… ranks phyloP above MutFormer in 4 of its 6 comparisons and MutFormer higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks MutFormer above phyloP in all 7 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks phyloP above MutFormer in 4 of 5 comparisons; other studies rank MutFormer above phyloP in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above phyloP in all 7 of its comparisons. An openly licensed benchmark… ranks phyloP above MutFormer in 4 of its 6 comparisons and MutFormer higher in 1; 1 is a tie.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks GERP above PANTHER in 4 of 5 comparisons; other studies rank PANTHER above GERP in 6 of 6.
Livesey and Marsh 2025 ranks PANTHER above GERP in all 6 of its comparisons. Clinical likelihood ratios and balanced… ranks GERP above PANTHER in 4 of its 6 comparisons and PANTHER higher in 1; 1 is a tie.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above MVP in 6 of 7 comparisons; other studies rank MVP above Condel in 4 of 5.
Clinical likelihood ratios and balanced… ranks MVP above Condel in 4 of its 5 comparisons and Condel higher in 1. Livesey and Marsh 2025 ranks Condel above MVP in 6 of its 7 comparisons and MVP higher in 1.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks CADD above CHASM in 2 of 2 comparisons; other studies rank CHASM above CADD in 9 of 9.
Comprehensive assessment… ranks CHASM above CADD in all 9 comparisons that are not ties (1 is a tie). Clinical likelihood ratios and balanced… ranks CADD above CHASM in all 2 comparisons that are not ties (2 are ties).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above MutFormer in 6 of 6.
An openly licensed benchmark… ranks FATHMM-XF above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above FATHMM-XF in all 7 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks DEOGEN2 above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above DEOGEN2 in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above DEOGEN2 in all 7 of its comparisons. An openly licensed benchmark… ranks DEOGEN2 above MutFormer in all 6 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above Condel in 5 of 5 comparisons; other studies rank Condel above FATHMM-MKL in 7 of 7.
Livesey and Marsh 2025 ranks Condel above FATHMM-MKL in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks FATHMM-MKL above Condel in all 5 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above MutScore in 6 of 7.
Livesey and Marsh 2025 ranks MutFormer above MutScore in 6 of its 7 comparisons and MutScore higher in 1. An openly licensed benchmark… ranks MutScore above MutFormer in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above VESPA in 6 of 6 comparisons; other studies rank VESPA above CARP in 18 of 19.
ProteinGym AMFR assay-level Spearman… ranks VESPA above CARP in 18 of its 19 comparisons and CARP higher in 1. Livesey and Marsh 2025 ranks CARP above VESPA in all 6 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks phyloP above SuSPect in 5 of 6 comparisons; other studies rank SuSPect above phyloP in 7 of 7.
Livesey and Marsh 2025 ranks SuSPect above phyloP in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks phyloP above SuSPect in 5 of its 6 comparisons and SuSPect higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ProGen2 above Tranception in 6 of 6 comparisons; other studies rank Tranception above ProGen2 in 18 of 18.
ProteinGym AMFR assay-level Spearman… ranks Tranception above ProGen2 in all 18 comparisons that are not ties (2 are ties). Livesey and Marsh 2025 ranks ProGen2 above Tranception in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above LIST-S2 in 7 of 7 comparisons; other studies rank LIST-S2 above M-CAP in 5 of 6.
An openly licensed benchmark… ranks LIST-S2 above M-CAP in 5 of its 6 comparisons and M-CAP higher in 1. Livesey and Marsh 2025 ranks M-CAP above LIST-S2 in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above DANN in 7 of 7 comparisons; other studies rank DANN above Condel in 5 of 6.
Clinical likelihood ratios and balanced… ranks DANN above Condel in 5 of its 6 comparisons and Condel higher in 1. Livesey and Marsh 2025 ranks Condel above DANN in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above SIFT in 16 of 19 comparisons; other studies rank SIFT above LRT in 15 of 17.
4 papers rank SIFT above LRT in 15 of their 17 comparisons and LRT higher in 2. Comprehensive assessment… ranks LRT above SIFT in 16 of its 20 comparisons and SIFT higher in 3; 1 is a tie.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above Tranception in 6 of 6 comparisons; other studies rank Tranception above CARP in 19 of 19.
ProteinGym AMFR assay-level Spearman… ranks Tranception above CARP in all 19 of its comparisons. Livesey and Marsh 2025 ranks CARP above Tranception in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutationAssessor in 7 of 7 comparisons; other studies rank MutationAssessor above MutFormer in 5 of 5.
An openly licensed benchmark… ranks MutationAssessor above MutFormer in all 5 comparisons that are not ties (1 is a tie). Livesey and Marsh 2025 ranks MutFormer above MutationAssessor in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above MVP in 6 of 6 comparisons; other studies rank MVP above PolyPhen-2 in 16 of 20.
3 papers rank MVP above PolyPhen-2 in 16 of their 21 comparisons and PolyPhen-2 higher in 4; 1 is a tie. Livesey and Marsh 2025 ranks PolyPhen-2 above MVP in all 6 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PhD-SNPg above PANTHER in 6 of 6 comparisons; other studies rank PANTHER above PhD-SNPg in 6 of 6.
Livesey and Marsh 2025 ranks PANTHER above PhD-SNPg in all 6 of its comparisons. Clinical likelihood ratios and balanced… ranks PhD-SNPg above PANTHER in all 6 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above ProGen2 in 18 of 18 comparisons; other studies rank ProGen2 above Tranception in 6 of 6.
Livesey and Marsh 2025 ranks ProGen2 above Tranception in all 6 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above ProGen2 in all 18 comparisons that are not ties (2 are ties).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above MetaSVM in 3 of 4 comparisons; other studies rank MetaSVM above LIST-S2 in 7 of 7.
Livesey and Marsh 2025 ranks MetaSVM above LIST-S2 in all 7 of its comparisons. An openly licensed benchmark… ranks LIST-S2 above MetaSVM in 3 of its 6 comparisons and MetaSVM higher in 1; 2 are ties.
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above Enformer in 4 of 4 comparisons; other studies rank Enformer above ChromBPNet in 4 of 5.
GUANinE v1.1 reveals complementarity… ranks Enformer above ChromBPNet in 4 of its 7 comparisons and ChromBPNet higher in 1; 2 are ties. Comparative Analysis of Deep Learning… ranks ChromBPNet above Enformer in all 4 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks GERP above Condel in 6 of 6 comparisons; other studies rank Condel above GERP in 7 of 7.
Livesey and Marsh 2025 ranks Condel above GERP in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks GERP above Condel in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MVP in 7 of 7 comparisons; other studies rank MVP above MutFormer in 6 of 6.
An openly licensed benchmark… ranks MVP above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above MVP in all 7 of its comparisons.
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above Nucleotide Transformer v2 in 6 of 6 comparisons; other studies rank Nucleotide Transformer v2 above Evo in 42 of 45.
GENEB leaderboard ranks Nucleotide Transformer v2 above Evo in 42 of its 45 comparisons and Evo higher in 3. Zero-shot benchmarking of RNA language… ranks Evo above Nucleotide Transformer v2 in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks VARITY above popEVE in 4 of 5 comparisons; other studies rank popEVE above VARITY in 6 of 6.
Livesey and Marsh 2025 ranks popEVE above VARITY in all 6 comparisons that are not ties (1 is a tie). An openly licensed benchmark… ranks VARITY above popEVE in 4 of its 6 comparisons and popEVE higher in 1; 1 is a tie.
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling ranks Majority-vote consensus of somatic callers above DeepSomatic in 20 of 20 comparisons; other studies rank DeepSomatic above Majority-vote consensus of somatic callers in 8 of 10.
A benchmarking study of individual… ranks DeepSomatic above Majority-vote consensus of somatic callers in 8 of its 12 comparisons and Majority-vote consensus of somatic callers higher in 2; 2 are ties. Prospectively frozen external… ranks Majority-vote consensus of somatic callers above DeepSomatic in all 20 comparisons that are not ties (4 are ties).
DeepSomatic higherTieMajority-vote consensus of somatic callers higher
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks M-CAP above MutFormer in 4 of 5 comparisons; other studies rank MutFormer above M-CAP in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above M-CAP in all 7 of its comparisons. An openly licensed benchmark… ranks M-CAP above MutFormer in 4 of its 6 comparisons and MutFormer higher in 1; 1 is a tie.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks DANN above Condel in 5 of 6 comparisons; other studies rank Condel above DANN in 7 of 7.
Livesey and Marsh 2025 ranks Condel above DANN in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks DANN above Condel in 5 of its 6 comparisons and Condel higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above MetaLR in 4 of 5 comparisons; other studies rank MetaLR above LIST-S2 in 7 of 7.
Livesey and Marsh 2025 ranks MetaLR above LIST-S2 in all 7 of its comparisons. An openly licensed benchmark… ranks LIST-S2 above MetaLR in 4 of its 6 comparisons and MetaLR higher in 1; 1 is a tie.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks VEST4 above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above VEST4 in 6 of 7.
Livesey and Marsh 2025 ranks MutFormer above VEST4 in 6 of its 7 comparisons and VEST4 higher in 1. An openly licensed benchmark… ranks VEST4 above MutFormer in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks REVEL above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above REVEL in 6 of 7.
Livesey and Marsh 2025 ranks MutFormer above REVEL in 6 of its 7 comparisons and REVEL higher in 1. An openly licensed benchmark… ranks REVEL above MutFormer in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SuSPect above MetaLR in 6 of 7 comparisons; other studies rank MetaLR above SuSPect in 4 of 5.
Clinical likelihood ratios and balanced… ranks MetaLR above SuSPect in 4 of its 6 comparisons and SuSPect higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks SuSPect above MetaLR in 6 of its 7 comparisons and MetaLR higher in 1.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PhD-SNPg above Condel in 5 of 6 comparisons; other studies rank Condel above PhD-SNPg in 7 of 7.
Livesey and Marsh 2025 ranks Condel above PhD-SNPg in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks PhD-SNPg above Condel in 5 of its 6 comparisons and Condel higher in 1.
Csendes et al. 2025, Supplementary Material 4 (Supplementary Tables 1-3) ranks scGPT above scFoundation in 15 of 16 comparisons; other studies rank scFoundation above scGPT in 4 of 5.
2 papers rank scFoundation above scGPT in 4 of their 6 comparisons and scGPT higher in 1; 1 is a tie. Csendes et al. 2025 ranks scGPT above scFoundation in 15 of its 20 comparisons and scFoundation higher in 1; 4 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above ProGen2 in 6 of 6 comparisons; other studies rank ProGen2 above CARP in 16 of 19.
ProteinGym AMFR assay-level Spearman… ranks ProGen2 above CARP in 16 of its 19 comparisons and CARP higher in 3. Livesey and Marsh 2025 ranks CARP above ProGen2 in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above MVP in 6 of 7 comparisons; other studies rank MVP above SIFT in 15 of 18.
3 papers rank MVP above SIFT in 15 of their 21 comparisons and SIFT higher in 3; 3 are ties. Livesey and Marsh 2025 ranks SIFT above MVP in 6 of its 7 comparisons and MVP higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above FATHMM-MKL in 7 of 7 comparisons; other studies rank FATHMM-MKL above Condel in 5 of 5.
Clinical likelihood ratios and balanced… ranks FATHMM-MKL above Condel in all 5 of its comparisons. Livesey and Marsh 2025 ranks Condel above FATHMM-MKL in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above VEST4 in 6 of 7 comparisons; other studies rank VEST4 above ESM-1b in 7 of 8.
2 papers rank VEST4 above ESM-1b in 7 of their 10 comparisons and ESM-1b higher in 1; 2 are ties. Livesey and Marsh 2025 ranks ESM-1b above VEST4 in 6 of its 7 comparisons and VEST4 higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MPC above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above MPC in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above MPC in all 7 of its comparisons. An openly licensed benchmark… ranks MPC above MutFormer in all 6 of its comparisons.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks UTR-LM above RNA-MSM in 3 of 3 comparisons; other studies rank RNA-MSM above UTR-LM in 8 of 8.
Zero-shot benchmarking of RNA language… ranks RNA-MSM above UTR-LM in all 8 of its comparisons. Benchmarking pre-trained genomic… ranks UTR-LM above RNA-MSM in all 3 comparisons that are not ties (12 are ties).