Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM above MVP in 10 of 10 comparisons; other studies rank MVP above FATHMM in 11 of 12.
2 papers rank MVP above FATHMM in 11 of their 12 comparisons and FATHMM higher in 1. Comprehensive assessment… ranks FATHMM above MVP in all 10 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above SIFT in 7 of 7 comparisons; other studies rank SIFT above MutFormer in 5 of 6.
An openly licensed benchmark… ranks SIFT above MutFormer in 5 of its 6 comparisons and MutFormer higher in 1. Livesey and Marsh 2025 ranks MutFormer above SIFT in all 7 of its comparisons.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks SpliceBERT above RNA-FM in 3 of 3 comparisons; other studies rank RNA-FM above SpliceBERT in 31 of 36.
Zero-shot benchmarking of RNA language… ranks RNA-FM above SpliceBERT in 31 of its 36 comparisons and SpliceBERT higher in 5. Benchmarking pre-trained genomic… ranks SpliceBERT above RNA-FM in all 3 comparisons that are not ties (12 are ties).
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaLR above ESM-1b in 2 of 2 comparisons; other studies rank ESM-1b above MetaLR in 13 of 13.
2 papers rank ESM-1b above MetaLR in all 13 of their comparisons. Brandes et al. 2023 ranks MetaLR above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks ESM-1b above ESM-2 in 26 of 28 comparisons; other studies rank ESM-2 above ESM-1b in 20 of 25.
2 papers rank ESM-2 above ESM-1b in 20 of their 26 comparisons and ESM-1b higher in 5; 1 is a tie. Genome modelling and design across all… ranks ESM-1b above ESM-2 in 26 of its 28 comparisons and ESM-2 higher in 2.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VESPAl above Tranception in 6 of 7 comparisons; other studies rank Tranception above VESPAl in 18 of 19.
ProteinGym AMFR assay-level Spearman… ranks Tranception above VESPAl in 18 of its 19 comparisons and VESPAl higher in 1. Livesey and Marsh 2025 ranks VESPAl above Tranception in 6 of its 7 comparisons and Tranception higher in 1.
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons; other studies rank Nucleotide Transformer above RNAErnie in 7 of 7.
Benchmarking pre-trained genomic… ranks Nucleotide Transformer above RNAErnie in all 7 comparisons that are not ties (8 are ties). RNAGym fitness leaderboard ranks RNAErnie above Nucleotide Transformer in all 4 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaSVM above ESM-1b in 2 of 2 comparisons; other studies rank ESM-1b above MetaSVM in 13 of 13.
2 papers rank ESM-1b above MetaSVM in all 13 of their comparisons. Brandes et al. 2023 ranks MetaSVM above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PANTHER above GERP in 6 of 6 comparisons; other studies rank GERP above PANTHER in 4 of 5.
Clinical likelihood ratios and balanced… ranks GERP above PANTHER in 4 of its 6 comparisons and PANTHER higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks PANTHER above GERP in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MetaRNN in 3 of 4 comparisons; other studies rank MetaRNN above MutScore in 7 of 7.
Livesey and Marsh 2025 ranks MetaRNN above MutScore in all 7 of its comparisons. An openly licensed benchmark… ranks MutScore above MetaRNN in 3 of its 6 comparisons and MetaRNN higher in 1; 2 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above M-CAP in 22 of 25.
2 papers rank FATHMM-XF above M-CAP in 22 of their 26 comparisons and M-CAP higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks M-CAP above FATHMM-XF in all 7 of its comparisons.
Achieving robust somatic mutation detection with deep learning models derived from reference data sets of a cancer sample ranks Octopus above Mutect2 in 154 of 154 comparisons; other studies rank Mutect2 above Octopus in 14 of 14.
ClairS-TO ranks Mutect2 above Octopus in all 14 of its comparisons. Achieving robust somatic mutation… ranks Octopus above Mutect2 in all 154 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above ESM-1b in 7 of 7 comparisons; other studies rank ESM-1b above MutFormer in 6 of 6.
An openly licensed benchmark… ranks ESM-1b above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above ESM-1b in all 7 of its comparisons.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks SpliceBERT above RNA-MSM in 7 of 7 comparisons; other studies rank RNA-MSM above SpliceBERT in 8 of 8.
Zero-shot benchmarking of RNA language… ranks RNA-MSM above SpliceBERT in all 8 of its comparisons. Benchmarking pre-trained genomic… ranks SpliceBERT above RNA-MSM in all 7 comparisons that are not ties (8 are ties).
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks CADD above ESM-2 in 14 of 14 comparisons; other studies rank ESM-2 above CADD in 6 of 7.
Livesey and Marsh 2025 ranks ESM-2 above CADD in 6 of its 7 comparisons and CADD higher in 1. Genome modelling and design across all… ranks CADD above ESM-2 in all 14 of its comparisons.
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) ranks Nucleotide Transformer v2 above Evo in 42 of 45 comparisons; other studies rank Evo above Nucleotide Transformer v2 in 6 of 6.
Zero-shot benchmarking of RNA language… ranks Evo above Nucleotide Transformer v2 in all 6 of its comparisons. GENEB leaderboard ranks Nucleotide Transformer v2 above Evo in 42 of its 45 comparisons and Evo higher in 3.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above CADD in 7 of 7 comparisons; other studies rank CADD above MutFormer in 6 of 6.
An openly licensed benchmark… ranks CADD above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above CADD in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above Eigen in 5 of 5 comparisons; other studies rank Eigen above PrimateAI in 15 of 18.
3 papers rank Eigen above PrimateAI in 15 of their 19 comparisons and PrimateAI higher in 3; 1 is a tie. Comprehensive assessment… ranks PrimateAI above Eigen in all 5 comparisons that are not ties (5 are ties).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks Eigen above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above Eigen in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above Eigen in all 7 of its comparisons. An openly licensed benchmark… ranks Eigen above MutFormer in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above CADD in 10 of 10 comparisons; other studies rank CADD above PrimateAI in 15 of 17.
3 papers rank CADD above PrimateAI in 15 of their 18 comparisons and PrimateAI higher in 2; 1 is a tie. Comprehensive assessment… ranks PrimateAI above CADD in all 10 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks DeepSequence above ESM-1v in 13 of 16 comparisons; other studies rank ESM-1v above DeepSequence in 6 of 7.
Livesey and Marsh 2025 ranks ESM-1v above DeepSequence in 6 of its 7 comparisons and DeepSequence higher in 1. ProteinGym AMFR assay-level Spearman… ranks DeepSequence above ESM-1v in 13 of its 19 comparisons and ESM-1v higher in 3; 3 are ties.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaLR above SuSPect in 4 of 5 comparisons; other studies rank SuSPect above MetaLR in 6 of 7.
Livesey and Marsh 2025 ranks SuSPect above MetaLR in 6 of its 7 comparisons and MetaLR higher in 1. Clinical likelihood ratios and balanced… ranks MetaLR above SuSPect in 4 of its 6 comparisons and SuSPect higher in 1; 1 is a tie.
TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) ranks phyloP above Evo 2 in 2 of 2 comparisons; other studies rank Evo 2 above phyloP in 72 of 84.
2 papers rank Evo 2 above phyloP in 72 of their 91 comparisons and phyloP higher in 12; 7 are ties. Of those papers, GUANinE v1.1 reveals complementarity… ranks phyloP higher on its own (1 of 1). TraitGym official metrics ranks phyloP above Evo 2 in all 2 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above VEST4 in 5 of 5 comparisons; other studies rank VEST4 above BayesDel in 11 of 13.
3 papers rank VEST4 above BayesDel in 11 of their 14 comparisons and BayesDel higher in 2; 1 is a tie. Of those papers, ProteinGym AMFR assay-level Spearman… ranks BayesDel higher on its own (1 of 1). An openly licensed benchmark… ranks BayesDel above VEST4 in all 5 comparisons that are not ties (1 is a tie).
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNABERT above DNABERT in 21 of 27 comparisons; other studies rank DNABERT above RNABERT in 12 of 12.
Benchmarking pre-trained genomic… ranks DNABERT above RNABERT in all 12 of its comparisons. Zero-shot benchmarking of RNA language… ranks RNABERT above DNABERT in 21 of its 30 comparisons and DNABERT higher in 6; 3 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above MetaLR in 22 of 25.
2 papers rank FATHMM-XF above MetaLR in 22 of their 26 comparisons and MetaLR higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks MetaLR above FATHMM-XF in all 7 of its comparisons.
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above Borzoi in 4 of 4 comparisons; other studies rank Borzoi above ChromBPNet in 12 of 15.
2 papers rank Borzoi above ChromBPNet in 12 of their 17 comparisons and ChromBPNet higher in 3; 2 are ties. Comparative Analysis of Deep Learning… ranks ChromBPNet above Borzoi in all 4 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above CADD in 6 of 7 comparisons; other studies rank CADD above Condel in 4 of 5.
Clinical likelihood ratios and balanced… ranks CADD above Condel in 4 of its 5 comparisons and Condel higher in 1. Livesey and Marsh 2025 ranks Condel above CADD in 6 of its 7 comparisons and CADD higher in 1.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above VESPAl in 18 of 19 comparisons; other studies rank VESPAl above Tranception in 6 of 7.
Livesey and Marsh 2025 ranks VESPAl above Tranception in 6 of its 7 comparisons and Tranception higher in 1. ProteinGym AMFR assay-level Spearman… ranks Tranception above VESPAl in 18 of its 19 comparisons and VESPAl higher in 1.
Evaluation of phenotype-driven gene prioritization methods for Mendelian diseases ranks AMELIE above Exomiser in 11 of 14 comparisons; other studies rank Exomiser above AMELIE in 14 of 14.
Refined preferences of prioritizers… ranks Exomiser above AMELIE in all 14 of its comparisons. Evaluation of phenotype-driven gene… ranks AMELIE above Exomiser in 11 of its 14 comparisons and Exomiser higher in 3.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MutScore in 6 of 6 comparisons; other studies rank MutScore above REVEL in 6 of 6.
An openly licensed benchmark… ranks MutScore above REVEL in all 6 of its comparisons. Livesey and Marsh 2025 ranks REVEL above MutScore in all 6 comparisons that are not ties (1 is a tie).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above REVEL in 6 of 6 comparisons; other studies rank REVEL above MutScore in 6 of 6.
Livesey and Marsh 2025 ranks REVEL above MutScore in all 6 comparisons that are not ties (1 is a tie). An openly licensed benchmark… ranks MutScore above REVEL in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MutPred in 9 of 10 comparisons; other studies rank MutPred above PrimateAI in 16 of 20.
4 papers rank MutPred above PrimateAI in 16 of their 20 comparisons and PrimateAI higher in 4. Of those papers, Clinical likelihood ratios and balanced… ranks PrimateAI higher on its own (4 of 6). Comprehensive assessment… ranks PrimateAI above MutPred in 9 of its 10 comparisons and MutPred higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above MetaSVM in 22 of 26.
2 papers rank FATHMM-XF above MetaSVM in 22 of their 26 comparisons and MetaSVM higher in 4. Livesey and Marsh 2025 ranks MetaSVM above FATHMM-XF in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above REVEL in 6 of 7 comparisons; other studies rank REVEL above MutFormer in 6 of 6.
An openly licensed benchmark… ranks REVEL above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above REVEL in 6 of its 7 comparisons and REVEL higher in 1.
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks HyenaDNA above DNABERT-2 in 3 of 4 comparisons; other studies rank DNABERT-2 above HyenaDNA in 89 of 108.
3 papers rank DNABERT-2 above HyenaDNA in 89 of their 115 comparisons and HyenaDNA higher in 19; 7 are ties. Comparative Analysis of Deep Learning… ranks HyenaDNA above DNABERT-2 in 3 of its 4 comparisons and DNABERT-2 higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks SIFT above MutFormer in 5 of 6 comparisons; other studies rank MutFormer above SIFT in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above SIFT in all 7 of its comparisons. An openly licensed benchmark… ranks SIFT above MutFormer in 5 of its 6 comparisons and MutFormer higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above REVEL in 6 of 6 comparisons; other studies rank REVEL above BayesDel in 11 of 13.
3 papers rank REVEL above BayesDel in 11 of their 14 comparisons and BayesDel higher in 2; 1 is a tie. Of those papers, ProteinGym AMFR assay-level Spearman… ranks BayesDel higher on its own (1 of 1). An openly licensed benchmark… ranks BayesDel above REVEL in all 6 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above SNAP2 in 3 of 4 comparisons; other studies rank SNAP2 above FATHMM-MKL in 7 of 7.
Livesey and Marsh 2025 ranks SNAP2 above FATHMM-MKL in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks FATHMM-MKL above SNAP2 in 3 of its 5 comparisons and SNAP2 higher in 1; 1 is a tie.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaSVM in 16 of 20 comparisons; other studies rank MetaSVM above LRT in 12 of 13.
2 papers rank MetaSVM above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaSVM in 16 of its 20 comparisons and MetaSVM higher in 4.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM above REVEL in 5 of 5 comparisons; other studies rank REVEL above FATHMM in 14 of 14.
3 papers rank REVEL above FATHMM in all 14 of their comparisons. Comprehensive assessment… ranks FATHMM above REVEL in all 5 comparisons that are not ties (5 are ties).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SuSPect above phyloP in 7 of 7 comparisons; other studies rank phyloP above SuSPect in 5 of 6.
Clinical likelihood ratios and balanced… ranks phyloP above SuSPect in 5 of its 6 comparisons and SuSPect higher in 1. Livesey and Marsh 2025 ranks SuSPect above phyloP in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PANTHER above PhD-SNPg in 6 of 6 comparisons; other studies rank PhD-SNPg above PANTHER in 6 of 6.
Clinical likelihood ratios and balanced… ranks PhD-SNPg above PANTHER in all 6 of its comparisons. Livesey and Marsh 2025 ranks PANTHER above PhD-SNPg in all 6 of its comparisons.
Transfer learning to leverage larger datasets for improved prediction of protein stability changes ranks mCSM above ThermoNet in 6 of 6 comparisons; other studies rank ThermoNet above mCSM in 8 of 10.
Predicting protein stability changes… ranks ThermoNet above mCSM in 8 of its 10 comparisons and mCSM higher in 2. Transfer learning to leverage larger… ranks mCSM above ThermoNet in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutPred in 6 of 7 comparisons; other studies rank MutPred above MutFormer in 6 of 6.
An openly licensed benchmark… ranks MutPred above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above MutPred in 6 of its 7 comparisons and MutPred higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above DeepSequence in 6 of 7 comparisons; other studies rank DeepSequence above ESM-1v in 13 of 16.
ProteinGym AMFR assay-level Spearman… ranks DeepSequence above ESM-1v in 13 of its 19 comparisons and ESM-1v higher in 3; 3 are ties. Livesey and Marsh 2025 ranks ESM-1v above DeepSequence in 6 of its 7 comparisons and DeepSequence higher in 1.
ProteinGym AMFR assay-level Spearman comparison ranks VESPA above DeepSequence in 13 of 17 comparisons; other studies rank DeepSequence above VESPA in 5 of 6.
Livesey and Marsh 2025 ranks DeepSequence above VESPA in 5 of its 7 comparisons and VESPA higher in 1; 1 is a tie. ProteinGym AMFR assay-level Spearman… ranks VESPA above DeepSequence in 13 of its 19 comparisons and DeepSequence higher in 4; 2 are ties.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above ClinPred in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above ClinPred in all 7 of its comparisons. An openly licensed benchmark… ranks ClinPred above MutFormer in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutPred in 6 of 7 comparisons; other studies rank MutPred above ESM-1b in 8 of 9.
3 papers rank MutPred above ESM-1b in 8 of their 11 comparisons and ESM-1b higher in 1; 2 are ties. Of those papers, ProteinGym AMFR assay-level Spearman… ranks ESM-1b higher on its own (1 of 1). Livesey and Marsh 2025 ranks ESM-1b above MutPred in 6 of its 7 comparisons and MutPred higher in 1.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above ESM-2 in 14 of 17 comparisons; other studies rank ESM-2 above Tranception in 7 of 7.
Livesey and Marsh 2025 ranks ESM-2 above Tranception in all 7 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above ESM-2 in 14 of its 19 comparisons and ESM-2 higher in 3; 2 are ties.