Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
BEELINE: 10.1038/s41592-019-0690-6 source data published 2020 ranks PPCOR (partial Pearson correlation) above GRNBoost2 in 9 of 12 comparisons; 2 of 2 other studies rank GRNBoost2 above PPCOR (partial Pearson correlation).
2 papers rank GRNBoost2 above PPCOR (partial Pearson correlation) in 47 of their 59 comparisons and PPCOR (partial Pearson correlation) higher in 8; 4 are ties. BEELINE ranks PPCOR (partial Pearson correlation) above GRNBoost2 in 9 of its 12 comparisons and GRNBoost2 higher in 3.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SIFT above PROVEAN in 3 of 3 comparisons; 3 of 3 other studies rank PROVEAN above SIFT.
5 papers rank PROVEAN above SIFT in 32 of their 39 comparisons and SIFT higher in 4; 3 are ties. Brandes et al. 2023 ranks SIFT above PROVEAN in all 3 comparisons that are not ties (1 is a tie).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks ESM3 above ESM-2 in 18 of 18 comparisons; pfmbench primary benchmark evidence ranks ESM-2 above ESM3 in 9 of 12 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank ESM-2 above ESM3 in 20 of their 59 comparisons and ESM3 higher in 32; 7 are ties. Of those papers, ProteinGym ranks ESM3 higher on its own (18 of 19).
On ProteinGym mutation effects, ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design reports ESM3 at a median relative difference of +0.20 from ESM-2; other studies report -0.06 to -0.01.
In 2 papers, ESM3 scores between 1% and 6% below ESM-2. In ProteinGym, ESM3 scores 20% above ESM-2.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaLR in 15 of 17 comparisons; 2 of 2 other studies rank MetaLR above LRT.
2 papers rank MetaLR above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaLR in 15 of its 20 comparisons and MetaLR higher in 2; 3 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above CADD in 6 of 7 comparisons; 2 of 2 other studies rank CADD above MetaLR.
3 papers rank CADD above MetaLR in 19 of their 21 comparisons and MetaLR higher in 2. Livesey and Marsh 2025 ranks MetaLR above CADD in 6 of its 7 comparisons and CADD higher in 1.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM-XF above CADD in 10 of 10 comparisons; 2 of 2 other studies rank CADD above FATHMM-XF.
2 papers rank CADD above FATHMM-XF in all 12 comparisons that are not ties (1 is a tie). Comprehensive assessment… ranks FATHMM-XF above CADD in all 10 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MVP above ESM-1b in 2 of 2 comparisons; 2 of 2 other studies rank ESM-1b above MVP.
2 papers rank ESM-1b above MVP in all 13 of their comparisons. Brandes et al. 2023 ranks MVP above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNA-FM above RNAErnie in 28 of 36 comparisons; 2 of 2 other studies rank RNAErnie above RNA-FM.
2 papers rank RNAErnie above RNA-FM in all 6 comparisons that are not ties (13 are ties). Zero-shot benchmarking of RNA language… ranks RNA-FM above RNAErnie in 28 of its 36 comparisons and RNAErnie higher in 8.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks GERP above DANN in 5 of 6 comparisons; 2 of 2 other studies rank DANN above GERP.
2 papers rank DANN above GERP in all 12 comparisons that are not ties (1 is a tie). Clinical likelihood ratios and balanced… ranks GERP above DANN in 5 of its 6 comparisons and DANN higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above ClinPred in 6 of 7 comparisons; 2 of 2 other studies rank ClinPred above REVEL.
3 papers rank ClinPred above REVEL in 10 of their 12 comparisons and REVEL higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks REVEL above ClinPred in 6 of its 7 comparisons and ClinPred higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above BayesDel in 6 of 7 comparisons; 2 of 2 other studies rank BayesDel above ESM-1b.
2 papers rank BayesDel above ESM-1b in all 10 of their comparisons. Livesey and Marsh 2025 ranks ESM-1b above BayesDel in 6 of its 7 comparisons and BayesDel higher in 1.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above VEST4 in 3 of 4 comparisons; 2 of 2 other studies rank VEST4 above MVP.
3 papers rank VEST4 above MVP in 17 of their 23 comparisons and MVP higher in 6. Clinical likelihood ratios and balanced… ranks MVP above VEST4 in 3 of its 5 comparisons and VEST4 higher in 1; 1 is a tie.
GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models ranks GENA-LM above Caduceus in 41 of 48 comparisons; 2 of 2 other studies rank Caduceus above GENA-LM.
3 papers rank Caduceus above GENA-LM in 23 of their 55 comparisons and GENA-LM higher in 31; 1 is a tie. Of those papers, GENEB leaderboard ranks GENA-LM higher on its own (30 of 45). GenBench ranks GENA-LM above Caduceus in 41 of its 51 comparisons and Caduceus higher in 7; 3 are ties.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MVP above CADD in 10 of 10 comparisons; 2 of 2 other studies rank CADD above MVP.
3 papers rank CADD above MVP in 11 of their 18 comparisons and MVP higher in 4; 3 are ties. Of those papers, Clinical likelihood ratios and balanced… ranks MVP higher on its own (2 of 5). Comprehensive assessment… ranks MVP above CADD in all 10 of its comparisons.
GenomeOcean: An Efficient Genome Foundation Model Trained on Large-Scale Metagenomic Assemblies ranks DNABERT-2 above Nucleotide Transformer in 3 of 3 comparisons; 7 of 8 other studies rank Nucleotide Transformer above DNABERT-2.
10 papers rank Nucleotide Transformer above DNABERT-2 in 104 of their 171 comparisons and DNABERT-2 higher in 52; 15 are ties. Of those papers, DNABERT-2, BEND rank DNABERT-2 higher on their own (24 of 28, 5 of 7). GenomeOcean 2025 ranks DNABERT-2 above Nucleotide Transformer in all 3 of its comparisons.
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines ranks Geneformer above scFoundation in 2 of 2 comparisons; 2 of 2 other studies rank scFoundation above Geneformer.
2 papers rank scFoundation above Geneformer in 13 of their 16 comparisons and Geneformer higher in 3. Deep-learning-based gene perturbation… ranks Geneformer above scFoundation in all 2 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above REVEL in 4 of 5 comparisons; 3 of 3 other studies rank REVEL above MVP.
3 papers rank REVEL above MVP in all 19 comparisons that are not ties (4 are ties). Clinical likelihood ratios and balanced… ranks MVP above REVEL in 4 of its 5 comparisons and REVEL higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above PrimateAI in 7 of 7 comparisons; 2 of 2 other studies rank PrimateAI above M-CAP.
3 papers rank PrimateAI above M-CAP in 24 of their 30 comparisons and M-CAP higher in 6. Livesey and Marsh 2025 ranks M-CAP above PrimateAI in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above FATHMM-MKL in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-MKL above MutationAssessor.
2 papers rank FATHMM-MKL above MutationAssessor in 19 of their 25 comparisons and MutationAssessor higher in 6. Livesey and Marsh 2025 ranks MutationAssessor above FATHMM-MKL in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks GenoCanyon above SIFT in 6 of 8 comparisons; 3 of 3 other studies rank SIFT above GenoCanyon.
3 papers rank SIFT above GenoCanyon in 14 of their 16 comparisons and GenoCanyon higher in 1; 1 is a tie. Comprehensive assessment… ranks GenoCanyon above SIFT in 6 of its 10 comparisons and SIFT higher in 2; 2 are ties.
DNABERT-2: Efficient Foundation Model and Benchmark for Multi-Species Genomes ranks DNABERT-2 above Nucleotide Transformer in 24 of 26 comparisons; 7 of 8 other studies rank Nucleotide Transformer above DNABERT-2.
10 papers rank Nucleotide Transformer above DNABERT-2 in 102 of their 146 comparisons and DNABERT-2 higher in 31; 13 are ties. Of those papers, BEND, GenomeOcean 2025 rank DNABERT-2 higher on their own (5 of 7, 3 of 3). DNABERT-2 ranks DNABERT-2 above Nucleotide Transformer in 24 of its 28 comparisons and Nucleotide Transformer higher in 2; 2 are ties.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaLR above ESM-1b in 2 of 2 comparisons; 2 of 2 other studies rank ESM-1b above MetaLR.
2 papers rank ESM-1b above MetaLR in all 13 of their comparisons. Brandes et al. 2023 ranks MetaLR above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaSVM above ESM-1b in 2 of 2 comparisons; 2 of 2 other studies rank ESM-1b above MetaSVM.
2 papers rank ESM-1b above MetaSVM in all 13 of their comparisons. Brandes et al. 2023 ranks MetaSVM above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above M-CAP.
2 papers rank FATHMM-XF above M-CAP in 22 of their 26 comparisons and M-CAP higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks M-CAP above FATHMM-XF in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above FATHMM-MKL in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-MKL above MetaLR.
2 papers rank FATHMM-MKL above MetaLR in 18 of their 25 comparisons and MetaLR higher in 5; 2 are ties. Livesey and Marsh 2025 ranks MetaLR above FATHMM-MKL in all 7 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks phyloP above SIFT in 3 of 4 comparisons; 2 of 2 other studies rank SIFT above phyloP.
3 papers rank SIFT above phyloP in 14 of their 16 comparisons and phyloP higher in 2. An openly licensed benchmark… ranks phyloP above SIFT in 3 of its 6 comparisons and SIFT higher in 1; 2 are ties.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above CADD in 10 of 10 comparisons; 2 of 2 other studies rank CADD above PrimateAI.
3 papers rank CADD above PrimateAI in 15 of their 18 comparisons and PrimateAI higher in 2; 1 is a tie. Comprehensive assessment… ranks PrimateAI above CADD in all 10 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above MetaLR.
2 papers rank FATHMM-XF above MetaLR in 22 of their 26 comparisons and MetaLR higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks MetaLR above FATHMM-XF in all 7 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above MutationAssessor in 4 of 5 comparisons; 2 of 2 other studies rank MutationAssessor above MetaSVM.
3 papers rank MutationAssessor above MetaSVM in 22 of their 33 comparisons and MetaSVM higher in 10; 1 is a tie. Clinical likelihood ratios and balanced… ranks MetaSVM above MutationAssessor in 4 of its 6 comparisons and MutationAssessor higher in 1; 1 is a tie.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MutPred in 9 of 10 comparisons; 2 of 2 other studies rank MutPred above PrimateAI.
4 papers rank MutPred above PrimateAI in 17 of their 20 comparisons and PrimateAI higher in 3. Comprehensive assessment… ranks PrimateAI above MutPred in 9 of its 10 comparisons and MutPred higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above MetaSVM.
2 papers rank FATHMM-XF above MetaSVM in 22 of their 26 comparisons and MetaSVM higher in 4. Livesey and Marsh 2025 ranks MetaSVM above FATHMM-XF in all 7 of its comparisons.
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks SpliceAI above Pangolin in 27 of 28 comparisons; 3 of 3 other studies rank Pangolin above SpliceAI.
7 papers rank Pangolin above SpliceAI in 29 of their 72 comparisons and SpliceAI higher in 18; 25 are ties. Genome modelling and design across all… ranks SpliceAI above Pangolin in 27 of its 28 comparisons and Pangolin higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above PrimateAI in 6 of 6 comparisons; 2 of 2 other studies rank PrimateAI above SIFT.
4 papers rank PrimateAI above SIFT in 25 of their 32 comparisons and SIFT higher in 7. Of those papers, ProteinGym ranks SIFT higher on its own (1 of 1). Livesey and Marsh 2025 ranks SIFT above PrimateAI in all 6 comparisons that are not ties (1 is a tie).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above REVEL in 6 of 6 comparisons; 2 of 2 other studies rank REVEL above BayesDel.
3 papers rank REVEL above BayesDel in 11 of their 14 comparisons and BayesDel higher in 2; 1 is a tie. Of those papers, ProteinGym ranks BayesDel higher on its own (1 of 1). An openly licensed benchmark… ranks BayesDel above REVEL in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaSVM in 16 of 20 comparisons; 2 of 2 other studies rank MetaSVM above LRT.
2 papers rank MetaSVM above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaSVM in 16 of its 20 comparisons and MetaSVM higher in 4.
GENEB leaderboard (leaderboard.json, 13 task-type aggregates) ranks Nucleotide Transformer above Nucleotide Transformer v2 in 37 of 44 comparisons; 2 of 2 other studies rank Nucleotide Transformer v2 above Nucleotide Transformer.
2 papers rank Nucleotide Transformer v2 above Nucleotide Transformer in 33 of their 46 comparisons and Nucleotide Transformer higher in 7; 6 are ties. GENEB leaderboard ranks Nucleotide Transformer above Nucleotide Transformer v2 in 37 of its 45 comparisons and Nucleotide Transformer v2 higher in 7; 1 is a tie.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutPred in 6 of 7 comparisons; 2 of 2 other studies rank MutPred above ESM-1b.
3 papers rank MutPred above ESM-1b in 8 of their 11 comparisons and ESM-1b higher in 1; 2 are ties. Of those papers, ProteinGym ranks ESM-1b higher on its own (1 of 1). Livesey and Marsh 2025 ranks ESM-1b above MutPred in 6 of its 7 comparisons and MutPred higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above MPC in 6 of 7 comparisons; 2 of 2 other studies rank MPC above MutationAssessor.
2 papers rank MPC above MutationAssessor in all 16 of their comparisons. Livesey and Marsh 2025 ranks MutationAssessor above MPC in 6 of its 7 comparisons and MPC higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above PrimateAI in 7 of 7 comparisons; 2 of 2 other studies rank PrimateAI above MetaSVM.
3 papers rank PrimateAI above MetaSVM in 25 of their 32 comparisons and MetaSVM higher in 5; 2 are ties. Of those papers, Clinical likelihood ratios and balanced… ranks MetaSVM higher on its own (4 of 6). Livesey and Marsh 2025 ranks MetaSVM above PrimateAI in all 7 of its comparisons.
A Phylogenetic Approach to Genomic Language Modeling ranks Caduceus above HyenaDNA in 9 of 12 comparisons; 2 of 2 other studies rank HyenaDNA above Caduceus.
5 papers rank HyenaDNA above Caduceus in 63 of their 133 comparisons and Caduceus higher in 49; 21 are ties. Of those papers, DNALONGBENCH ranks Caduceus higher on its own (16 of 27). A Phylogenetic Approach to Genomic… ranks Caduceus above HyenaDNA in 9 of its 12 comparisons and HyenaDNA higher in 3.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above SIFT in 18 of 20 comparisons; 2 of 2 other studies rank SIFT above LRT.
4 papers rank SIFT above LRT in 14 of their 17 comparisons and LRT higher in 2; 1 is a tie. Comprehensive assessment… ranks LRT above SIFT in 18 of its 20 comparisons and SIFT higher in 2.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above MVP in 6 of 6 comparisons; 3 of 3 other studies rank MVP above PolyPhen-2.
3 papers rank MVP above PolyPhen-2 in 18 of their 21 comparisons and PolyPhen-2 higher in 2; 1 is a tie. Livesey and Marsh 2025 ranks PolyPhen-2 above MVP in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks DANN above MetaLR in 10 of 10 comparisons; 2 of 2 other studies rank MetaLR above DANN.
3 papers rank MetaLR above DANN in 15 of their 19 comparisons and DANN higher in 3; 1 is a tie. Comprehensive assessment… ranks DANN above MetaLR in all 10 of its comparisons.
NABench: Large-Scale Benchmarks of Nucleotide Foundation Models for Fitness Prediction ranks RNA-FM above Nucleotide Transformer in 22 of 25 comparisons; 2 of 2 other studies rank Nucleotide Transformer above RNA-FM.
4 papers rank Nucleotide Transformer above RNA-FM in 42 of their 81 comparisons and RNA-FM higher in 25; 14 are ties. NABench ranks RNA-FM above Nucleotide Transformer in 22 of its 27 comparisons and Nucleotide Transformer higher in 3; 2 are ties.
BacBench LICENSE file at commit 64b7d45 ranks Nucleotide Transformer above Evo 2 in 10 of 10 comparisons; 4 of 4 other studies rank Evo 2 above Nucleotide Transformer.
6 papers rank Evo 2 above Nucleotide Transformer in 127 of their 147 comparisons and Nucleotide Transformer higher in 19; 1 is a tie. BacBench LICENSE file at commit 64b7d45 ranks Nucleotide Transformer above Evo 2 in all 10 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks GenoCanyon above phyloP in 3 of 3 comparisons; 2 of 2 other studies rank phyloP above GenoCanyon.
2 papers rank phyloP above GenoCanyon in 11 of their 13 comparisons and GenoCanyon higher in 1; 1 is a tie. Brandes et al. 2023 ranks GenoCanyon above phyloP in all 3 comparisons that are not ties (1 is a tie).
TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) ranks phyloP above Evo 2 in 2 of 2 comparisons; 2 of 2 other studies rank Evo 2 above phyloP.
3 papers rank Evo 2 above phyloP in 83 of their 107 comparisons and phyloP higher in 16; 8 are ties. TraitGym official metrics ranks phyloP above Evo 2 in all 2 of its comparisons.
Csendes et al. 2025, Supplementary Material 4 (Supplementary Tables 1-3) ranks scGPT above scFoundation in 15 of 16 comparisons; 2 of 2 other studies rank scFoundation above scGPT.
3 papers rank scFoundation above scGPT in 18 of their 25 comparisons and scGPT higher in 5; 2 are ties. Csendes et al. 2025 ranks scGPT above scFoundation in 15 of its 20 comparisons and scFoundation higher in 1; 4 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above VEST4 in 6 of 7 comparisons; 2 of 2 other studies rank VEST4 above ESM-1b.
2 papers rank VEST4 above ESM-1b in 7 of their 10 comparisons and ESM-1b higher in 1; 2 are ties. Livesey and Marsh 2025 ranks ESM-1b above VEST4 in 6 of its 7 comparisons and VEST4 higher in 1.