Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks LRT above MutPred in 5 of 6 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutPred above LRT in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank LRT above MutPred in 12 of their 24 comparisons and MutPred higher in 12. Of those papers, Livesey and Marsh 2025, ProteinGym rank MutPred higher on their own (7 of 7, 1 of 1).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above BayesDel in 3 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks BayesDel above FATHMM-MKL in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank FATHMM-MKL above BayesDel in 3 of their 12 comparisons and BayesDel higher in 8; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks BayesDel higher on its own (7 of 7).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MutScore in 6 of 6 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above REVEL in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutScore above REVEL in 6 of their 13 comparisons and REVEL higher in 6; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks REVEL higher on its own (6 of 7).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PrimateAI in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PrimateAI above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutFormer above PrimateAI in 7 of their 13 comparisons and PrimateAI higher in 6. Of those papers, An openly licensed benchmark… ranks PrimateAI higher on its own (6 of 6).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks MSA Transformer above ESM-1v in 16 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above MSA Transformer in 5 of 5 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ESM-1v above MSA Transformer in 8 of their 25 comparisons and MSA Transformer higher in 16; 1 is a tie. Of those papers, ProteinGym ranks MSA Transformer higher on its own (16 of 19).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MPC above REVEL in 10 of 10 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MPC in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank MPC above REVEL in 12 of their 24 comparisons and REVEL higher in 10; 2 are ties. Of those papers, Livesey and Marsh 2025, ProteinGym rank REVEL higher on their own (7 of 7, 1 of 1).
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above RNAErnie in 6 of 6 comparisons; RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Evo in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank RNAErnie above Evo in 4 of their 10 comparisons and Evo higher in 6. Of those papers, Zero-shot benchmarking of RNA language… ranks Evo higher on its own (6 of 6).
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks UTR-LM above RNA-MSM in 3 of 3 comparisons; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNA-MSM above UTR-LM in 8 of 8 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank RNA-MSM above UTR-LM in 8 of their 23 comparisons and UTR-LM higher in 3; 12 are ties. Of those papers, Benchmarking pre-trained genomic… ranks UTR-LM higher on its own (3 of 15).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VEST4 above BayesDel in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above VEST4 in 5 of 5 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank VEST4 above BayesDel in 8 of their 20 comparisons and BayesDel higher in 10; 2 are ties. Of those papers, An openly licensed benchmark…, ProteinGym rank BayesDel higher on their own (5 of 6, 1 of 1).
Transfer learning to leverage larger datasets for improved prediction of protein stability changes ranks MAESTRO above ACDC-NN in 5 of 5 comparisons; Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset ranks ACDC-NN above MAESTRO in 7 of 9 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ACDC-NN above MAESTRO in 7 of their 16 comparisons and MAESTRO higher in 7; 2 are ties. Of those papers, Transfer learning to leverage larger… ranks MAESTRO higher on its own (5 of 6).
de Vries et al. medRxiv preprint v1, Supplementary Tables 2-4 ranks Centrifuge above Genome Detective in 5 of 5 comparisons; Carbo et al. medRxiv preprint v1, Supplementary Tables 1 and 2 ranks Genome Detective above Centrifuge in 12 of 13 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Centrifuge above Genome Detective in 6 of their 35 comparisons and Genome Detective higher in 12; 17 are ties. Of those papers, Carbo et al. 2022 ranks Genome Detective higher on its own (12 of 28).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above EVmutation in 15 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks EVmutation above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank EVmutation above Tranception in 10 of their 26 comparisons and Tranception higher in 15; 1 is a tie. Of those papers, ProteinGym ranks Tranception higher on its own (15 of 19).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above VESPAl in 17 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VESPAl above Tranception in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Tranception above VESPAl in 18 of their 26 comparisons and VESPAl higher in 7; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks VESPAl higher on its own (6 of 7).
Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) ranks ESM-1b above ESM-1v in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above ESM-1b in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank ESM-1v above ESM-1b in 18 of their 31 comparisons and ESM-1b higher in 12; 1 is a tie. Of those papers, Genomic heterogeneity inflates… ranks ESM-1b higher on its own (5 of 5).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ESM-1v in 14 of 17 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ESM-1v above Tranception in 10 of their 26 comparisons and Tranception higher in 14; 2 are ties. Of those papers, ProteinGym ranks Tranception higher on its own (14 of 19).
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks Evo 2 above Mistral-DNA in 6 of 6 comparisons; BacBench LICENSE file at commit 64b7d45 ranks Mistral-DNA above Evo 2 in 9 of 10 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Evo 2 above Mistral-DNA in 7 of their 16 comparisons and Mistral-DNA higher in 9. Of those papers, BacBench LICENSE file at commit 64b7d45 ranks Mistral-DNA higher on its own (9 of 10).
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks Evo 2 above ESM-2 in 28 of 28 comparisons; BacBench LICENSE file at commit 64b7d45 ranks ESM-2 above Evo 2 in 10 of 10 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank ESM-2 above Evo 2 in 12 of their 43 comparisons and Evo 2 higher in 30; 1 is a tie. Of those papers, Genome modelling and design across all… ranks Evo 2 higher on its own (28 of 28).
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks ESM-1b above ESM-2 in 26 of 28 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above ESM-1b in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
5 papers rank ESM-2 above ESM-1b in 31 of their 87 comparisons and ESM-1b higher in 47; 9 are ties. Of those papers, Genome modelling and design across all… ranks ESM-1b higher on its own (26 of 28).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above Eigen in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks Eigen above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Eigen above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks SpliceBERT above RNA-MSM in 7 of 7 comparisons; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNA-MSM above SpliceBERT in 8 of 8 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank RNA-MSM above SpliceBERT in 8 of their 23 comparisons and SpliceBERT higher in 7; 8 are ties. Of those papers, Benchmarking pre-trained genomic… ranks SpliceBERT higher on its own (7 of 15).
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above AIDO.RNA in 5 of 5 comparisons; RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks AIDO.RNA above Evo in 3 of 3 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Evo above AIDO.RNA in 5 of their 10 comparisons and AIDO.RNA higher in 3; 2 are ties. Of those papers, RNAGym fitness leaderboard ranks AIDO.RNA higher on its own (3 of 4).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ProGen2 in 19 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ProGen2 above Tranception in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ProGen2 above Tranception in 6 of their 26 comparisons and Tranception higher in 19; 1 is a tie. Of those papers, ProteinGym ranks Tranception higher on its own (19 of 20).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks EVmutation above CARP in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above EVmutation in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank EVmutation above CARP in 16 of their 25 comparisons and CARP higher in 9. Of those papers, Livesey and Marsh 2025 ranks CARP higher on its own (5 of 6).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above VEST4 in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks VEST4 above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutFormer above VEST4 in 6 of their 13 comparisons and VEST4 higher in 7. Of those papers, An openly licensed benchmark… ranks VEST4 higher on its own (6 of 6).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MutationAssessor in 19 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
5 papers rank MutationAssessor above PrimateAI in 15 of their 40 comparisons and PrimateAI higher in 24; 1 is a tie. Of those papers, Comprehensive assessment… ranks PrimateAI higher on its own (19 of 20).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above LIST-S2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above MetaLR in 4 of 5 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank LIST-S2 above MetaLR in 4 of their 13 comparisons and MetaLR higher in 8; 1 is a tie. Of those papers, Livesey and Marsh 2025 ranks MetaLR higher on its own (7 of 7).
Transfer learning to leverage larger datasets for improved prediction of protein stability changes ranks mCSM above ThermoNet in 6 of 6 comparisons; Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset ranks ThermoNet above mCSM in 8 of 10 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank mCSM above ThermoNet in 8 of their 16 comparisons and ThermoNet higher in 8. Of those papers, Predicting protein stability changes… ranks ThermoNet higher on its own (8 of 10).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above Eigen in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Eigen above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank Eigen above PrimateAI in 13 of their 29 comparisons and PrimateAI higher in 9; 7 are ties. Of those papers, Comprehensive assessment… ranks PrimateAI higher on its own (5 of 10).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above MutScore in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above ESM-1b in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutScore above ESM-1b in 7 of their 13 comparisons and ESM-1b higher in 6. Of those papers, Livesey and Marsh 2025 ranks ESM-1b higher on its own (6 of 7).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above MutPred in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutPred above REVEL in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank MutPred above REVEL in 15 of their 29 comparisons and REVEL higher in 14. Of those papers, Livesey and Marsh 2025, Clinical likelihood ratios and balanced… rank REVEL higher on their own (6 of 7, 4 of 6).
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks GENA-LM above ChromBPNet in 3 of 4 comparisons; Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above GENA-LM in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ChromBPNet above GENA-LM in 5 of their 10 comparisons and GENA-LM higher in 3; 2 are ties. Of those papers, GUANinE v1.1 reveals complementarity… ranks GENA-LM higher on its own (3 of 6).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PolyPhen-2 above MetaLR in 17 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above PolyPhen-2 in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank MetaLR above PolyPhen-2 in 13 of their 38 comparisons and PolyPhen-2 higher in 22; 3 are ties. Of those papers, Comprehensive assessment… ranks PolyPhen-2 higher on its own (17 of 20).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above LIST-S2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above M-CAP in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank LIST-S2 above M-CAP in 5 of their 13 comparisons and M-CAP higher in 8. Of those papers, Livesey and Marsh 2025 ranks M-CAP higher on its own (7 of 7).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks SIFT above PANTHER in 3 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PANTHER above SIFT in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank PANTHER above SIFT in 6 of their 11 comparisons and SIFT higher in 4; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks SIFT higher on its own (3 of 5).
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks Borzoi above ChromBPNet in 5 of 5 comparisons; Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above Borzoi in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank ChromBPNet above Borzoi in 7 of their 21 comparisons and Borzoi higher in 12; 2 are ties. Of those papers, AlphaGenome Nature 2026, GUANinE v1.1 reveals complementarity… rank Borzoi higher on their own (7 of 10, 5 of 7).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks CADD above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above CADD in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Condel above CADD in 7 of their 12 comparisons and CADD higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks CADD higher on its own (4 of 5).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MutScore in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutFormer above MutScore in 6 of their 13 comparisons and MutScore higher in 7. Of those papers, An openly licensed benchmark… ranks MutScore higher on its own (6 of 6).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks REVEL above VEST4 in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks VEST4 above REVEL in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
5 papers rank REVEL above VEST4 in 14 of their 30 comparisons and VEST4 higher in 12; 4 are ties. Of those papers, An openly licensed benchmark… ranks VEST4 higher on its own (3 of 6).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaSVM above LIST-S2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks LIST-S2 above MetaSVM in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank LIST-S2 above MetaSVM in 3 of their 13 comparisons and MetaSVM higher in 8; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks MetaSVM higher on its own (7 of 7).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks MVP above MutationAssessor in 10 of 10 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above MVP in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank MutationAssessor above MVP in 11 of their 28 comparisons and MVP higher in 17. Of those papers, Comprehensive assessment…, An openly licensed benchmark… rank MVP higher on their own (10 of 10, 4 of 6).
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines ranks Geneformer above UCE (Universal Cell Embeddings) in 2 of 2 comparisons; Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13) ranks UCE (Universal Cell Embeddings) above Geneformer in 3 of 3 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank Geneformer above UCE (Universal Cell Embeddings) in 6 of their 18 comparisons and UCE (Universal Cell Embeddings) higher in 9; 3 are ties. Of those papers, Wu et al. 2025 ranks UCE (Universal Cell Embeddings) higher on its own (3 of 4).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above CADD in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank CADD above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).