Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PHACTboost above BayesDel in 7 of 7 comparisons; other studies rank BayesDel above PHACTboost in 4 of 5.
An openly licensed benchmark… ranks BayesDel above PHACTboost in 4 of its 6 comparisons and PHACTboost higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks PHACTboost above BayesDel in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above MetaLR in 15 of 17 comparisons; other studies rank MetaLR above LRT in 12 of 13.
2 papers rank MetaLR above LRT in 12 of their 13 comparisons and LRT higher in 1. Comprehensive assessment… ranks LRT above MetaLR in 15 of its 20 comparisons and MetaLR higher in 2; 3 are ties.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks GERP above GenoCanyon in 4 of 5 comparisons; other studies rank GenoCanyon above GERP in 9 of 11.
2 papers rank GenoCanyon above GERP in 9 of their 11 comparisons and GERP higher in 2. Clinical likelihood ratios and balanced… ranks GERP above GenoCanyon in 4 of its 6 comparisons and GenoCanyon higher in 1; 1 is a tie.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MetaSVM above MutFormer in 4 of 5 comparisons; other studies rank MutFormer above MetaSVM in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above MetaSVM in all 7 of its comparisons. An openly licensed benchmark… ranks MetaSVM above MutFormer in 4 of its 6 comparisons and MutFormer higher in 1; 1 is a tie.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks GERP above FATHMM-MKL in 3 of 4 comparisons; other studies rank FATHMM-MKL above GERP in 6 of 6.
Livesey and Marsh 2025 ranks FATHMM-MKL above GERP in all 6 comparisons that are not ties (1 is a tie). Clinical likelihood ratios and balanced… ranks GERP above FATHMM-MKL in 3 of its 5 comparisons and FATHMM-MKL higher in 1; 1 is a tie.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above CADD in 6 of 7 comparisons; other studies rank CADD above MetaLR in 19 of 21.
3 papers rank CADD above MetaLR in 19 of their 21 comparisons and MetaLR higher in 2. Livesey and Marsh 2025 ranks MetaLR above CADD in 6 of its 7 comparisons and CADD higher in 1.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks CADD above Condel in 4 of 5 comparisons; other studies rank Condel above CADD in 6 of 7.
Livesey and Marsh 2025 ranks Condel above CADD in 6 of its 7 comparisons and CADD higher in 1. Clinical likelihood ratios and balanced… ranks CADD above Condel in 4 of its 5 comparisons and Condel higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above Tranception in 7 of 7 comparisons; other studies rank Tranception above ESM-1v in 14 of 17.
ProteinGym AMFR assay-level Spearman… ranks Tranception above ESM-1v in 14 of its 19 comparisons and ESM-1v higher in 3; 2 are ties. Livesey and Marsh 2025 ranks ESM-1v above Tranception in all 7 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM-XF above CADD in 10 of 10 comparisons; other studies rank CADD above FATHMM-XF in 12 of 12.
2 papers rank CADD above FATHMM-XF in all 12 comparisons that are not ties (1 is a tie). Comprehensive assessment… ranks FATHMM-XF above CADD in all 10 of its comparisons.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks DNABERT above RNA-MSM in 4 of 4 comparisons; other studies rank RNA-MSM above DNABERT in 8 of 8.
Zero-shot benchmarking of RNA language… ranks RNA-MSM above DNABERT in all 8 of its comparisons. Benchmarking pre-trained genomic… ranks DNABERT above RNA-MSM in all 4 comparisons that are not ties (8 are ties).
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MVP above ESM-1b in 2 of 2 comparisons; other studies rank ESM-1b above MVP in 13 of 13.
2 papers rank ESM-1b above MVP in all 13 of their comparisons. Brandes et al. 2023 ranks MVP above ESM-1b in all 2 comparisons that are not ties (2 are ties).
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above RiNALMo in 3 of 4 comparisons; other studies rank RiNALMo above RNAErnie in 29 of 35.
Zero-shot benchmarking of RNA language… ranks RiNALMo above RNAErnie in 29 of its 36 comparisons and RNAErnie higher in 6; 1 is a tie. RNAGym fitness leaderboard ranks RNAErnie above RiNALMo in 3 of its 4 comparisons and RiNALMo higher in 1.
Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above Caduceus in 4 of 4 comparisons; other studies rank Caduceus above ChromBPNet in 6 of 6.
GUANinE v1.1 reveals complementarity… ranks Caduceus above ChromBPNet in all 6 of its comparisons. Comparative Analysis of Deep Learning… ranks ChromBPNet above Caduceus in all 4 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks FATHMM-XF above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above FATHMM-XF in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above FATHMM-XF in all 7 of its comparisons. An openly licensed benchmark… ranks FATHMM-XF above MutFormer in all 6 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MVP above Condel in 4 of 5 comparisons; other studies rank Condel above MVP in 6 of 7.
Livesey and Marsh 2025 ranks Condel above MVP in 6 of its 7 comparisons and MVP higher in 1. Clinical likelihood ratios and balanced… ranks MVP above Condel in 4 of its 5 comparisons and Condel higher in 1.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM-XF above PolyPhen-2 in 14 of 16 comparisons; other studies rank PolyPhen-2 above FATHMM-XF in 9 of 11.
2 papers rank PolyPhen-2 above FATHMM-XF in 9 of their 12 comparisons and FATHMM-XF higher in 2; 1 is a tie. Comprehensive assessment… ranks FATHMM-XF above PolyPhen-2 in 14 of its 20 comparisons and PolyPhen-2 higher in 2; 4 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above MSA Transformer in 5 of 5 comparisons; other studies rank MSA Transformer above ESM-1v in 16 of 19.
ProteinGym AMFR assay-level Spearman… ranks MSA Transformer above ESM-1v in 16 of its 19 comparisons and ESM-1v higher in 3. Livesey and Marsh 2025 ranks ESM-1v above MSA Transformer in all 5 comparisons that are not ties (1 is a tie).
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNA-FM above RNAErnie in 28 of 36 comparisons; other studies rank RNAErnie above RNA-FM in 6 of 6.
2 papers rank RNAErnie above RNA-FM in all 6 comparisons that are not ties (13 are ties). Zero-shot benchmarking of RNA language… ranks RNA-FM above RNAErnie in 28 of its 36 comparisons and RNAErnie higher in 8.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above GERP in 7 of 7 comparisons; other studies rank GERP above Condel in 6 of 6.
Clinical likelihood ratios and balanced… ranks GERP above Condel in all 6 of its comparisons. Livesey and Marsh 2025 ranks Condel above GERP in all 7 of its comparisons.
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks AIDO.RNA above Evo in 4 of 4 comparisons; other studies rank Evo above AIDO.RNA in 5 of 5.
Zero-shot benchmarking of RNA language… ranks Evo above AIDO.RNA in all 5 comparisons that are not ties (1 is a tie). RNAGym fitness leaderboard ranks AIDO.RNA above Evo in all 4 of its comparisons.
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Evo in 4 of 4 comparisons; other studies rank Evo above RNAErnie in 6 of 6.
Zero-shot benchmarking of RNA language… ranks Evo above RNAErnie in all 6 of its comparisons. RNAGym fitness leaderboard ranks RNAErnie above Evo in all 4 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above ESM-1b in 16 of 19 comparisons; other studies rank ESM-1b above Tranception in 7 of 7.
Livesey and Marsh 2025 ranks ESM-1b above Tranception in all 7 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above ESM-1b in 16 of its 19 comparisons and ESM-1b higher in 3.
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) ranks Kraken2 above sourmash in 7 of 9 comparisons; other studies rank sourmash above Kraken2 in 24 of 29.
Evaluation of taxonomic classification… ranks sourmash above Kraken2 in 24 of its 48 comparisons and Kraken2 higher in 5; 19 are ties. Taxonomic classification of DNA… ranks Kraken2 above sourmash in 7 of its 11 comparisons and sourmash higher in 2; 2 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PHACTboost above MutScore in 7 of 7 comparisons; other studies rank MutScore above PHACTboost in 5 of 5.
An openly licensed benchmark… ranks MutScore above PHACTboost in all 5 comparisons that are not ties (1 is a tie). Livesey and Marsh 2025 ranks PHACTboost above MutScore in all 7 of its comparisons.
Transfer learning to leverage larger datasets for improved prediction of protein stability changes ranks MUpro above ThermoNet in 5 of 6 comparisons; other studies rank ThermoNet above MUpro in 9 of 9.
Predicting protein stability changes… ranks ThermoNet above MUpro in all 9 comparisons that are not ties (1 is a tie). Transfer learning to leverage larger… ranks MUpro above ThermoNet in 5 of its 6 comparisons and ThermoNet higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PrimateAI in 7 of 7 comparisons; other studies rank PrimateAI above MutFormer in 6 of 6.
An openly licensed benchmark… ranks PrimateAI above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above PrimateAI in all 7 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above EVmutation in 15 of 18 comparisons; other studies rank EVmutation above Tranception in 7 of 7.
Livesey and Marsh 2025 ranks EVmutation above Tranception in all 7 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above EVmutation in 15 of its 19 comparisons and EVmutation higher in 3; 1 is a tie.
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above CARP in 19 of 19 comparisons; other studies rank CARP above Tranception in 6 of 6.
Livesey and Marsh 2025 ranks CARP above Tranception in all 6 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above CARP in all 19 of its comparisons.
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks UTR-LM above DNABERT in 23 of 25 comparisons; other studies rank DNABERT above UTR-LM in 8 of 8.
Benchmarking pre-trained genomic… ranks DNABERT above UTR-LM in all 8 comparisons that are not ties (4 are ties). Zero-shot benchmarking of RNA language… ranks UTR-LM above DNABERT in 23 of its 30 comparisons and DNABERT higher in 2; 5 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above BayesDel in 6 of 7 comparisons; other studies rank BayesDel above ESM-1b in 14 of 14.
2 papers rank BayesDel above ESM-1b in all 14 of their comparisons. Livesey and Marsh 2025 ranks ESM-1b above BayesDel in 6 of its 7 comparisons and BayesDel higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above LIST-S2 in 7 of 7 comparisons; other studies rank LIST-S2 above MetaLR in 4 of 5.
An openly licensed benchmark… ranks LIST-S2 above MetaLR in 4 of its 6 comparisons and MetaLR higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks MetaLR above LIST-S2 in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PolyPhen-2 in 6 of 6 comparisons; other studies rank PolyPhen-2 above MutFormer in 6 of 6.
An openly licensed benchmark… ranks PolyPhen-2 above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above PolyPhen-2 in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PROVEAN above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above PROVEAN in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above PROVEAN in all 7 of its comparisons. An openly licensed benchmark… ranks PROVEAN above MutFormer in all 6 of its comparisons.
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above LucaOne in 6 of 6 comparisons; other studies rank LucaOne above Evo in 40 of 45.
GENEB leaderboard ranks LucaOne above Evo in 40 of its 45 comparisons and Evo higher in 5. Zero-shot benchmarking of RNA language… ranks Evo above LucaOne in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above DEOGEN2 in 7 of 7 comparisons; other studies rank DEOGEN2 above MutFormer in 6 of 6.
An openly licensed benchmark… ranks DEOGEN2 above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above DEOGEN2 in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above PrimateAI in 7 of 7 comparisons; other studies rank PrimateAI above M-CAP in 24 of 30.
3 papers rank PrimateAI above M-CAP in 24 of their 30 comparisons and M-CAP higher in 6. Livesey and Marsh 2025 ranks M-CAP above PrimateAI in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks popEVE above VARITY in 6 of 6 comparisons; other studies rank VARITY above popEVE in 4 of 5.
An openly licensed benchmark… ranks VARITY above popEVE in 4 of its 6 comparisons and popEVE higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks popEVE above VARITY in all 6 comparisons that are not ties (1 is a tie).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM above MVP in 10 of 10 comparisons; other studies rank MVP above FATHMM in 11 of 12.
2 papers rank MVP above FATHMM in 11 of their 12 comparisons and FATHMM higher in 1. Comprehensive assessment… ranks FATHMM above MVP in all 10 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above SIFT in 7 of 7 comparisons; other studies rank SIFT above MutFormer in 5 of 6.
An openly licensed benchmark… ranks SIFT above MutFormer in 5 of its 6 comparisons and MutFormer higher in 1. Livesey and Marsh 2025 ranks MutFormer above SIFT in all 7 of its comparisons.
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks SpliceBERT above RNA-FM in 3 of 3 comparisons; other studies rank RNA-FM above SpliceBERT in 31 of 36.
Zero-shot benchmarking of RNA language… ranks RNA-FM above SpliceBERT in 31 of its 36 comparisons and SpliceBERT higher in 5. Benchmarking pre-trained genomic… ranks SpliceBERT above RNA-FM in all 3 comparisons that are not ties (12 are ties).
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaLR above ESM-1b in 2 of 2 comparisons; other studies rank ESM-1b above MetaLR in 13 of 13.
2 papers rank ESM-1b above MetaLR in all 13 of their comparisons. Brandes et al. 2023 ranks MetaLR above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) ranks ESM-1b above ESM-2 in 26 of 28 comparisons; other studies rank ESM-2 above ESM-1b in 20 of 25.
2 papers rank ESM-2 above ESM-1b in 20 of their 26 comparisons and ESM-1b higher in 5; 1 is a tie. Genome modelling and design across all… ranks ESM-1b above ESM-2 in 26 of its 28 comparisons and ESM-2 higher in 2.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VESPAl above Tranception in 6 of 7 comparisons; other studies rank Tranception above VESPAl in 18 of 19.
ProteinGym AMFR assay-level Spearman… ranks Tranception above VESPAl in 18 of its 19 comparisons and VESPAl higher in 1. Livesey and Marsh 2025 ranks VESPAl above Tranception in 6 of its 7 comparisons and Tranception higher in 1.
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons; other studies rank Nucleotide Transformer above RNAErnie in 7 of 7.
Benchmarking pre-trained genomic… ranks Nucleotide Transformer above RNAErnie in all 7 comparisons that are not ties (8 are ties). RNAGym fitness leaderboard ranks RNAErnie above Nucleotide Transformer in all 4 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks MetaSVM above ESM-1b in 2 of 2 comparisons; other studies rank ESM-1b above MetaSVM in 13 of 13.
2 papers rank ESM-1b above MetaSVM in all 13 of their comparisons. Brandes et al. 2023 ranks MetaSVM above ESM-1b in all 2 comparisons that are not ties (2 are ties).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PANTHER above GERP in 6 of 6 comparisons; other studies rank GERP above PANTHER in 4 of 5.
Clinical likelihood ratios and balanced… ranks GERP above PANTHER in 4 of its 6 comparisons and PANTHER higher in 1; 1 is a tie. Livesey and Marsh 2025 ranks PANTHER above GERP in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MetaRNN in 3 of 4 comparisons; other studies rank MetaRNN above MutScore in 7 of 7.
Livesey and Marsh 2025 ranks MetaRNN above MutScore in all 7 of its comparisons. An openly licensed benchmark… ranks MutScore above MetaRNN in 3 of its 6 comparisons and MetaRNN higher in 1; 2 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above M-CAP in 22 of 25.
2 papers rank FATHMM-XF above M-CAP in 22 of their 26 comparisons and M-CAP higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks M-CAP above FATHMM-XF in all 7 of its comparisons.
Achieving robust somatic mutation detection with deep learning models derived from reference data sets of a cancer sample ranks Octopus above Mutect2 in 154 of 154 comparisons; other studies rank Mutect2 above Octopus in 14 of 14.
ClairS-TO ranks Mutect2 above Octopus in all 14 of its comparisons. Achieving robust somatic mutation… ranks Octopus above Mutect2 in all 154 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above ESM-1b in 7 of 7 comparisons; other studies rank ESM-1b above MutFormer in 6 of 6.
An openly licensed benchmark… ranks ESM-1b above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above ESM-1b in all 7 of its comparisons.