Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PROVEAN above MutationAssessor in 6 of 6 comparisons; 2 of 2 other studies rank MutationAssessor above PROVEAN.
5 papers rank MutationAssessor above PROVEAN in 21 of their 38 comparisons and PROVEAN higher in 15; 2 are ties. Of those papers, ProteinGym ranks PROVEAN higher on its own (1 of 1). An openly licensed benchmark… ranks PROVEAN above MutationAssessor in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above FATHMM-XF in 7 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above MutationAssessor.
2 papers rank FATHMM-XF above MutationAssessor in 21 of their 26 comparisons and MutationAssessor higher in 5. Livesey and Marsh 2025 ranks MutationAssessor above FATHMM-XF in all 7 of its comparisons.
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks GENA-LM above DNABERT-2 in 6 of 6 comparisons; 2 of 2 other studies rank DNABERT-2 above GENA-LM.
4 papers rank DNABERT-2 above GENA-LM in 62 of their 112 comparisons and GENA-LM higher in 26; 24 are ties. GUANinE v1.1 reveals complementarity… ranks GENA-LM above DNABERT-2 in all 6 of its comparisons.
BacBench LICENSE file at commit 64b7d45 ranks Evo above Evo 2 in 8 of 10 comparisons; 2 of 2 other studies rank Evo 2 above Evo.
2 papers rank Evo 2 above Evo in all 94 of their comparisons. BacBench LICENSE file at commit 64b7d45 ranks Evo above Evo 2 in 8 of its 10 comparisons and Evo 2 higher in 2.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks GenoCanyon above MetaSVM in 6 of 8 comparisons; 2 of 2 other studies rank MetaSVM above GenoCanyon.
2 papers rank MetaSVM above GenoCanyon in 12 of their 13 comparisons and GenoCanyon higher in 1. Comprehensive assessment… ranks GenoCanyon above MetaSVM in 6 of its 10 comparisons and MetaSVM higher in 2; 2 are ties.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above CADD in 6 of 6 comparisons; 2 of 2 other studies rank CADD above PolyPhen-2.
4 papers rank CADD above PolyPhen-2 in 19 of their 22 comparisons and PolyPhen-2 higher in 2; 1 is a tie. Livesey and Marsh 2025 ranks PolyPhen-2 above CADD in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks M-CAP above MVP in 6 of 7 comparisons; 2 of 2 other studies rank MVP above M-CAP.
3 papers rank MVP above M-CAP in 16 of their 20 comparisons and M-CAP higher in 1; 3 are ties. Livesey and Marsh 2025 ranks M-CAP above MVP in 6 of its 7 comparisons and MVP higher in 1.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MVP in 10 of 10 comparisons; 2 of 2 other studies rank MVP above PrimateAI.
3 papers rank MVP above PrimateAI in 13 of their 18 comparisons and PrimateAI higher in 3; 2 are ties. Comprehensive assessment… ranks PrimateAI above MVP in all 10 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above PrimateAI in 6 of 6 comparisons; 2 of 2 other studies rank PrimateAI above PolyPhen-2.
3 papers rank PrimateAI above PolyPhen-2 in 23 of their 32 comparisons and PolyPhen-2 higher in 8; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks PolyPhen-2 higher on its own (4 of 6). Livesey and Marsh 2025 ranks PolyPhen-2 above PrimateAI in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks DANN above M-CAP in 5 of 5 comparisons; 2 of 2 other studies rank M-CAP above DANN.
3 papers rank M-CAP above DANN in 13 of their 17 comparisons and DANN higher in 3; 1 is a tie. Comprehensive assessment… ranks DANN above M-CAP in all 5 comparisons that are not ties (5 are ties).
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SiPhy above phyloP in 3 of 4 comparisons; 2 of 2 other studies rank phyloP above SiPhy.
2 papers rank phyloP above SiPhy in all 13 of their comparisons. Brandes et al. 2023 ranks SiPhy above phyloP in 3 of its 4 comparisons and phyloP higher in 1.
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx) ranks scGPT above Geneformer in 83 of 89 comparisons; 2 of 2 other studies rank Geneformer above scGPT.
4 papers rank Geneformer above scGPT in 10 of their 19 comparisons and scGPT higher in 5; 4 are ties. Of those papers, Parameter-Efficient Fine-Tuning… ranks scGPT higher on its own (1 of 1). Kedzierska et al. 2025 ranks scGPT above Geneformer in 83 of its 97 comparisons and Geneformer higher in 6; 8 are ties.
TraitGym official metrics: AUPRC by chromosome, weighted average (Hugging Face dataset songlab/TraitGym) ranks HyenaDNA above Nucleotide Transformer in 2 of 2 comparisons; 4 of 4 other studies rank Nucleotide Transformer above HyenaDNA.
5 papers rank Nucleotide Transformer above HyenaDNA in 77 of their 82 comparisons and HyenaDNA higher in 5. TraitGym official metrics ranks HyenaDNA above Nucleotide Transformer in all 2 of its comparisons.
RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks Evo above Nucleotide Transformer in 3 of 4 comparisons; 2 of 2 other studies rank Nucleotide Transformer above Evo.
3 papers rank Nucleotide Transformer above Evo in 72 of their 145 comparisons and Evo higher in 65; 8 are ties. Of those papers, Genome modelling and design across all… ranks Evo higher on its own (61 of 90). RNAGym fitness leaderboard ranks Evo above Nucleotide Transformer in 3 of its 4 comparisons and Nucleotide Transformer higher in 1.
BacBench LICENSE file at commit 64b7d45 ranks DNABERT-2 above Evo 2 in 10 of 10 comparisons; 3 of 3 other studies rank Evo 2 above DNABERT-2.
4 papers rank Evo 2 above DNABERT-2 in 63 of their 80 comparisons and DNABERT-2 higher in 14; 3 are ties. BacBench LICENSE file at commit 64b7d45 ranks DNABERT-2 above Evo 2 in all 10 of its comparisons.
Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks Evo above DNABERT-2 in 6 of 6 comparisons; 2 of 2 other studies rank DNABERT-2 above Evo.
2 papers rank DNABERT-2 above Evo in 45 of their 55 comparisons and Evo higher in 10. Zero-shot benchmarking of RNA language… ranks Evo above DNABERT-2 in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above FATHMM-XF in 6 of 7 comparisons; 2 of 2 other studies rank FATHMM-XF above SIFT.
2 papers rank FATHMM-XF above SIFT in 25 of their 26 comparisons and SIFT higher in 1. Livesey and Marsh 2025 ranks SIFT above FATHMM-XF in 6 of its 7 comparisons and FATHMM-XF higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above Eigen in 6 of 6 comparisons; 2 of 2 other studies rank Eigen above CADD.
3 papers rank Eigen above CADD in 19 of their 22 comparisons and CADD higher in 3. An openly licensed benchmark… ranks CADD above Eigen in all 6 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above FATHMM-XF in 6 of 6 comparisons; 2 of 2 other studies rank FATHMM-XF above PolyPhen-2.
2 papers rank FATHMM-XF above PolyPhen-2 in 20 of their 26 comparisons and PolyPhen-2 higher in 1; 5 are ties. Livesey and Marsh 2025 ranks PolyPhen-2 above FATHMM-XF in all 6 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SiPhy above GERP in 4 of 4 comparisons; 2 of 2 other studies rank GERP above SiPhy.
2 papers rank GERP above SiPhy in 11 of their 13 comparisons and SiPhy higher in 1; 1 is a tie. Brandes et al. 2023 ranks SiPhy above GERP in all 4 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above MVP in 6 of 7 comparisons; 2 of 2 other studies rank MVP above MetaLR.
3 papers rank MVP above MetaLR in 18 of their 21 comparisons and MetaLR higher in 3. Livesey and Marsh 2025 ranks MetaLR above MVP in 6 of its 7 comparisons and MVP higher in 1.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks PrimateAI above PROVEAN in 5 of 6 comparisons; 2 of 2 other studies rank PROVEAN above PrimateAI.
4 papers rank PROVEAN above PrimateAI in 20 of their 34 comparisons and PrimateAI higher in 12; 2 are ties. Clinical likelihood ratios and balanced… ranks PrimateAI above PROVEAN in 5 of its 6 comparisons and PROVEAN higher in 1.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above MetaLR in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above MetaSVM in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank MetaLR above MetaSVM in 15 of their 39 comparisons and MetaSVM higher in 19; 5 are ties. Of those papers, Comprehensive assessment…, Clinical likelihood ratios and balanced… rank MetaSVM higher on their own (12 of 20, 4 of 6).
Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) ranks ESM-2 above ESM-1v in 3 of 3 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above ESM-2 in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank ESM-1v above ESM-2 in 14 of their 31 comparisons and ESM-2 higher in 11; 6 are ties. Of those papers, Genomic heterogeneity inflates… ranks ESM-2 higher on its own (3 of 5).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above DEOGEN2 in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks DEOGEN2 above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank DEOGEN2 above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks VESPAl above CARP in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above VESPAl in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank CARP above VESPAl in 10 of their 25 comparisons and VESPAl higher in 15. Of those papers, ProteinGym ranks VESPAl higher on its own (15 of 19).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaLR above SuSPect in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SuSPect above MetaLR in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank SuSPect above MetaLR in 7 of their 13 comparisons and MetaLR higher in 5; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks MetaLR higher on its own (4 of 6).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks PrimateAI above MetaLR in 19 of 20 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above PrimateAI in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
4 papers rank MetaLR above PrimateAI in 14 of their 39 comparisons and PrimateAI higher in 25. Of those papers, Comprehensive assessment…, An openly licensed benchmark… rank PrimateAI higher on their own (19 of 20, 4 of 6).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks VARITY above MetaRNN in 6 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MetaRNN above VARITY in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank MetaRNN above VARITY in 5 of their 14 comparisons and VARITY higher in 7; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks VARITY higher on its own (6 of 7).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above MPC in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MPC above MutFormer in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MPC above MutFormer in 6 of their 13 comparisons and MutFormer higher in 7. Of those papers, Livesey and Marsh 2025 ranks MutFormer higher on its own (7 of 7).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ESM-1b in 15 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ESM-1b above Tranception in 11 of their 26 comparisons and Tranception higher in 15. Of those papers, ProteinGym ranks Tranception higher on its own (15 of 19).
Shen et al. 2025, Chemical Science (Europe PMC full text) ranks Boltz-2 above Protenix in 26 of 28 comparisons; POISK Explorer project site ranks Protenix above Boltz-2 in 10 of 10 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Boltz-2 above Protenix in 26 of their 42 comparisons and Protenix higher in 12; 4 are ties. Of those papers, POISK Explorer project site ranks Protenix higher on its own (10 of 10).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks FATHMM-MKL above SNAP2 in 3 of 4 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SNAP2 above FATHMM-MKL in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank SNAP2 above FATHMM-MKL in 8 of their 12 comparisons and FATHMM-MKL higher in 3; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks FATHMM-MKL higher on its own (3 of 5).
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks Nucleotide Transformer above RNAErnie in 7 of 7 comparisons; RNAGym fitness leaderboard (v0.1.1), leaderboard/fitness/README.md ranks RNAErnie above Nucleotide Transformer in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Nucleotide Transformer above RNAErnie in 7 of their 19 comparisons and RNAErnie higher in 4; 8 are ties. Of those papers, RNAGym fitness leaderboard ranks RNAErnie higher on its own (4 of 4).
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks DNABERT-2 above PhyloGPN in 6 of 6 comparisons; Genomic heterogeneity inflates the performance of variant pathogenicity predictions (Lu, Liu, Lin and Brandes, bioRxiv 2025, v2) ranks PhyloGPN above DNABERT-2 in 14 of 14 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank DNABERT-2 above PhyloGPN in 6 of their 20 comparisons and PhyloGPN higher in 14. Of those papers, Genomic heterogeneity inflates… ranks PhyloGPN higher on its own (14 of 14).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above SIFT in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks SIFT above MutFormer in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutFormer above SIFT in 8 of their 13 comparisons and SIFT higher in 3; 2 are ties. Of those papers, An openly licensed benchmark… ranks SIFT higher on its own (3 of 6).
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks DNABERT above RNA-MSM in 4 of 4 comparisons; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks RNA-MSM above DNABERT in 8 of 8 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank DNABERT above RNA-MSM in 4 of their 20 comparisons and RNA-MSM higher in 8; 8 are ties. Of those papers, Zero-shot benchmarking of RNA language… ranks RNA-MSM higher on its own (8 of 8).
GUANinE v1.1 reveals complementarity of supervised and genomic language models ranks Caduceus above ChromBPNet in 6 of 6 comparisons; Comparative Analysis of Deep Learning Models for Predicting Causative Regulatory Variants ranks ChromBPNet above Caduceus in 4 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ChromBPNet above Caduceus in 4 of their 10 comparisons and Caduceus higher in 6. Of those papers, GUANinE v1.1 reveals complementarity… ranks Caduceus higher on its own (6 of 6).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks VESPA above CARP in 18 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above VESPA in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank CARP above VESPA in 7 of their 25 comparisons and VESPA higher in 18. Of those papers, ProteinGym ranks VESPA higher on its own (18 of 19).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above Condel in 4 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above MetaSVM in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Condel above MetaSVM in 8 of their 13 comparisons and MetaSVM higher in 4; 1 is a tie. Of those papers, Clinical likelihood ratios and balanced… ranks MetaSVM higher on its own (4 of 6).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above ESM-2 in 16 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above Tranception in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ESM-2 above Tranception in 9 of their 26 comparisons and Tranception higher in 16; 1 is a tie. Of those papers, ProteinGym ranks Tranception higher on its own (16 of 19).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks DeepSequence above ESM-1v in 13 of 17 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1v above DeepSequence in 6 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank ESM-1v above DeepSequence in 10 of their 26 comparisons and DeepSequence higher in 14; 2 are ties. Of those papers, ProteinGym ranks DeepSequence higher on its own (13 of 19).
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks SIFT above Condel in 5 of 5 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks Condel above SIFT in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank Condel above SIFT in 7 of their 12 comparisons and SIFT higher in 5. Of those papers, Clinical likelihood ratios and balanced… ranks SIFT higher on its own (5 of 5).
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above PolyPhen-2 in 14 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks PolyPhen-2 above LRT in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank LRT above PolyPhen-2 in 17 of their 32 comparisons and PolyPhen-2 higher in 13; 2 are ties. Of those papers, Livesey and Marsh 2025 ranks PolyPhen-2 higher on its own (6 of 6).
Benchmarking pre-trained genomic language models for RNA sequence-related predictive applications ranks DNABERT above UTR-LM in 8 of 8 comparisons; Zero-shot benchmarking of RNA language models in structural, functional, and evolutionary learning ranks UTR-LM above DNABERT in 25 of 26 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank DNABERT above UTR-LM in 9 of their 42 comparisons and UTR-LM higher in 25; 8 are ties. Of those papers, Zero-shot benchmarking of RNA language… ranks UTR-LM higher on its own (25 of 30).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks ESM-1b above CARP in 14 of 18 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above ESM-1b in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank CARP above ESM-1b in 10 of their 25 comparisons and ESM-1b higher in 14; 1 is a tie. Of those papers, ProteinGym ranks ESM-1b higher on its own (14 of 19).
ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design ranks Tranception above CARP in 19 of 19 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks CARP above Tranception in 6 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank CARP above Tranception in 6 of their 25 comparisons and Tranception higher in 19. Of those papers, ProteinGym ranks Tranception higher on its own (19 of 19).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaRNN above MutScore in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutScore above MetaRNN in 3 of 4 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MetaRNN above MutScore in 8 of their 13 comparisons and MutScore higher in 3; 2 are ties. Of those papers, An openly licensed benchmark… ranks MutScore higher on its own (3 of 6).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PolyPhen-2 in 6 of 6 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks PolyPhen-2 above MutFormer in 5 of 6 comparisons. No other study gives a verdict, so neither is the outlier.
2 papers rank MutFormer above PolyPhen-2 in 7 of their 12 comparisons and PolyPhen-2 higher in 5. Of those papers, An openly licensed benchmark… ranks PolyPhen-2 higher on its own (5 of 6).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaRNN above ClinPred in 7 of 7 comparisons; An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above MetaRNN in 2 of 2 comparisons. No other study gives a verdict, so neither is the outlier.
3 papers rank ClinPred above MetaRNN in 2 of their 14 comparisons and MetaRNN higher in 7; 5 are ties. Of those papers, Livesey and Marsh 2025 ranks MetaRNN higher on its own (7 of 7).