Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.
These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.
A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.
Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset ranks ThermoNet above mCSM in 8 of 10 comparisons; other studies rank mCSM above ThermoNet in 6 of 6.
Transfer learning to leverage larger… ranks mCSM above ThermoNet in all 6 of its comparisons. Predicting protein stability changes… ranks ThermoNet above mCSM in 8 of its 10 comparisons and mCSM higher in 2.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-2 above CADD in 6 of 7 comparisons; other studies rank CADD above ESM-2 in 14 of 14.
Genome modelling and design across all… ranks CADD above ESM-2 in all 14 of its comparisons. Livesey and Marsh 2025 ranks ESM-2 above CADD in 6 of its 7 comparisons and CADD higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks SIFT above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above SIFT in 22 of 25.
2 papers rank FATHMM-XF above SIFT in 22 of their 26 comparisons and SIFT higher in 3; 1 is a tie. Livesey and Marsh 2025 ranks SIFT above FATHMM-XF in all 7 of its comparisons.
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks MetaSVM above SuSPect in 5 of 6 comparisons; other studies rank SuSPect above MetaSVM in 7 of 7.
Livesey and Marsh 2025 ranks SuSPect above MetaSVM in all 7 of its comparisons. Clinical likelihood ratios and balanced… ranks MetaSVM above SuSPect in 5 of its 6 comparisons and SuSPect higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ESM-1b above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above ESM-1b in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above ESM-1b in all 7 of its comparisons. An openly licensed benchmark… ranks ESM-1b above MutFormer in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks CADD above Eigen in 6 of 6 comparisons; other studies rank Eigen above CADD in 19 of 22.
3 papers rank Eigen above CADD in 19 of their 22 comparisons and CADD higher in 3. An openly licensed benchmark… ranks CADD above Eigen in all 6 of its comparisons.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks ClinPred above MetaRNN in 2 of 2 comparisons; other studies rank MetaRNN above ClinPred in 7 of 7.
2 papers rank MetaRNN above ClinPred in all 7 comparisons that are not ties (1 is a tie). An openly licensed benchmark… ranks ClinPred above MetaRNN in all 2 comparisons that are not ties (4 are ties).
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutFormer above PROVEAN in 7 of 7 comparisons; other studies rank PROVEAN above MutFormer in 6 of 6.
An openly licensed benchmark… ranks PROVEAN above MutFormer in all 6 of its comparisons. Livesey and Marsh 2025 ranks MutFormer above PROVEAN in all 7 of its comparisons.
ProteinGym AMFR assay-level Spearman comparison ranks VESPA above CARP in 18 of 19 comparisons; other studies rank CARP above VESPA in 6 of 6.
Livesey and Marsh 2025 ranks CARP above VESPA in all 6 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks VESPA above CARP in 18 of its 19 comparisons and CARP higher in 1.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above MVP in 7 of 7 comparisons; other studies rank MVP above MutationAssessor in 17 of 21.
3 papers rank MVP above MutationAssessor in 17 of their 21 comparisons and MutationAssessor higher in 4. Livesey and Marsh 2025 ranks MutationAssessor above MVP in all 7 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks ESM-1b above Tranception in 7 of 7 comparisons; other studies rank Tranception above ESM-1b in 16 of 19.
ProteinGym AMFR assay-level Spearman… ranks Tranception above ESM-1b in 16 of its 19 comparisons and ESM-1b higher in 3. Livesey and Marsh 2025 ranks ESM-1b above Tranception in all 7 of its comparisons.
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods ranks SiPhy above GERP in 4 of 4 comparisons; other studies rank GERP above SiPhy in 12 of 13.
2 papers rank GERP above SiPhy in 12 of their 13 comparisons and SiPhy higher in 1. Brandes et al. 2023 ranks SiPhy above GERP in all 4 of its comparisons.
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MetaLR above MVP in 6 of 7 comparisons; other studies rank MVP above MetaLR in 18 of 21.
3 papers rank MVP above MetaLR in 18 of their 21 comparisons and MetaLR higher in 3. Livesey and Marsh 2025 ranks MetaLR above MVP in 6 of its 7 comparisons and MVP higher in 1.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutationAssessor above MutFormer in 5 of 5 comparisons; other studies rank MutFormer above MutationAssessor in 7 of 7.
Livesey and Marsh 2025 ranks MutFormer above MutationAssessor in all 7 of its comparisons. An openly licensed benchmark… ranks MutationAssessor above MutFormer in all 5 comparisons that are not ties (1 is a tie).
ProteinGym AMFR assay-level Spearman comparison ranks Tranception above ESM-1v in 14 of 17 comparisons; other studies rank ESM-1v above Tranception in 7 of 7.
Livesey and Marsh 2025 ranks ESM-1v above Tranception in all 7 of its comparisons. ProteinGym AMFR assay-level Spearman… ranks Tranception above ESM-1v in 14 of its 19 comparisons and ESM-1v higher in 3; 2 are ties.
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks MutPred above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above MutPred in 6 of 7.
Livesey and Marsh 2025 ranks MutFormer above MutPred in 6 of its 7 comparisons and MutPred higher in 1. An openly licensed benchmark… ranks MutPred above MutFormer in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks LRT above M-CAP in 16 of 20 comparisons; other studies rank M-CAP above LRT in 9 of 11.
2 papers rank M-CAP above LRT in 9 of their 11 comparisons and LRT higher in 2. Comprehensive assessment… ranks LRT above M-CAP in 16 of its 20 comparisons and M-CAP higher in 4.
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing ranks MuSE above NeuSomatic in 3 of 3 comparisons; other studies rank NeuSomatic above MuSE in 86 of 100.
2 papers rank NeuSomatic above MuSE in 86 of their 101 comparisons and MuSE higher in 14; 1 is a tie. A benchmarking study of individual… ranks MuSE above NeuSomatic in all 3 comparisons that are not ties (3 are ties).
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers ranks BayesDel above MutFormer in 6 of 6 comparisons; other studies rank MutFormer above BayesDel in 6 of 7.
Livesey and Marsh 2025 ranks MutFormer above BayesDel in 6 of its 7 comparisons and BayesDel higher in 1. An openly licensed benchmark… ranks BayesDel above MutFormer in all 6 of its comparisons.
Comprehensive assessment of computational algorithms in predicting cancer driver mutations ranks FATHMM above CADD in 10 of 10 comparisons; other studies rank CADD above FATHMM in 11 of 13.
3 papers rank CADD above FATHMM in 11 of their 13 comparisons and FATHMM higher in 2. Comprehensive assessment… ranks FATHMM above CADD in all 10 of its comparisons.