LRT and MutPred: Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) disagree
- Low severity
- Not yet reviewed
- Two-study conflict
- unknown
Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes ranks LRT above MutPred in 5 of 6 comparisons; Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutPred above LRT in 7 of 7 comparisons. No other study gives a verdict, so neither is the outlier.
An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.
Which studies agree
4 papers rank LRT above MutPred in 12 of their 24 comparisons and MutPred higher in 12. Of those papers, Livesey and Marsh 2025, ProteinGym rank MutPred higher on their own (7 of 7, 1 of 1).
- Rank LRT higher (2): Comprehensive assessment…, Clinical likelihood ratios and balanced…
- Rank MutPred higher (2): Livesey and Marsh 2025, ProteinGym
- Mixed (0): none
Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.
| Paper | Role | Ranks LRT higher | Ranks MutPred higher | Ties |
|---|---|---|---|---|
| Comprehensive assessment of computational algorithms in predicting cancer driver mutations and 4 more records · DOI | Agrees | 7 | 3 | 0 |
| Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) and 1 more record · DOI | Agrees | 0 | 7 | 0 |
| Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes and 1 more record · DOI | Agrees | 5 | 1 | 0 |
| ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design and 6 more records · DOI | Agrees | 0 | 1 | 0 |
Comparisons in Comprehensive assessment… (10)
Comparisons in Livesey and Marsh 2025 (7)
| Protocol | Dataset | Metric | LRT | MutPred | Higher |
|---|---|---|---|---|---|
| Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense | Livesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins) | Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent | 18.8 | 65.1 | MutPred |
| Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS sets | Livesey and Marsh 2025 human DMS benchmark set (36 proteins) | Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets), percent | 13.6 | 47.9 | MutPred |
| Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS sets | Livesey and Marsh 2025 human DMS benchmark set (36 proteins) | Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets), percent | 1.22 | 14.4 | MutPred |
| Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays only | Livesey and Marsh 2025 human DMS benchmark set (36 proteins) | Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only), percent | 15.4 | 46.5 | MutPred |
| Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays only | Livesey and Marsh 2025 human DMS benchmark set (36 proteins) | Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only), percent | 12.3 | 49 | MutPred |
| Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions only | Livesey and Marsh 2025 human DMS benchmark set (36 proteins) | Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only), percent | 13.6 | 47.9 | MutPred |
| Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filled | Livesey and Marsh 2025 human DMS benchmark set (36 proteins) | Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent | 14.3 | 49.2 | MutPred |
Comparisons in Clinical likelihood ratios and balanced… (6)
Comparisons in ProteinGym (1)
| Protocol | Dataset | Metric | LRT | MutPred | Higher |
|---|---|---|---|---|---|
| ProteinGym clinical substitutions, zero-shot, average AUC (commit 144fe22) | ProteinGym clinical substitution benchmark at commit 144fe22 | AUROC (ProteinGym clinical substitutions, zero-shot, Average_AUC over proteins), unitless | 0.805 | 0.875 | MutPred |
What disagrees
- studies
- 2
Follow-up
Current state: Not yet reviewed.
No one has reviewed this finding yet.
How to help resolve this
If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.
Finding discrepancy-b8029ae0331d3e9a · All contested results · How findings are made and followed up