rewirebio.iobenchmarks
Contested result

FATHMM-XF and MutationAssessor: Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • Different tasks across studies

Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML) ranks MutationAssessor above FATHMM-XF in 7 of 7 comparisons; other studies rank FATHMM-XF above MutationAssessor in 21 of 26.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

2 papers rank FATHMM-XF above MutationAssessor in 21 of their 26 comparisons and MutationAssessor higher in 5. Livesey and Marsh 2025 ranks MutationAssessor above FATHMM-XF in all 7 of its comparisons.

Papers grouped by which system they rank higher: FATHMM-XF or MutationAssessorRank FATHMM-XF higher: Comprehensive assessment…, An openly licensed benchmark…. Rank MutationAssessor higher: Livesey and Marsh 2025 (disagrees). Mixed: none.Rank FATHMM-XF higherMixedRank MutationAssessor higherComprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutationsAn openly licensedbenchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer driversLivesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)
  • Rank FATHMM-XF higher (2): Comprehensive assessment…, An openly licensed benchmark…
  • Rank MutationAssessor higher (1): Livesey and Marsh 2025 (disagrees)
  • Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores FATHMM-XF or MutationAssessor higherLivesey and Marsh 2025 (disagrees): FATHMM-XF higher in 0, MutationAssessor higher in 7, 0 ties. Comprehensive assessment…: FATHMM-XF higher in 16, MutationAssessor higher in 4, 0 ties. An openly licensed benchmark…: FATHMM-XF higher in 5, MutationAssessor higher in 1, 0 ties.Disagrees: Livesey and Marsh 2025Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)7 MutationAssessor higher7Comprehensive assessment…Comprehensive assessment of computational algorithms in predicting cancer driver mutations16 FATHMM-XF higher164 MutationAssessor higher4An openly licensed benchmark…An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers5 FATHMM-XF higher51 MutationAssessor higher1
FATHMM-XF higherTieMutationAssessor higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in Livesey and Marsh 2025 (7)
ProtocolDatasetMetricFATHMM-XFMutationAssessorHigher
Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missenseLivesey and Marsh 2025 clinical set: ClinVar pathogenic versus gnomAD v4.1 missense (819 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each), percent3251.4MutationAssessor
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets), percent24.558.8MutationAssessor
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets), percent1.8310.3MutationAssessor
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only), percent27.159.3MutationAssessor
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only), percent22.658.3MutationAssessor
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only), percent24.558.8MutationAssessor
Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledLivesey and Marsh 2025 human DMS benchmark set (36 proteins)Pairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score), percent25.557.3MutationAssessor
Comparisons in Comprehensive assessment… (20)
ProtocolDatasetMetricFATHMM-XFMutationAssessorHigher
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Accuracy (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.660.58FATHMM-XF
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Negative predictive value (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.850.59FATHMM-XF
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Precision (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.60.57FATHMM-XF
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Recall (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.930.66FATHMM-XF
OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Specificity (OncoKB benchmark (positive set not stated), default categorical calls), fraction0.390.5MutationAssessor
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.650.59FATHMM-XF
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.650.59FATHMM-XF
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Precision (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.650.59FATHMM-XF
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Recall (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.650.61FATHMM-XF
OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)OncoKB-annotated somatic missense mutations (oncogenic, likely oncogenic and likely neutral)Specificity (OncoKB benchmark (positive set not stated), median-score threshold), fraction0.650.57FATHMM-XF
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Accuracy (cell viability drivers versus neutral, default categorical calls), fraction0.590.6MutationAssessor
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Negative predictive value (cell viability drivers versus neutral, default categorical calls), fraction0.720.59FATHMM-XF
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Precision (cell viability drivers versus neutral, default categorical calls), fraction0.560.6MutationAssessor
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Recall (cell viability drivers versus neutral, default categorical calls), fraction0.890.6FATHMM-XF
Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Specificity (cell viability drivers versus neutral, default categorical calls), fraction0.280.6MutationAssessor
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Accuracy (cell viability drivers versus neutral, median-score threshold), fraction0.630.6FATHMM-XF
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Negative predictive value (cell viability drivers versus neutral, median-score threshold), fraction0.630.6FATHMM-XF
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Precision (cell viability drivers versus neutral, median-score threshold), fraction0.630.6FATHMM-XF
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Recall (cell viability drivers versus neutral, median-score threshold), fraction0.630.6FATHMM-XF
Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Missense mutations with Ba/F3 and MCF10A cell viability calls (published and new)Specificity (cell viability drivers versus neutral, median-score threshold), fraction0.630.6FATHMM-XF
Comparisons in An openly licensed benchmark… (6)
ProtocolDatasetMetricFATHMM-XFMutationAssessorHigher
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAUROC (CGC missense, all genes), unitless0.8130.794FATHMM-XF
CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAverage precision (CGC missense, all genes), unitless0.5680.554FATHMM-XF
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAUROC (CGC missense, oncogenes (gain of function)), unitless0.810.769FATHMM-XF
CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAverage precision (CGC missense, oncogenes (gain of function)), unitless0.6440.604FATHMM-XF
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAUROC (CGC missense, tumour suppressors (loss of function)), unitless0.8570.824FATHMM-XF
CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Missense variants in 768 Cancer Gene Census genes with open oncogenicity labelsAverage precision (CGC missense, tumour suppressors (loss of function)), unitless0.4760.535MutationAssessor

What disagrees

Other studies
rank FATHMM-XF above MutationAssessor in 21 of 26 comparisons
This study
ranks MutationAssessor above FATHMM-XF in 7 of 7 comparisons
Compared
FATHMM-XF, MutationAssessor
Study that disagrees
Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML), Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8)
Other studies
7 studiesComprehensive assessment of computational algorithms in predicting cancer driver mutations, Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB), Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability), Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability), Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB), An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers, OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)
Rule
Ordering reversal; different tasks across studies
Model identity
Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (14)
StudyToolProtocol or benchmarkMetricValue
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missensePairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)32
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missensePairwise win rate (average pairwise win rate against all other VEPs, per-protein AUROC on shared variants, ClinVar P/LP versus gnomAD v4.1 missense, proteins with at least 10 of each)51.4
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets)24.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets)1.83
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, 36 human DMS sets)58.8
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, all 36 DMS setsPairwise win rate (average pairwise win rate against the five top-ranked VEPs (column top_5_winrate; the five are not listed in the table), |Spearman| with DMS on shared variants, 36 human DMS sets)10.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only)27.1
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, direct assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, direct DMS assays only)59.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only)22.6
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, indirect assays onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, indirect DMS assays only)58.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only)24.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)FathmmXF (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)25.5
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions with gaps filledPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS, SNV-accessible substitutions, missing predictions filled with the VEP's most benign score)57.3
Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8); Livesey and Marsh 2025, Genome Biology 26:104 (Europe PMC full text XML)MutationAssessor (Livesey and Marsh 2025)Livesey and Marsh 2025 DMS benchmark: pairwise |Spearman| win rate, SNV-accessible substitutions onlyPairwise win rate (average pairwise win rate against all other VEPs, |Spearman| with DMS on shared variants, SNV-accessible substitutions only)58.8
In the other studies (52 results)
In the other studies
StudyToolProtocol or benchmarkMetricValue
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Accuracy (OncoKB benchmark (positive set not stated), default categorical calls)0.66
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Negative predictive value (OncoKB benchmark (positive set not stated), default categorical calls)0.85
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Precision (OncoKB benchmark (positive set not stated), default categorical calls)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Recall (OncoKB benchmark (positive set not stated), default categorical calls)0.93
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Specificity (OncoKB benchmark (positive set not stated), default categorical calls)0.39
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Precision (OncoKB benchmark (positive set not stated), median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Recall (OncoKB benchmark (positive set not stated), median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)FATHMM-XF (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Specificity (OncoKB benchmark (positive set not stated), median-score threshold)0.65
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Accuracy (cell viability drivers versus neutral, default categorical calls)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Negative predictive value (cell viability drivers versus neutral, default categorical calls)0.72
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Precision (cell viability drivers versus neutral, default categorical calls)0.56
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Recall (cell viability drivers versus neutral, default categorical calls)0.89
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Specificity (cell viability drivers versus neutral, default categorical calls)0.28
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Accuracy (cell viability drivers versus neutral, median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Negative predictive value (cell viability drivers versus neutral, median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Precision (cell viability drivers versus neutral, median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Recall (cell viability drivers versus neutral, median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)FATHMM-XF (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Specificity (cell viability drivers versus neutral, median-score threshold)0.63
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Accuracy (OncoKB benchmark (positive set not stated), default categorical calls)0.58
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Negative predictive value (OncoKB benchmark (positive set not stated), default categorical calls)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Precision (OncoKB benchmark (positive set not stated), default categorical calls)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Recall (OncoKB benchmark (positive set not stated), default categorical calls)0.66
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 10: Performance metrics of 17 algorithms, default categories, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), default categorical calls (Chen et al. 2020 Additional file 10)Specificity (OncoKB benchmark (positive set not stated), default categorical calls)0.5
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Accuracy (OncoKB benchmark (positive set not stated), median-score threshold)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Negative predictive value (OncoKB benchmark (positive set not stated), median-score threshold)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Precision (OncoKB benchmark (positive set not stated), median-score threshold)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Recall (OncoKB benchmark (positive set not stated), median-score threshold)0.61
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 9: Performance metrics of 33 algorithms, median-score threshold, benchmark 2 (OncoKB)MutationAssessor (Chen et al. 2020)OncoKB benchmark (positive set not stated), median-score threshold (Chen et al. 2020 Additional file 9)Specificity (OncoKB benchmark (positive set not stated), median-score threshold)0.57
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Accuracy (cell viability drivers versus neutral, default categorical calls)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Negative predictive value (cell viability drivers versus neutral, default categorical calls)0.59
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Precision (cell viability drivers versus neutral, default categorical calls)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Recall (cell viability drivers versus neutral, default categorical calls)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 22: Performance metrics of 17 algorithms, default categories, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, default categorical calls (Chen et al. 2020 Additional file 22)Specificity (cell viability drivers versus neutral, default categorical calls)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Accuracy (cell viability drivers versus neutral, median-score threshold)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Negative predictive value (cell viability drivers versus neutral, median-score threshold)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Precision (cell viability drivers versus neutral, median-score threshold)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Recall (cell viability drivers versus neutral, median-score threshold)0.6
Comprehensive assessment of computational algorithms in predicting cancer driver mutations; Chen et al. 2020, Additional file 21: Performance metrics of 33 algorithms, median-score threshold, benchmark 5 (cell viability)MutationAssessor (Chen et al. 2020)Cell viability drivers versus neutral, median-score threshold (Chen et al. 2020 Additional file 21)Specificity (cell viability drivers versus neutral, median-score threshold)0.6
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)fathmm-XF_coding rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, all genes)0.568
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)fathmm-XF_coding rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, all genes)0.813
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)fathmm-XF_coding rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, oncogenes (gain of function))0.644
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)fathmm-XF_coding rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, oncogenes (gain of function))0.81
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)fathmm-XF_coding rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, tumour suppressors (loss of function))0.476
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)fathmm-XF_coding rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, tumour suppressors (loss of function))0.857
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MutationAssessor rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, all genes)0.554
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MutationAssessor rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, all genes (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, all genes)0.794
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MutationAssessor rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, oncogenes (gain of function))0.604
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MutationAssessor rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, oncogenes (gain of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, oncogenes (gain of function))0.769
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MutationAssessor rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)Average precision (CGC missense, tumour suppressors (loss of function))0.535
An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role)MutationAssessor rank score, dbNSFP 5.3.1a (Lee 2026)CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)AUROC (CGC missense, tumour suppressors (loss of function))0.824

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-b4b51658775800f6 · All contested results · How findings are made and followed up