Protocol

HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)

Global ROC-AUC and balanced PRC-AUC on the 12,357 HGMD/gnomAD variants with predictions from all 19 methods eligible for clinical benchmarking.

19 evaluations · 38 results

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

19 recorded evaluations, 38 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

19 evaluations · 38 results. Different protocols are not a single leaderboard.

Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) · Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)

Sorted by AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	AUROC (global ROC-AUC over all variants; zero-shot)
SIFT (Brandes et al. 2023 clinical benchmarks)Evaluation	0.882 (best in column)	0.883
EVE (Brandes et al. 2023 clinical benchmarks)Evaluation	0.881	0.885
ESM1b (Brandes et al. 2023 clinical benchmarks)Evaluation	0.881	0.887 (best in column)
MutationAssessor (Brandes et al. 2023 clinical benchmarks)Evaluation	0.876	0.879
SIFT4G (Brandes et al. 2023 clinical benchmarks)Evaluation	0.866	0.869
PROVEAN (Brandes et al. 2023 clinical benchmarks)Evaluation	0.864	0.879
MTBAN (Brandes et al. 2023 clinical benchmarks)Evaluation	0.856	0.843
phastCons (vertebrate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.791	0.712
phyloP (vertebrate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.789	0.803
LRT (Brandes et al. 2023 clinical benchmarks)Evaluation	0.787	0.76
GenoCanyon (Brandes et al. 2023 clinical benchmarks)Evaluation	0.734	0.725
phyloP (mammalian) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.68	0.724
phastCons (mammalian) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.675	0.666
SiPhy (Brandes et al. 2023 clinical benchmarks)Evaluation	0.671	0.743
phyloP (primate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.65	0.688
GERP++ (Brandes et al. 2023 clinical benchmarks)Evaluation	0.645	0.7
phastCons (primate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.624	0.645
bStatistic (Brandes et al. 2023 clinical benchmarks)Evaluation	0.534	0.536
fitCons (Brandes et al. 2023 clinical benchmarks)Evaluation	0.477	0.468

All 38 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: bStatistic (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.534 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source bStatistic on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C19, row 'bStatistic', column 'AUC_Balanced_AUPRC'
Configuration: bStatistic (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.536 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source bStatistic on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B19, row 'bStatistic', column 'AUC_AUROC'
Configuration: ESM1b (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.881 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source ESM1b on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C2, row 'ESM1b', column 'AUC_Balanced_AUPRC'
Configuration: ESM1b (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.887 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source ESM1b on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B2, row 'ESM1b', column 'AUC_AUROC'
Configuration: EVE (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.881 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C3, row 'EVE', column 'AUC_Balanced_AUPRC'
Configuration: EVE (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.885 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B3, row 'EVE', column 'AUC_AUROC'
Configuration: fitCons (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.477 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source fitCons on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C20, row 'fitCons', column 'AUC_Balanced_AUPRC'
Configuration: fitCons (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.468 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source fitCons on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B20, row 'fitCons', column 'AUC_AUROC'
Configuration: GenoCanyon (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.734 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GenoCanyon on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C12, row 'GenoCanyon', column 'AUC_Balanced_AUPRC'
Configuration: GenoCanyon (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.725 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GenoCanyon on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B12, row 'GenoCanyon', column 'AUC_AUROC'
Configuration: GERP++ (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.645 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GERP++ on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C15, row 'GERP++', column 'AUC_Balanced_AUPRC'
Configuration: GERP++ (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.7 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GERP++ on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B15, row 'GERP++', column 'AUC_AUROC'
Configuration: LRT (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.787 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source LRT on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C10, row 'LRT', column 'AUC_Balanced_AUPRC'
Configuration: LRT (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.76 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source LRT on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B10, row 'LRT', column 'AUC_AUROC'
Configuration: MTBAN (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.856 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MTBAN on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C8, row 'MTBAN', column 'AUC_Balanced_AUPRC'
Configuration: MTBAN (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.843 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MTBAN on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B8, row 'MTBAN', column 'AUC_AUROC'
Configuration: MutationAssessor (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.876 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MutationAssessor on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C5, row 'MutationAssessor', column 'AUC_Balanced_AUPRC'
Configuration: MutationAssessor (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.879 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MutationAssessor on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B5, row 'MutationAssessor', column 'AUC_AUROC'
Configuration: phastCons (mammalian) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.675 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (mammalian) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C17, row 'phastCons (mammalian)', column 'AUC_Balanced_AUPRC'
Configuration: phastCons (mammalian) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.666 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (mammalian) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B17, row 'phastCons (mammalian)', column 'AUC_AUROC'
Configuration: phastCons (primate) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.624 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (primate) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C18, row 'phastCons (primate)', column 'AUC_Balanced_AUPRC'
Configuration: phastCons (primate) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.645 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (primate) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B18, row 'phastCons (primate)', column 'AUC_AUROC'
Configuration: phastCons (vertebrate) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.791 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (vertebrate) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C14, row 'phastCons (vertebrate)', column 'AUC_Balanced_AUPRC'
Configuration: phastCons (vertebrate) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.712 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (vertebrate) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell B14, row 'phastCons (vertebrate)', column 'AUC_AUROC'
Configuration: phyloP (mammalian) (Brandes et al. 2023 clinical benchmarks)	Protocol: HGMD/gnomAD missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: HGMD disease-causing vs gnomAD common missense variants, primary isoforms (Brandes et al. 2023)	0.68 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phyloP (mammalian) on HGMD/gnomAD missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_HGMD' cell C13, row 'phyloP (mammalian)', column 'AUC_Balanced_AUPRC'

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 2 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

No execution recipe linked to this protocol. Recipe availability does not establish a completed evaluation.

Author-reported evaluations: 1
External evaluations: 18

Literature evidence is not a Rewire measurement. Executed but unpublished runs and private review status are not included.

Null control

Proposed control: requires review

Select a task-valid null control after reviewing inputs and metric

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Conventional reference

Proposed control: requires review

Select an upstream conventional reference after reviewing the full protocol

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)

Coverage is derived from release 2026-10-22-1b7cf027e4f6. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

2 source records and release history

Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) · Original source · Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10)
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Original source · Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10)

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: gfm-guides-20261014-esm1b-protocol-hgmd-all-19-methods

areas: dna-genomes; protein-function
contexts: clinical_research
protocol: Scores used directly (no training on these labels). Global ROC-AUC treats all variants as one genome-wide task. Balanced PRC-AUC: downsample to equal pathogenic and benign counts (80% of the minority class), compute PRC-AUC, average over 100 repeats. Variant set restricted to variants scored by all 19 eligible methods.
version: Supplementary Table 2 sheet 'Table_of_results_HGMD', n_var 12357
metric: auroc
metric direction: higher
unit: unitless
denominator: 12357
limitations: Benchmark built by the ESM1b workflow's authors (author-reported for ESM1b).; Most other scores come from dbNSFP 4.3a; methods trained on clinical databases or using allele frequency are excluded from this comparison but reported head to head in Supplementary Table 2 (not extracted).; Balanced PRC-AUC results are recorded under the same protocol with metric auprc and their qualifier.
source locator: Methods 'AUC metrics for pathogenicity classification' paragraphs 1-2; Supplementary Table 2 'Contents' (header definitions) and column n_var

Related records

Suggest a correction