Protocol

ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2)

Global ROC-AUC and balanced PRC-AUC on the 12,095 ClinVar variants with predictions from all 19 methods eligible for clinical benchmarking.

19 evaluations · 38 results

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

19 recorded evaluations, 38 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

19 evaluations · 38 results. Different protocols are not a single leaderboard.

Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) · Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)

Sorted by AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot)	AUROC (global ROC-AUC over all variants; zero-shot)
ESM1b (Brandes et al. 2023 clinical benchmarks)Evaluation	0.884 (best in column)	0.892 (best in column)
EVE (Brandes et al. 2023 clinical benchmarks)Evaluation	0.875	0.883
SIFT (Brandes et al. 2023 clinical benchmarks)Evaluation	0.872	0.874
MutationAssessor (Brandes et al. 2023 clinical benchmarks)Evaluation	0.857	0.86
SIFT4G (Brandes et al. 2023 clinical benchmarks)Evaluation	0.856	0.862
PROVEAN (Brandes et al. 2023 clinical benchmarks)Evaluation	0.844	0.864
MTBAN (Brandes et al. 2023 clinical benchmarks)Evaluation	0.838	0.832
phastCons (vertebrate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.794	0.699
LRT (Brandes et al. 2023 clinical benchmarks)Evaluation	0.779	0.753
phyloP (vertebrate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.77	0.798
GenoCanyon (Brandes et al. 2023 clinical benchmarks)Evaluation	0.682	0.685
phastCons (mammalian) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.668	0.648
SiPhy (Brandes et al. 2023 clinical benchmarks)Evaluation	0.666	0.738
GERP++ (Brandes et al. 2023 clinical benchmarks)Evaluation	0.641	0.704
phyloP (mammalian) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.633	0.671
phastCons (primate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.612	0.628
phyloP (primate) (Brandes et al. 2023 clinical benchmarks)Evaluation	0.598	0.626
bStatistic (Brandes et al. 2023 clinical benchmarks)Evaluation	0.543	0.563
fitCons (Brandes et al. 2023 clinical benchmarks)Evaluation	0.512	0.529

All 38 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: bStatistic (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.543 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source bStatistic on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C19, row 'bStatistic', column 'AUC_Balanced_AUPRC'
Configuration: bStatistic (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.563 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source bStatistic on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B19, row 'bStatistic', column 'AUC_AUROC'
Configuration: ESM1b (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.884 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source ESM1b on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C2, row 'ESM1b', column 'AUC_Balanced_AUPRC'
Configuration: ESM1b (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.892 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source ESM1b on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B2, row 'ESM1b', column 'AUC_AUROC'
Configuration: EVE (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.875 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C3, row 'EVE', column 'AUC_Balanced_AUPRC'
Configuration: EVE (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.883 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source EVE on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B3, row 'EVE', column 'AUC_AUROC'
Configuration: fitCons (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.512 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source fitCons on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C20, row 'fitCons', column 'AUC_Balanced_AUPRC'
Configuration: fitCons (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.529 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source fitCons on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B20, row 'fitCons', column 'AUC_AUROC'
Configuration: GenoCanyon (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.682 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GenoCanyon on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C14, row 'GenoCanyon', column 'AUC_Balanced_AUPRC'
Configuration: GenoCanyon (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.685 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GenoCanyon on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B14, row 'GenoCanyon', column 'AUC_AUROC'
Configuration: GERP++ (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.641 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GERP++ on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C12, row 'GERP++', column 'AUC_Balanced_AUPRC'
Configuration: GERP++ (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.704 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source GERP++ on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B12, row 'GERP++', column 'AUC_AUROC'
Configuration: LRT (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.779 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source LRT on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C10, row 'LRT', column 'AUC_Balanced_AUPRC'
Configuration: LRT (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.753 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source LRT on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B10, row 'LRT', column 'AUC_AUROC'
Configuration: MTBAN (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.838 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MTBAN on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C8, row 'MTBAN', column 'AUC_Balanced_AUPRC'
Configuration: MTBAN (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.832 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MTBAN on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B8, row 'MTBAN', column 'AUC_AUROC'
Configuration: MutationAssessor (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.857 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MutationAssessor on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C7, row 'MutationAssessor', column 'AUC_Balanced_AUPRC'
Configuration: MutationAssessor (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.86 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source MutationAssessor on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B7, row 'MutationAssessor', column 'AUC_AUROC'
Configuration: phastCons (mammalian) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.668 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (mammalian) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C16, row 'phastCons (mammalian)', column 'AUC_Balanced_AUPRC'
Configuration: phastCons (mammalian) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.648 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (mammalian) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B16, row 'phastCons (mammalian)', column 'AUC_AUROC'
Configuration: phastCons (primate) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.612 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (primate) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C17, row 'phastCons (primate)', column 'AUC_Balanced_AUPRC'
Configuration: phastCons (primate) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.628 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (primate) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B17, row 'phastCons (primate)', column 'AUC_AUROC'
Configuration: phastCons (vertebrate) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.794 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (vertebrate) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C13, row 'phastCons (vertebrate)', column 'AUC_Balanced_AUPRC'
Configuration: phastCons (vertebrate) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.699 AUROC (global ROC-AUC over all variants; zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phastCons (vertebrate) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell B13, row 'phastCons (vertebrate)', column 'AUC_AUROC'
Configuration: phyloP (mammalian) (Brandes et al. 2023 clinical benchmarks)	Protocol: ClinVar missense pathogenicity, variants scored by all 19 eligible methods (Brandes et al. 2023 Fig. 2e-f, Supplementary Table 2) Dataset: ClinVar missense pathogenic vs benign, primary isoforms (Brandes et al. 2023, from the EVE portal)	0.633 AUPRC (balanced PRC-AUC: mean over 100 random downsamples to equal pathogenic and benign counts (80% of the minority class); zero-shot) unitless · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source phyloP (mammalian) on ClinVar missense, variants scored by all 19 methods (Brandes et al. 2023) gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods Aggregation: Not reported Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_ClinVar' cell C15, row 'phyloP (mammalian)', column 'AUC_Balanced_AUPRC'

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 2 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

No execution recipe linked to this protocol. Recipe availability does not establish a completed evaluation.

Author-reported evaluations: 1
External evaluations: 18

Literature evidence is not a Rewire measurement. Executed but unpublished runs and private review status are not included.

Null control

Proposed control: requires review

Select a task-valid null control after reviewing inputs and metric

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Conventional reference

Proposed control: requires review

Select an upstream conventional reference after reviewing the full protocol

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)

Coverage is derived from release 2026-10-22-1b7cf027e4f6. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

2 source records and release history

Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) · Original source · Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10)
Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Original source · Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10)

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: gfm-guides-20261014-esm1b-protocol-clinvar-all-19-methods

areas: dna-genomes; protein-function
contexts: clinical_research
protocol: Scores used directly (no training on these labels). Global ROC-AUC treats all variants as one genome-wide task. Balanced PRC-AUC: downsample to equal pathogenic and benign counts (80% of the minority class), compute PRC-AUC, average over 100 repeats. Variant set restricted to variants scored by all 19 eligible methods.
version: Supplementary Table 2 sheet 'Table_of_results_ClinVar', n_var 12095
metric: auroc
metric direction: higher
unit: unitless
denominator: 12095
limitations: Benchmark built by the ESM1b workflow's authors (author-reported for ESM1b).; Most other scores come from dbNSFP 4.3a; methods trained on clinical databases or using allele frequency are excluded from this comparison but reported head to head in Supplementary Table 2 (not extracted).; Balanced PRC-AUC results are recorded under the same protocol with metric auprc and their qualifier.
source locator: Methods 'AUC metrics for pathogenicity classification' paragraphs 1-2; Supplementary Table 2 'Contents' (header definitions) and column n_var

Related records

Suggest a correction