Benchmark

ENIGMA 2024 BRCA evidence-calibration and curation study

Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

5 evaluations · 39 results

Overview

Published-study grouping of the catalogued evaluation protocols. This grouping does not claim an executable suite, full-paper extraction or independent clinical validation.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

5 recorded evaluations, 39 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

5 evaluations · 39 results. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: BayesDel generic Pejaver evidence thresholds in ENIGMA comparison	Protocol: BRCA1 generic BayesDel threshold evidence-code assignment Dataset: BRCA1 functional reference calibration set	<10% benign_reference_assigned_bp4 percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 generic BayesDel threshold comparator Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Results, BayesDel calibration, BP4 code would be assigned to <10% of BRCA1 and BRCA2 benign reference variants
Configuration: BayesDel generic Pejaver evidence thresholds in ENIGMA comparison	Protocol: BRCA1 generic BayesDel threshold evidence-code assignment Dataset: BRCA1 functional reference calibration set	29% benign_reference_assigned_pp3 percent · lower Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 generic BayesDel threshold comparator Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Results, BayesDel calibration, sentence 29% for BRCA1 and 36% for BRCA2 incorrectly assigned PP3
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	149 high_benign_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≥0.28, benign_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	12% high_benign_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≥0.28, benign_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	7.09 high_likelihood_ratio likelihood ratio · unknown Uncertainty: type: confidence_interval; level: 0.95; lower: 6.06; upper: 8.30; printed: 6.06–8.30 Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≥0.28, likelihood_ratio
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	377 high_pathogenic_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≥0.28, pathogenic_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	83% high_pathogenic_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≥0.28, pathogenic_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	946 low_benign_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≤0.15, benign_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	74% low_benign_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≤0.15, benign_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	0.1 low_likelihood_ratio likelihood ratio · unknown Uncertainty: type: confidence_interval; level: 0.95; lower: 0.07; upper: 0.14; printed: 0.07–0.14 Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≤0.15, likelihood_ratio
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	34 low_pathogenic_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≤0.15, pathogenic_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	7% low_pathogenic_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel ≤0.15, pathogenic_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	183 middle_benign_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel >0.15 and <0.28, benign_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	14% middle_benign_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel >0.15 and <0.28, benign_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	0.69 middle_likelihood_ratio likelihood ratio · unknown Uncertainty: type: confidence_interval; level: 0.95; lower: 0.51; upper: 0.94; printed: 0.51–0.94 Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel >0.15 and <0.28, likelihood_ratio
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	45 middle_pathogenic_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel >0.15 and <0.28, pathogenic_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA1 BayesDel evidence calibration Dataset: BRCA1 functional reference calibration set	10% middle_pathogenic_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA1 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA1, BayesDel >0.15 and <0.28, pathogenic_fraction
Configuration: BayesDel generic Pejaver evidence thresholds in ENIGMA comparison	Protocol: BRCA2 generic BayesDel threshold evidence-code assignment Dataset: BRCA2 functional reference calibration set	<10% benign_reference_assigned_bp4 percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 generic BayesDel threshold comparator Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Results, BayesDel calibration, BP4 code would be assigned to <10% of BRCA1 and BRCA2 benign reference variants
Configuration: BayesDel generic Pejaver evidence thresholds in ENIGMA comparison	Protocol: BRCA2 generic BayesDel threshold evidence-code assignment Dataset: BRCA2 functional reference calibration set	36% benign_reference_assigned_pp3 percent · lower Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 generic BayesDel threshold comparator Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Results, BayesDel calibration, sentence 29% for BRCA1 and 36% for BRCA2 incorrectly assigned PP3
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA2 BayesDel evidence calibration Dataset: BRCA2 functional reference calibration set	29 high_benign_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA2, BayesDel ≥0.30, benign_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA2 BayesDel evidence calibration Dataset: BRCA2 functional reference calibration set	10% high_benign_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA2, BayesDel ≥0.30, benign_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA2 BayesDel evidence calibration Dataset: BRCA2 functional reference calibration set	6.59 high_likelihood_ratio likelihood ratio · unknown Uncertainty: type: confidence_interval; level: 0.95; lower: 4.40; upper: 9.90; printed: 4.40–9.90 Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA2, BayesDel ≥0.30, likelihood_ratio
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA2 BayesDel evidence calibration Dataset: BRCA2 functional reference calibration set	88 high_pathogenic_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA2, BayesDel ≥0.30, pathogenic_count
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA2 BayesDel evidence calibration Dataset: BRCA2 functional reference calibration set	66% high_pathogenic_fraction percent · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA2, BayesDel ≥0.30, pathogenic_fraction
Configuration: BayesDel with ENIGMA BRCA gene-specific score categories	Protocol: BRCA2 BayesDel evidence calibration Dataset: BRCA2 functional reference calibration set	216 low_benign_count count · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source BRCA2 BayesDel gene-specific calibration Not reported Aggregation: Not reported Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Table 2, BRCA2, BayesDel ≤0.18, benign_count

Source checking is not independent reproduction. Release 2026-09-30-e37e3ab1284d.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Protocols

These source-backed links do not make different protocols or scores interchangeable.

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 10 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

Baseline status by linked protocol

BRCA1 generic BayesDel threshold evidence-code assignment · 0/2 roles measured
BRCA1 BayesDel evidence calibration · 0/2 roles measured
BRCA2 generic BayesDel threshold evidence-code assignment · 0/2 roles measured
BRCA2 BayesDel evidence calibration · 0/2 roles measured
ENIGMA specification pilot resolution · 0/2 roles measured

Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums

Coverage is derived from release 2026-09-30-e37e3ab1284d. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run this benchmark

Choose a concrete protocol before running an evaluation. Its inputs, split and scoring rules determine which results can be compared.

Run instructions

No runnable recipe has been reviewed for this benchmark. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-30-e37e3ab1284d
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-30-e37e3ab1284d · Record review: source checked

1 source records and release history

Evidence-based recommendations for gene-specific ACMG/AMP variant classification from the ClinGen ENIGMA BRCA1 and BRCA2 Variant Curation Expert Panel · Original source · 2024 final article; reported VCEP specifications v1.0

Download this release

Technical metadata and extraction receipts

Stable ID: uc-clinical-20260930-benchmark-enigma-study

areas: dna-genomes
contexts: clinical_research
entity level: suite
grouping type: published_study_evaluations
executable suite: false
source locator: Table2; calibration Results; pilot curation
review: method: automated_source_review; reviewer: Codex clinical coverage worker; date: 2026-09-30; note: Transcription from inspected primary source. No execution, independent replication or human scientific review.; source id: uc-clinical-20260930-source-enigma

Related records

part of: BRCA1 generic BayesDel threshold evidence-code assignment
part of: BRCA1 BayesDel evidence calibration
part of: BRCA2 generic BayesDel threshold evidence-code assignment
part of: BRCA2 BayesDel evidence calibration
part of: ENIGMA specification pilot resolution

Suggest a correction