rewirebio.iobenchmarks

Use caseDNA and genomesResearch and clinical research

Interpret BRCA1/BRCA2 germline variants

Which evidence-support methods improve BRCA1/BRCA2 variant review without increasing serious classification errors?

You bring
  • Confirmed germline variant, reference transcript and assay limitations
  • Population frequency, segregation, phenotype and functional evidence with dates
  • A relevant, versioned ClinGen ENIGMA specification and evidence cutoff
You want
A classification dossier with criterion-level evidence, conflicts, uncertainty and further information needed for review.
Evidence in this release
16 comparisons shown · 121 tools · 220 configurations
Kept up to date
Evidence last checked 14 October 2026 · re-checked monthly, next due 14 November 2026
Executive Takeaway

For "Interpret BRCA1/BRCA2 germline variants", on Livesey and Marsh 2025 VEP rankings, Clinical benchmark, ClinVar P/LP versus gnomAD missense (Pairwise win rate, a proxy protocol), SNPred scored 83.24%, reported by Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8).

Best reported valueSNPredLivesey and Marsh 2025, Supplementary Material 2 (Tables…
Pairwise win rate83.24%Livesey and Marsh 2025 VEP rankings, Clinical benchmark, ClinVar P/LP versus gnomAD missense
Baseline on this protocolNone scoredNo margin is computed
Ranked tools8585 from independent studies

85 tools have a value on this protocol; 85 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.

Ranking

Ranked on Livesey and Marsh 2025 VEP rankings, Clinical benchmark, ClinVar P/LP versus gnomAD missense (Pairwise win rate, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.

80 tools in 85 rows, from 2 sources. 85 of 85 results (100%) come from an independent study.

Rank▲ModelPairwise win rate (higher is better)EvidenceCoverageReported byLicense
#183.24%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#282.19%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#380.74%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#477.49%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#577.15%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#676.58%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#776.56%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#875.95%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#974.34%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#10
BayesDel_noAF[-] Baseline
71.79%Source-checked Independent study9/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1171.66%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1270.61%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1370.42%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1469.57%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1569.44%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1669.30%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1768.66%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1867.37%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#1966.99%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2066.94%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2166.86%Source-checked Independent study9/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2265.92%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2365.46%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2465.10%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2563.98%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2663.67%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2763.66%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2862.40%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#2961.69%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3061.40%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Code: MIT
#3160.83%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3259.11%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3358.32%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3458.09%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3557.33%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3657.09%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3755.21%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3855.01%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#3954.94%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4054.59%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4154.28%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4254.27%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4353.43%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4452.09%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4551.42%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…The official facebook/esm2_…
#4651.40%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4750.55%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4850.44%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#4949.89%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5049.13%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5148.97%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5248.93%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5348.62%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5447.92%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5546.50%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5646.04%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5745.66%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5845.53%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#5944.98%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6043.13%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6142.40%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6241.41%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6339.70%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6437.33%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6535.26%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6634.76%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6732.00%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6831.63%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#6927.90%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7025.34%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7123.37%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7222.55%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7322.22%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7421.20%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7520.77%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7620.17%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7718.80%Source-checked Independent study3/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7816.62%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#7914.28%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#8014.08%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#8112.37%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#8211.25%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#839.35%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#848.73%Source-checked Independent study2/16Livesey and Marsh 2025, Supplementary Material…Not recorded
#854.59%Source-checked Independent study1/16Livesey and Marsh 2025, Supplementary Material…Not recorded
41 tools are evaluated elsewhere in this use case but not on this protocol, so not ranked

Baseline

BayesDel under the ENIGMA specification. The ClinGen ENIGMA BRCA1/BRCA2 specification uses BayesDel, with gene-specific cutpoints, as its computational evidence for missense variants (PP3/BP4), so a new predictor is judged against it. BayesDel is a meta-predictor that combines other scores, so it is a simple trained model rather than a pre-machine-learning method; manual review under the specification, the full comparator, has no method record.

Machine-learning methods are compared with the baseline on the same data in 5 tables: Binary in silico calls against BRCA1 and BRCA2 functional truth sets (Cubuk et al. 2021), Binary LoF discrimination for BRCA1 missense variants (Ramadane-Morchadi et al. 2025), PP3/BP4 evidence strength for BRCA1 missense variants (Ramadane-Morchadi et al. 2025), Livesey and Marsh 2025 VEP rankings, ProteinGym clinical benchmark (commit 144fe22). Baseline rows are marked in each table.

Tools compared

Comparisons

Unified model comparison for this use case. Compare tools against protocol baselines in a single view.

Rows marked Contested rest on results that other studies disagree with. The results are still shown. See the disagreements that involve this use case for what disagrees and what we are doing about it.

Livesey and Marsh 2025 VEP rankings

Proxy evidenceIndependent study: Reported in a study whose authors did not develop the evaluated tool.

Average pairwise win rate by per-protein AUROC, ClinVar P/LP versus gnomAD v4.1 missense, 819 proteins

Pairwise win rate (higher is better), with the other metrics below each value. Showing the top 10 of 80 rows. The best value in each column is highlighted.

ToolClinical benchmark, ClinVar P/LP versus gnomAD missense
SNPredmachine learning
83.24% (best in column)
MetaRNNmachine learning
82.19%
PHACTboostmachine learning
80.74%
AlphaMissensemachine learning
77.49%
77.15%
popEVEspecialist model
76.58%
MutScoremachine learning
76.56%
75.95%
MOIpred_benignmachine learning
74.34%
BayesDel_noAFBaseline2 sweepsmachine learning
71.79%
CPT-1machine learning
71.66%
GEMMEspecialist model
70.61%
iGEMMEspecialist model
70.42%
EVEspecialist model
69.57%
MISTICmachine learning
69.44%
VARITY_R2 sweepsmachine learning
69.30%
VESPAmachine learning
68.66%
UNEECONspecialist model
67.37%
gMVPmachine learning
66.99%
ESCOTTspecialist model
66.94%
DeepSequencespecialist model
65.92%
SaProtfoundation model
65.46%
65.10%
MutFormermachine learning
63.98%
CAPICEmachine learning
63.67%
62.40%
Rhapsodymachine learning
61.69%
ESM-1vfoundation model
61.40%
DEOGEN2machine learning
60.83%
DeepSAVmachine learning
59.11%
PHACTspecialist model
58.32%
EVmutation_epistatic2 sweepsspecialist model
58.09%
mvPPTmachine learning
57.33%
Maverick_benignmachine learning
57.09%
VESPAlmachine learning
55.21%
SNAP2machine learning
55.01%
mutationTCNspecialist model
54.94%
Eigenspecialist model
54.59%
54.28%
SPRImachine learning
54.27%
InMeRFmachine learning
53.43%
52.09%
ESM2foundation model
51.42%
51.40%
50.55%
SIGMAmachine learning
50.44%
CONDELspecialist model
49.89%
49.13%
SIFT4G2 sweepsconventional
48.97%
PROVEANconventional
48.62%
47.92%
DeMaSkmachine learning
46.50%
SuSPectmachine learning
46.04%
LASSIEspecialist model
45.66%
45.53%
44.98%
PonP2machine learning
43.13%
Envisionmachine learning
42.40%
Tranceptionfoundation model
39.70%
PhD-SNPgmachine learning
37.33%
LIST-S2specialist model
35.26%
32.00%
31.63%
DANNmachine learning
27.90%
AlphScoremachine learning
25.34%
phyloPspecialist model
23.37%
FATHMMspecialist model
22.55%
COSMISspecialist model
22.22%
ProtGPT2foundation model
21.20%
3Cnetmachine learning
20.77%
20.17%
LRTspecialist model
18.80%
SiPhyspecialist model
16.62%
GenoCanyonspecialist model
14.28%
sequence_unetspecialist model
14.08%
GERP++specialist model
12.37%
BLOSUM62conventional
11.25%
Granthamspecialist model
9.35%
phastConsspecialist model
8.73%
fitConsspecialist model
4.59%
Show all 80 rowsShow the top 10 rows only
Caveats and method
  • Whether BRCA1 or BRCA2 is among the 819 proteins is not stated in the extracted tables.
  • No calibration to ACMG/AMP evidence strength.
  • The negative class is gnomAD v4.1 missense variants not classed pathogenic ('putatively benign'); the authors note it will include unclassified pathogenic variants, particularly in recessive and incomplete-penetrance genes (Results).
  • VEPs that use allele frequency as a feature likely benefit from circularity, because frequency is used to classify variants as benign (Discussion); the gnomAD negative class reduces but does not remove this.
  • No training variants were removed; the authors attribute most of the clinical-trained VEPs' advantage over the DMS ranking to circularity (Results, Fig. 4).
  • AUROC is computed per protein, so the ranking does not test whether scores are comparable across genes.
  • The text says 83 VEPs; Table S8 prints 85 rows (see the protocol's limitations).
  • Transcribed from the source; no reproduction by Rewire and no human scientific review.

Protocol: Livesey and Marsh 2025 clinical benchmark: pairwise AUROC win rate, ClinVar pathogenic versus gnomAD missense

Primary benchmark evaluation shown (Livesey and Marsh 2025 VEP rankings). 15 additional study protocols (Binary in silico calls against BRCA1 and BRCA2 functional truth sets (Cubuk et al. 2021), Binary LoF discrimination for BRCA1 missense variants (Ramadane-Morchadi et al. 2025), PP3/BP4 evidence strength for BRCA1 missense variants (Ramadane-Morchadi et al. 2025) and others) are cited in individual tool drawer rows and under Question, sources and review.

Not covered yet

  • Independent complete clinical classifications, serious-error adjudication and equal-evidence reviewer-time comparison were not found in selected evidence.
  • BayesDel labels derive from functional assays; threshold calibration is not held-out pathogenicity validation.
  • Study v1.0 manual pilot revises specifications on the same variants; current specification version must be pinned for any new run.
  • Generic BayesDel thresholds from Pejaver are a relevant alternative described in Table S4; supplement download returned challenge HTML, so the four primary-body percentages/bounds are entered while unverified Table S4 cells remain missing.
  • Qualified human scientific review remains outstanding; automated source transcription does not establish experimental replication.
  • No linked study measures complete BRCA1/BRCA2 classifications, serious misclassification or reviewer time when computational evidence is added; Cubuk et al. 2021 and Ramadane-Morchadi et al. 2025 score evidence inputs against functional classes.
  • Cubuk et al. 2021 BRCA1 and Ramadane-Morchadi et al. 2025 both use the Findlay et al. 2018 saturation genome editing data as truth, so they are not independent replications; BRCA2 functional truth in Cubuk et al. covers 188 variants in the DNA-binding domain.
  • Cubuk et al. 2021 benignity likelihood ratio intervals are inconsistent with the published counts and are not used; ratios that divide by a zero count are not shown.
  • Ramadane-Morchadi et al. 2025 chose the AlphaMissense and ΔΔG thresholds on the same variants they report; no held-out BRCA1 test of those thresholds is linked.
  • BRCA2 saturation genome editing (Huang et al. 2025), splicing predictors and non-missense variants are not covered by the linked comparisons.

Contribute evidence or propose a correction

How to cite

Interpret BRCA1/BRCA2 germline variants. Rewire biological model benchmark database, release 2026-11-21-adc15377fab2. https://benchmarks.rewirebio.io/use-cases/brca1-brca2-germline-interpretation/

Release 2026-11-21-adc15377fab2. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.

The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence

Question, sources and review

Who this is for, setting and exclusions

Choose methods that assemble and assess variant evidence under gene-specific criteria for a professional BRCA1/BRCA2 classification review.

Who this is for
  • Germline variant scientists
  • Hereditary-cancer genetics reviewers
Biological setting
Germline BRCA1/BRCA2 interpretation in hereditary-cancer testing. Additional genes require their own specifications and comparisons.
Clinical research scope
Clinical research on hereditary-cancer evidence review. A VUS does not justify management changes; a negative panel does not remove risk associated with family history.

Outside this use case

  • Germline classification does not estimate an individual's absolute cancer risk or select treatment.
  • Tumour-only sequencing does not confirm germline status.
  • Functional scores and computational predictions are evidence inputs, not complete classifications.
Evidence collection plan
Evidence collection planCollecting evidence

Already on record: 16 comparisons shown · 121 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.

Comparison question

Can an evidence-support method reduce review effort without increasing serious classification or criteria errors relative to standard gene-specific review?

Baselines, outcomes and validation requirements

Baselines to include

  • Manual criteria-based review using the same evidence cutoff and ClinGen ENIGMA specification

Outcomes to measure

  • Serious classification disagreements and criterion-assignment errors
  • Coverage of unresolved cases and preservation of conflicts
  • Review time against independent expert adjudication

Validation requirements

  • Record the specification version, classification date and germline confirmation.
  • Audit predictor circularity and overlap with functional training data.
  • Use temporal and gene/domain separation appropriate to the intended claim.
  • Retain uncertain and conflicting cases and predefine error severity.
  • Obtain blinded independent adjudication by qualified hereditary-cancer reviewers.

Next collection task

Close the remaining endpoint, independence and comparator gaps documented in the 30 September 2026 audit and linked article issue; obtain qualified scientific review.

Related articles
Sources, review and downloads

Automated source review · 2026-09-30 · Codex source curation, independent worker cross-review and root integration review

Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.

Release 2026-11-21-adc15377fab2 · question use-case-brca1-brca2-germline-interpretation

Use-case input digest bfbb7ab9303becad1e3cb395568c26f41e72ddb67cd1a89a06d9eaa0f76bd95f

Download questions and review metadata (JSON) (gzip) · Verify release checksums (gzip)