Evidence last checked 14 October 2026 · re-checked monthly, next due 14 November 2026
Executive Takeaway
For "Interpret BRCA1/BRCA2 germline variants", on Livesey and Marsh 2025 VEP rankings, Clinical benchmark, ClinVar P/LP versus gnomAD missense (Pairwise win rate, a proxy protocol), SNPred scored 83.24%, reported by Livesey and Marsh 2025, Supplementary Material 2 (Tables S1-S8).
Best reported valueSNPredLivesey and Marsh 2025, Supplementary Material 2 (Tables…
Pairwise win rate83.24%Livesey and Marsh 2025 VEP rankings, Clinical benchmark, ClinVar P/LP versus gnomAD missense
Baseline on this protocolNone scoredNo margin is computed
Ranked tools8585 from independent studies
85 tools have a value on this protocol; 85 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.
Ranking
Ranked on Livesey and Marsh 2025 VEP rankings, Clinical benchmark, ClinVar P/LP versus gnomAD missense (Pairwise win rate, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.
80 tools in 85 rows, from 2 sources. 85 of 85 results (100%) come from an independent study.
BayesDel under the ENIGMA specification. The ClinGen ENIGMA BRCA1/BRCA2 specification uses BayesDel, with gene-specific cutpoints, as its computational evidence for missense variants (PP3/BP4), so a new predictor is judged against it. BayesDel is a meta-predictor that combines other scores, so it is a simple trained model rather than a pre-machine-learning method; manual review under the specification, the full comparator, has no method record.
Unified model comparison for this use case. Compare tools against protocol baselines in a single view.
Rows marked Contested rest on results that other studies disagree with. The results are still shown. See the disagreements that involve this use case for what disagrees and what we are doing about it.
Livesey and Marsh 2025 VEP rankings
Proxy evidenceIndependent study: Reported in a study whose authors did not develop the evaluated tool.
Average pairwise win rate by per-protein AUROC, ClinVar P/LP versus gnomAD v4.1 missense, 819 proteins
Pairwise win rate (higher is better), with the other metrics below each value. Showing the top 10 of 80 rows. The best value in each column is highlighted.
Tool
Clinical benchmark, ClinVar P/LP versus gnomAD missense
Show all 80 rowsShow the top 10 rows onlyCaveats and method
Whether BRCA1 or BRCA2 is among the 819 proteins is not stated in the extracted tables.
No calibration to ACMG/AMP evidence strength.
The negative class is gnomAD v4.1 missense variants not classed pathogenic ('putatively benign'); the authors note it will include unclassified pathogenic variants, particularly in recessive and incomplete-penetrance genes (Results).
VEPs that use allele frequency as a feature likely benefit from circularity, because frequency is used to classify variants as benign (Discussion); the gnomAD negative class reduces but does not remove this.
No training variants were removed; the authors attribute most of the clinical-trained VEPs' advantage over the DMS ranking to circularity (Results, Fig. 4).
AUROC is computed per protein, so the ranking does not test whether scores are comparable across genes.
The text says 83 VEPs; Table S8 prints 85 rows (see the protocol's limitations).
Transcribed from the source; no reproduction by Rewire and no human scientific review.
Primary benchmark evaluation shown (Livesey and Marsh 2025 VEP rankings). 15 additional study protocols (Binary in silico calls against BRCA1 and BRCA2 functional truth sets (Cubuk et al. 2021), Binary LoF discrimination for BRCA1 missense variants (Ramadane-Morchadi et al. 2025), PP3/BP4 evidence strength for BRCA1 missense variants (Ramadane-Morchadi et al. 2025) and others) are cited in individual tool drawer rows and under Question, sources and review.
Not covered yet
Independent complete clinical classifications, serious-error adjudication and equal-evidence reviewer-time comparison were not found in selected evidence.
BayesDel labels derive from functional assays; threshold calibration is not held-out pathogenicity validation.
Study v1.0 manual pilot revises specifications on the same variants; current specification version must be pinned for any new run.
Generic BayesDel thresholds from Pejaver are a relevant alternative described in Table S4; supplement download returned challenge HTML, so the four primary-body percentages/bounds are entered while unverified Table S4 cells remain missing.
Qualified human scientific review remains outstanding; automated source transcription does not establish experimental replication.
No linked study measures complete BRCA1/BRCA2 classifications, serious misclassification or reviewer time when computational evidence is added; Cubuk et al. 2021 and Ramadane-Morchadi et al. 2025 score evidence inputs against functional classes.
Cubuk et al. 2021 BRCA1 and Ramadane-Morchadi et al. 2025 both use the Findlay et al. 2018 saturation genome editing data as truth, so they are not independent replications; BRCA2 functional truth in Cubuk et al. covers 188 variants in the DNA-binding domain.
Cubuk et al. 2021 benignity likelihood ratio intervals are inconsistent with the published counts and are not used; ratios that divide by a zero count are not shown.
Ramadane-Morchadi et al. 2025 chose the AlphaMissense and ΔΔG thresholds on the same variants they report; no held-out BRCA1 test of those thresholds is linked.
BRCA2 saturation genome editing (Huang et al. 2025), splicing predictors and non-missense variants are not covered by the linked comparisons.
Release 2026-11-21-adc15377fab2. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.
The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence
Question, sources and review
Who this is for, setting and exclusions
Choose methods that assemble and assess variant evidence under gene-specific criteria for a professional BRCA1/BRCA2 classification review.
Who this is for
Germline variant scientists
Hereditary-cancer genetics reviewers
Biological setting
Germline BRCA1/BRCA2 interpretation in hereditary-cancer testing. Additional genes require their own specifications and comparisons.
Clinical research scope
Clinical research on hereditary-cancer evidence review. A VUS does not justify management changes; a negative panel does not remove risk associated with family history.
Outside this use case
Germline classification does not estimate an individual's absolute cancer risk or select treatment.
Tumour-only sequencing does not confirm germline status.
Functional scores and computational predictions are evidence inputs, not complete classifications.
Evidence collection plan
Evidence collection planCollecting evidence
Already on record: 16 comparisons shown · 121 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.
Comparison question
Can an evidence-support method reduce review effort without increasing serious classification or criteria errors relative to standard gene-specific review?
Baselines, outcomes and validation requirements
Baselines to include
Manual criteria-based review using the same evidence cutoff and ClinGen ENIGMA specification
Outcomes to measure
Serious classification disagreements and criterion-assignment errors
Coverage of unresolved cases and preservation of conflicts
Review time against independent expert adjudication
Validation requirements
Record the specification version, classification date and germline confirmation.
Audit predictor circularity and overlap with functional training data.
Use temporal and gene/domain separation appropriate to the intended claim.
Retain uncertain and conflicting cases and predefine error severity.
Obtain blinded independent adjudication by qualified hereditary-cancer reviewers.
Next collection task
Close the remaining endpoint, independence and comparator gaps documented in the 30 September 2026 audit and linked article issue; obtain qualified scientific review.
Related articles
Read the background, methods and practical considerations behind this question.
Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.