Evidence claim

Relevance of GFMBench-API BRCA1, zs-mlm-llr to "Interpret BRCA1/BRCA2 germline variants"

Use case: Interpret BRCA1/BRCA2 germline variants
Assessed by: GFMBench-API BRCA1, zs-mlm-llr
Relevance: Proxy evidence
Endpoint: Zero-shot AUROC and AUPRC of masked log-likelihood ratio for masked genomic language models against 3,893 BRCA1 variants labelled LOF, INT or FUNC from experimentally measured DNA-repair activity

GFMBench-API zero-shot masked log-likelihood ratio scores against BRCA1 DNA-repair function classes as evidence for BRCA1 variant review.

4 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
Citation locations 1 values source id lit-genomics-20261011-gfmbench-source-larey2026 locator Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.citation_locators` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Comparison group gfmbench-brca1 Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.comparison_group` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Comparison title GFMBench-API zero-shot scores on BRCA1 function classes Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.comparison_title` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Constraints 3 values Inspect every linked evaluation's source locator before citing a result. Do not combine this mapping's evaluations with any other protocol's results. Not a calibrated evidence-strength estimate; do not translate AUROC into ACMG/AMP points. Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.constraints` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Endpoint Zero-shot AUROC and AUPRC of masked log-likelihood ratio for masked genomic language models against 3,893 BRCA1 variants labelled LOF, INT or FUNC from experimentally measured DNA-repair activity Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.endpoint` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Claimed field links:assessed_by:lit-genomics-20261011-gfmbench-protocol-brca1-zs-mlm-llr Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.field` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Headline metric auroc Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.headline_metric` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Limitations 5 values Preprint, not peer reviewed. Single run; no uncertainty printed. The source does not state how the three function classes (LOF, INT, FUNC) were made binary for AUROC. BRCA1 only; no BRCA2 task. Assay labels, not clinical classifications. Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.limitations` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Pinned records 15 values record id lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-brca1-zs-mlm-llr fields status links attributes.origin attributes.comparison sha256 f8f9d08db79335a9cc41563d63e1a0f3d32eb43ae33761f72ea817332e9c1ef4 record id lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-brca1-zs-mlm-llr fields status links attributes.origin attributes.comparison sha256 0f32f2e1422ec032109480382687a8db56c365b03645e4d7996abcfa535a26dc record id lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-brca1-zs-mlm-llr fields status links attributes.origin attributes.comparison sha256 5090378d6d368912a47c924aaacdbb7f660d94cb1e7029704f1ffc0a12e5c0e0 record id lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-brca1-zs-mlm-llr fields status links attributes.origin attributes.comparison sha256 383f828e26ab2e07e13c248a0a92716562196c0aa031655a406a06b6ab5089e6 record id lit-genomics-20261011-gfmbench-protocol-brca1-zs-mlm-llr fields status name description facets source_ids links attributes sha256 9709c04fdb8abda319c01b5fd8366edd503698213e37c4fec2efa06d1be4764c record id lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-mlm-llr-msk-llr-prc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 5d6757e28b4362b385690ac15ea621415b7161fb549be5adc16c48d88d19ae91 record id lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-mlm-llr-msk-llr-roc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 9b070b73781bf58851de75d6a96e553ae7d320ebd7e451e6d704b29064f7ec2d record id lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-mlm-llr-msk-llr-prc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 9f8b0933de973cc26789ecf3c19981dc23379f0802ac77f16460477322090edb record id lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-mlm-llr-msk-llr-roc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 e2c4a630a568cae465fd56c26b6e76f133411b6e5ad0600e88b906e5a873698c record id lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-mlm-llr-msk-llr-prc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 65f44796d380a38fe7a43ca5025a25e817c7d8e6eec1bb0be57b55e4b0cd314b record id lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-mlm-llr-msk-llr-roc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 115b469f34d6e08b5c20b7a184d3a2151573feac6ba08215547bd1799f032b1e record id lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-mlm-llr-msk-llr-prc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 d2155773d23cea602de09658b9aa0823d083bfed4257476d20b67cfd3a968cad record id lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-mlm-llr-msk-llr-roc fields status attributes.metric attributes.metric_qualifier attributes.printed_value attributes.numeric_value sha256 99ff6c776b4fb92360f1fd6a9395d224114dc1ef5357463d985e255a844e5f5e record id lit-genomics-20261011-gfmbench-source-larey2026 fields status attributes.artifact_sha256 sha256 a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc record id use-case-brca1-brca2-germline-interpretation fields status name description facets attributes.question attributes.decision attributes.inputs attributes.output attributes.exclusions attributes.setting attributes.clinical_scope attributes.intended_users sha256 9681ae92e727dde7a801aec8d4884ec28b009d134a52d806a266f7d3b64bd5bd Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.pins` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Rationale Proxy, not direct: assay-derived function classes are one line of evidence a BRCA1 reviewer weighs, and this shows how well unsupervised scores track them; it does not test a classification workflow or ACMG/AMP evidence strength. Context-only references	GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models Original source ↗ lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task Version: bioRxiv preprint version 2, posted 2026-08-11 Retrieved: 2026-10-11T00:44:58Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.rationale` Source artifact SHA-256: `df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

1 source record and release history

GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models · Original source · bioRxiv preprint version 2, posted 2026-08-11

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-genomics-20261011-gfmbench-brca1-zs-mlm-llr

field: links:assessed_by:lit-genomics-20261011-gfmbench-protocol-brca1-zs-mlm-llr
value: lit-genomics-20261011-gfmbench-protocol-brca1-zs-mlm-llr
relevance: proxy
endpoint: Zero-shot AUROC and AUPRC of masked log-likelihood ratio for masked genomic language models against 3,893 BRCA1 variants labelled LOF, INT or FUNC from experimentally measured DNA-repair activity
rationale: Proxy, not direct: assay-derived function classes are one line of evidence a BRCA1 reviewer weighs, and this shows how well unsupervised scores track them; it does not test a classification workflow or ACMG/AMP evidence strength.
constraints: Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results.; Not a calibrated evidence-strength estimate; do not translate AUROC into ACMG/AMP points.
limitations: Preprint, not peer reviewed.; Single run; no uncertainty printed.; The source does not state how the three function classes (LOF, INT, FUNC) were made binary for AUROC.; BRCA1 only; no BRCA2 task.; Assay labels, not clinical classifications.
citation locators: source id: lit-genomics-20261011-gfmbench-source-larey2026; locator: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task
source locator: lit-genomics-20261011-gfmbench-source-larey2026: Table 4, task 'BRCA1'; Supplementary Note, BRCA1 task
headline metric: auroc
reason: Add a relevance judgement for a genomic foundation model benchmark protocol from the 2026-10-11 genomics suite pass, reviewed in docs/reviews/2026-10-11-genomics-gfm-suites.md.
comparison group: gfmbench-brca1
comparison title: GFMBench-API zero-shot scores on BRCA1 function classes
stratum label: masked log-likelihood ratio
stratum order: 2
reviewed evaluations: lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-brca1-zs-mlm-llr; lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-brca1-zs-mlm-llr; lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-brca1-zs-mlm-llr; lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-brca1-zs-mlm-llr
review: method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs and output, the proxy rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-3, independent of the extractor lit-genomics; no human review claimed; reviewed at: 2026-10-11T14:40:00Z; note: Automated review; not human scientific review.
pins: record id: lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-brca1-zs-mlm-llr; fields: status; links; attributes.origin; attributes.comparison; sha256: f8f9d08db79335a9cc41563d63e1a0f3d32eb43ae33761f72ea817332e9c1ef4; record id: lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-brca1-zs-mlm-llr; fields: status; links; attributes.origin; attributes.comparison; sha256: 0f32f2e1422ec032109480382687a8db56c365b03645e4d7996abcfa535a26dc; record id: lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-brca1-zs-mlm-llr; fields: status; links; attributes.origin; attributes.comparison; sha256: 5090378d6d368912a47c924aaacdbb7f660d94cb1e7029704f1ffc0a12e5c0e0; record id: lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-brca1-zs-mlm-llr; fields: status; links; attributes.origin; attributes.comparison; sha256: 383f828e26ab2e07e13c248a0a92716562196c0aa031655a406a06b6ab5089e6; record id: lit-genomics-20261011-gfmbench-protocol-brca1-zs-mlm-llr; fields: status; name; description; facets; source_ids; links; attributes; sha256: 9709c04fdb8abda319c01b5fd8366edd503698213e37c4fec2efa06d1be4764c; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-mlm-llr-msk-llr-prc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5d6757e28b4362b385690ac15ea621415b7161fb549be5adc16c48d88d19ae91; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-mlm-llr-msk-llr-roc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b070b73781bf58851de75d6a96e553ae7d320ebd7e451e6d704b29064f7ec2d; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-mlm-llr-msk-llr-prc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9f8b0933de973cc26789ecf3c19981dc23379f0802ac77f16460477322090edb; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-mlm-llr-msk-llr-roc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e2c4a630a568cae465fd56c26b6e76f133411b6e5ad0600e88b906e5a873698c; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-mlm-llr-msk-llr-prc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 65f44796d380a38fe7a43ca5025a25e817c7d8e6eec1bb0be57b55e4b0cd314b; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-mlm-llr-msk-llr-roc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 115b469f34d6e08b5c20b7a184d3a2151573feac6ba08215547bd1799f032b1e; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-mlm-llr-msk-llr-prc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d2155773d23cea602de09658b9aa0823d083bfed4257476d20b67cfd3a968cad; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-mlm-llr-msk-llr-roc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 99ff6c776b4fb92360f1fd6a9395d224114dc1ef5357463d985e255a844e5f5e; record id: lit-genomics-20261011-gfmbench-source-larey2026; fields: status; attributes.artifact_sha256; sha256: a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc; record id: use-case-brca1-brca2-germline-interpretation; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 9681ae92e727dde7a801aec8d4884ec28b009d134a52d806a266f7d3b64bd5bd

Related records

subject: Interpret BRCA1/BRCA2 germline variants

Suggest a correction