rewirebio.iobenchmarks
Evidence claim

Relevance of GFMBench-API BRCA1, zs-embedding to "Interpret BRCA1/BRCA2 germline variants"

Use case
Interpret BRCA1/BRCA2 germline variants
Assessed by
GFMBench-API BRCA1, zs-embedding
Relevance
Proxy evidence
Endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models against 3,893 BRCA1 variants labelled LOF, INT or FUNC from experimentally measured DNA-repair activity

GFMBench-API zero-shot embedding distance scores scores against BRCA1 DNA-repair function classes as evidence for BRCA1 variant review.

5 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    lit-genomics-20261011-gfmbench-source-larey2026
    locator
    Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
gfmbench-brca1
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
GFMBench-API zero-shot scores on BRCA1 function classes
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
3 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Do not combine this mapping's evaluations with any other protocol's results.
  • Not a calibrated evidence-strength estimate; do not translate AUROC into ACMG/AMP points.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models against 3,893 BRCA1 variants labelled LOF, INT or FUNC from experimentally measured DNA-repair activity
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:lit-genomics-20261011-gfmbench-protocol-brca1-zs-embedding
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
auroc
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
5 values
  • Preprint, not peer reviewed.
  • Single run; no uncertainty printed.
  • The source does not state how the three function classes (LOF, INT, FUNC) were made binary for AUROC.
  • BRCA1 only; no BRCA2 task.
  • Assay labels, not clinical classifications.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
38 values
  • record id
    lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-brca1-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    daea412c25045da95d8044a6f15c538476513cc000c4409fdf5f64097750c676
  • record id
    lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-brca1-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    b98364d0f5aaaf123f919b3ccafdab261cd42b73a7805f25f2885c4571602f9d
  • record id
    lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-brca1-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    53475d0039a7f50d6724c399dcb366cf5c8f947cb85dfe38189f55851383f8b4
  • record id
    lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-brca1-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    69c186b5cc114becec54d76d95a48bc6393d3eef21b150b57ceb2300f9d46a5e
  • record id
    lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-brca1-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    51ba600e62ba11e3f910a199e4b44a437b089b9e1dae8dbc0708006fa9e63c79
  • record id
    lit-genomics-20261011-gfmbench-protocol-brca1-zs-embedding
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    64d5713dbd2c2a513355722ba98de51a29eea4558d2d5ef1c8623c3ee6974d24
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b2755e2ddb07568cbc6a8e1b52dac2863d83dfa8abb66d87a15c8a3861a70209
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9e04e7336a4fb7dadd0f981a05439643e264ea1e92c717b9f187af21b19b44ef
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a59be8f4688c97eea7b3e2bf963a9cf8e93be364742f73cf1da2960cb55cd2e7
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0d0fb0936240f774816e48c750ea150bd010458e81ecd6a4f5ad5fb80c7d92a6
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7b5a5f52d96171d0a8875a31ff561998a8c754ed1a31ee1d15120011630c0634
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    be8af26026c3b41f2f8b8e563b86049c47e392b5092c230aa7bd22bfb64e955a
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    e037b8ea007c24fc8e957963c62be2ef734b81f4886cf4cd467293d41a17858d
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0861da4bb4ec51875ce4338f6b44f482ded7605bcb4cd3da8c0090eb78a084bb
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    207eed17c0df56c2e871ac0141c7d50d19ce5309b858521bd0cf9aac4087abb2
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b319166d9b68edaca579b3a224c409f88dc71499674a8ced3a121f6d9955cbf0
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    116b6311a23f53881f0168e2058f7a60abb4c880aff58e3f7abef40ad8df2e29
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    db64ad55bbd1d1993012f77245b2afc7b11886a8d8150d28b5a16ecba87b3bb1
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7e03d1f0590281bb4a8e2ffbf016b2e8404e191e9c420bddfd83de8830c65088
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b3435460d2dab37c6d85ef8e35ab1293b138b48fa3dd82eade94269058e697c9
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f3b8e566e8d6acdf62e9b75a107683c883bee63e2e04fb89a18193623d3ccf65
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    df664b6d3e0733dc33f5aab174e4f3bfca3d1e52bb028be882adf532e5f448f4
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ea6635336fde11ca731464f239cd47a2f5cf1bb3126cf887981960dc1ba4e543
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8aa15de93443ed13fc3682448e60c66ac02c4dc4d3e159e4bed3d316edfbbf16
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    36b05e9820a33fc1b71bdef1c66cd6423abed253fecc66e8d2878a010eea0566
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d109add4bffd6ff4f553fbf0532bb040dba13ec3dd0e4d19c8ded677a53ef276
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ca4528053d0c9121fb6b4007ca96860dcf06502078bd371795035747e82114f2
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    24930ee0b9a1bbded5444bc5a80cff4372eadc422ea3f4dbe4dcab0e0805b85b
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d00bebdf07e25aea31251331ec8024af4259a2ec4e1ebc7890ebf0031ac9902a
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    eb6020162f64594d419375f701747d38767631dfe65535aa839b2a1e57c4d4a1
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    c23dcec849cc46e0ecfb59049613901e89563801944a7efe422f314b6ea61108
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9c77f24c5ef74f5120261fbc56c59510d189090855ed82feda3d75f2521c3eac
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    03541952564a7afa282813b837433850cd2f047a3e23b5b5e53c0aefa742bf9f
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    60fee6ff9bf76481a2e9986ee3bb49db9b6b044281010b8dd9ac00732cc75b47
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ccde0c4160aa3d9976941977881cb7f2b24a8c60715e98fab929af21bad9f45d
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7e7f98e85c502ae0fd700cda7f6601eb50efca6ff8295de7442753ad7445d92e
  • record id
    lit-genomics-20261011-gfmbench-source-larey2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc
  • record id
    use-case-brca1-brca2-germline-interpretation
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    9681ae92e727dde7a801aec8d4884ec28b009d134a52d806a266f7d3b64bd5bd
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Proxy, not direct: assay-derived function classes are one line of evidence a BRCA1 reviewer weighs, and this shows how well unsupervised scores track them; it does not test a classification workflow or ACMG/AMP evidence strength.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-genomics-20261011-gfmbench-brca1-zs-embedding

field
links:assessed_by:lit-genomics-20261011-gfmbench-protocol-brca1-zs-embedding
value
lit-genomics-20261011-gfmbench-protocol-brca1-zs-embedding
relevance
proxy
endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models against 3,893 BRCA1 variants labelled LOF, INT or FUNC from experimentally measured DNA-repair activity
rationale
Proxy, not direct: assay-derived function classes are one line of evidence a BRCA1 reviewer weighs, and this shows how well unsupervised scores track them; it does not test a classification workflow or ACMG/AMP evidence strength.
constraints
Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results.; Not a calibrated evidence-strength estimate; do not translate AUROC into ACMG/AMP points.
limitations
Preprint, not peer reviewed.; Single run; no uncertainty printed.; The source does not state how the three function classes (LOF, INT, FUNC) were made binary for AUROC.; BRCA1 only; no BRCA2 task.; Assay labels, not clinical classifications.
citation locators
source id: lit-genomics-20261011-gfmbench-source-larey2026; locator: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task
source locator
lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'BRCA1'; Supplementary Note, BRCA1 task
headline metric
auroc
reason
Add a relevance judgement for a genomic foundation model benchmark protocol from the 2026-10-11 genomics suite pass, reviewed in docs/reviews/2026-10-11-genomics-gfm-suites.md.
comparison group
gfmbench-brca1
comparison title
GFMBench-API zero-shot scores on BRCA1 function classes
stratum label
embedding distance scores
stratum order
1
reviewed evaluations
lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-brca1-zs-embedding; lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-brca1-zs-embedding; lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-brca1-zs-embedding; lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-brca1-zs-embedding; lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-brca1-zs-embedding
review
method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs and output, the proxy rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-3, independent of the extractor lit-genomics; no human review claimed; reviewed at: 2026-10-11T14:40:00Z; note: Automated review; not human scientific review.
pins
record id: lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-brca1-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: daea412c25045da95d8044a6f15c538476513cc000c4409fdf5f64097750c676; record id: lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-brca1-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: b98364d0f5aaaf123f919b3ccafdab261cd42b73a7805f25f2885c4571602f9d; record id: lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-brca1-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 53475d0039a7f50d6724c399dcb366cf5c8f947cb85dfe38189f55851383f8b4; record id: lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-brca1-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 69c186b5cc114becec54d76d95a48bc6393d3eef21b150b57ceb2300f9d46a5e; record id: lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-brca1-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 51ba600e62ba11e3f910a199e4b44a437b089b9e1dae8dbc0708006fa9e63c79; record id: lit-genomics-20261011-gfmbench-protocol-brca1-zs-embedding; fields: status; name; description; facets; source_ids; links; attributes; sha256: 64d5713dbd2c2a513355722ba98de51a29eea4558d2d5ef1c8623c3ee6974d24; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b2755e2ddb07568cbc6a8e1b52dac2863d83dfa8abb66d87a15c8a3861a70209; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9e04e7336a4fb7dadd0f981a05439643e264ea1e92c717b9f187af21b19b44ef; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a59be8f4688c97eea7b3e2bf963a9cf8e93be364742f73cf1da2960cb55cd2e7; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0d0fb0936240f774816e48c750ea150bd010458e81ecd6a4f5ad5fb80c7d92a6; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7b5a5f52d96171d0a8875a31ff561998a8c754ed1a31ee1d15120011630c0634; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-brca1-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: be8af26026c3b41f2f8b8e563b86049c47e392b5092c230aa7bd22bfb64e955a; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e037b8ea007c24fc8e957963c62be2ef734b81f4886cf4cd467293d41a17858d; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0861da4bb4ec51875ce4338f6b44f482ded7605bcb4cd3da8c0090eb78a084bb; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 207eed17c0df56c2e871ac0141c7d50d19ce5309b858521bd0cf9aac4087abb2; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b319166d9b68edaca579b3a224c409f88dc71499674a8ced3a121f6d9955cbf0; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 116b6311a23f53881f0168e2058f7a60abb4c880aff58e3f7abef40ad8df2e29; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-brca1-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: db64ad55bbd1d1993012f77245b2afc7b11886a8d8150d28b5a16ecba87b3bb1; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7e03d1f0590281bb4a8e2ffbf016b2e8404e191e9c420bddfd83de8830c65088; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b3435460d2dab37c6d85ef8e35ab1293b138b48fa3dd82eade94269058e697c9; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f3b8e566e8d6acdf62e9b75a107683c883bee63e2e04fb89a18193623d3ccf65; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: df664b6d3e0733dc33f5aab174e4f3bfca3d1e52bb028be882adf532e5f448f4; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ea6635336fde11ca731464f239cd47a2f5cf1bb3126cf887981960dc1ba4e543; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-brca1-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8aa15de93443ed13fc3682448e60c66ac02c4dc4d3e159e4bed3d316edfbbf16; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 36b05e9820a33fc1b71bdef1c66cd6423abed253fecc66e8d2878a010eea0566; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d109add4bffd6ff4f553fbf0532bb040dba13ec3dd0e4d19c8ded677a53ef276; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ca4528053d0c9121fb6b4007ca96860dcf06502078bd371795035747e82114f2; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 24930ee0b9a1bbded5444bc5a80cff4372eadc422ea3f4dbe4dcab0e0805b85b; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d00bebdf07e25aea31251331ec8024af4259a2ec4e1ebc7890ebf0031ac9902a; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-brca1-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eb6020162f64594d419375f701747d38767631dfe65535aa839b2a1e57c4d4a1; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c23dcec849cc46e0ecfb59049613901e89563801944a7efe422f314b6ea61108; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9c77f24c5ef74f5120261fbc56c59510d189090855ed82feda3d75f2521c3eac; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 03541952564a7afa282813b837433850cd2f047a3e23b5b5e53c0aefa742bf9f; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 60fee6ff9bf76481a2e9986ee3bb49db9b6b044281010b8dd9ac00732cc75b47; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ccde0c4160aa3d9976941977881cb7f2b24a8c60715e98fab929af21bad9f45d; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-brca1-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7e7f98e85c502ae0fd700cda7f6601eb50efca6ff8295de7442753ad7445d92e; record id: lit-genomics-20261011-gfmbench-source-larey2026; fields: status; attributes.artifact_sha256; sha256: a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc; record id: use-case-brca1-brca2-germline-interpretation; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 9681ae92e727dde7a801aec8d4884ec28b009d134a52d806a266f7d3b64bd5bd
Related records

Suggest a correction