rewirebio.iobenchmarks
Evidence claim

Relevance of GFMBench-API TraitGym Mendelian, zs-embedding to "Select regulatory variants and genes for functional follow-up"

Use case
Select regulatory variants and genes for functional follow-up
Assessed by
GFMBench-API TraitGym Mendelian, zs-embedding
Relevance
Proxy evidence
Endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Mendelian

GFMBench-API zero-shot embedding distance scores scores on TraitGym Mendelian as evidence for choosing noncoding variants to edit.

5 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    lit-genomics-20261011-gfmbench-source-larey2026
    locator
    Table 3, task 'TraitGym Mendelian'; appendix task description
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
gfmbench-traitgym
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
GFMBench-API zero-shot scores on TraitGym
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
3 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Do not combine this mapping's evaluations with any other protocol's results.
  • Compare only with other rows of the same GFMBench-API table; TraitGym's own paper uses different context lengths and scoring.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Mendelian
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
auroc
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
3 values
  • Preprint, not peer reviewed.
  • Mendelian AUROCs reach 0.60-0.63 (Evo2 and DNABERT-2 L2 distance); no uncertainty is printed.
  • Curated Mendelian regulatory variants, not experimental allele effects.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
38 values
  • record id
    lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    d9f72273d01a90dd8ccff055d42a462c2ce02c01364a1134a0717f72a172ed30
  • record id
    lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-mendelian-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    093f0dc8575f4ea186473b7af7554e4527a7540ff4d7a2463f0a3cad06997e5d
  • record id
    lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    55ea286a3a7189b81365417f2e9a75725c39e362cfa9dd694c841b188777a0d0
  • record id
    lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-mendelian-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    361d589a542d6739013fa60f519cadc5634daa9f0b19ee5049da3e3d423c284a
  • record id
    lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-mendelian-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    397d29ee00d5a7204646396dd4a7d729cfdba4499b8e51656c1a4cf73f1ef4b5
  • record id
    lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    27563f0e9ac8c3b1421003c8307f0a2c0d3e6a9175178603ba95a61c2f7e320b
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    54127ba2be121d743876e25732f408a5e442fa031dad07c1000a7a9a60cfc8c2
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    18dcc008efdc40b7fe11b04c511254395f3a6234f9c471a2761453c64006914c
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5937648877e8786fe31f925b3f1a93347841039548c63fc66e18387b62b50826
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5cb621ebe2ed0feeb76b2b779796f285949d29824b4334fd8fde4088799fbd41
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    87a9730652242f33bbba7ee88879043e138fc2e09a67d89b54b526ad4d22432b
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7ec717cf2e67f727d9fad14d093a24a3ea7af7acdbcbfe8d1509b2fa8e32e0cd
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    fa44ca4f2688e74f0b279f4781e199a8dcf8643229288e61556d71fcea642b55
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f879c2d3cf8c5954219708d269709bf88300189c3561e97b367325a2f6d1b1b3
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a362d7a0217d949872c23c118a363d2fdd0646a1bc51f0ae88ac2e5a83e9139c
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    40d1b8eae31c2be1b2c3f6ef8ca1b85490d7556cdaed7643a5117b56351d6756
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5cd3ba4c61b8648030f5fabd09129f6f0dcbbe0c78fe184340ea5b64b1e9317d
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0fa86afe5262ba03d713826e3b58588fe0bf2b0df67e56eedd257fa3202573a5
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    44785a3262fa4dbb54649cca8bf3cef1e2c7c33b927cb2f10d40f8bd46a568ae
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    3c1b3d3eb898c783addf1e68042e5514e120be95d94c0cf3f37ab758946d8221
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b143ca95284e4ebef758e4ee6a1af085d5a00cbf824a101860b67e9da89f94e0
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    29628712f0e758040633d7ca894e52295845e0d0d910e154bd02f15f2683ea27
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    576cc699aa55f801c83b143aae47f611238b5fc313ae922888a925b363e1ffaf
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f89cbe81fe52d12a2496f87621a4fccf267c925f1cb9ab9b7f3911df320fce39
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    eeed987c1e112ac87e0ed94ae8143d50789627232d2f708f8a816b9ed0f0ebde
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6d7634bc63f3aa95aedff5cc1b71a996d68edd0a9a128b89bed451e781ad152f
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    3db080f61f7e265e22fc79dfbdd0b562afa057b1ea81f51877f1f7d1f3304df0
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d0f4ba9b343a7dc2176db919bbbaab36c1c3e33c88647a03fd84df79f675a2cc
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5eb53f7e4908518c8e84cf34c7f929e173b79a7fd949563390ddcc106f1512ed
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    eb32bebfc5a3a27afb09eed5928455171dd8396df4270bca213c773c784e4de0
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    e46ce4fa6fe4b95c75b29fa0961e8b55bcb1612382383259420d29424e0f73e2
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7dd6ffe0a37b00c4e52c9fcfccbf4351a706fac59f2f360d08070ee3d422affe
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    13437e9e44559d0400fb2ee4910dce378daa95aea64fa85cb43e2285ba44d124
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    35bc05b2c404ada49a2b6853d680c1f169b347b2c6fb60cb537ed0b31f6ec706
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    fb469cd9d330e49dec7cda42e63ac7f620bfc8e3118452cd817f34d74b909f0f
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2e0dcb5a7eb53797b674761d3bd81d61dd835f9d18a92dba15c545820182b854
  • record id
    lit-genomics-20261011-gfmbench-source-larey2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc
  • record id
    use-case-regulatory-variant-gene-follow-up
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    0071547e01ccaec30238aea9e4882dd92e6f51be9c967c7fb88bc61d2788b870
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Proxy, not direct: the protocol ranks candidate noncoding variants by zero-shot model scores, which matches the first step of choosing allele edits, but it gives no element-gene link and no effect direction, and it is scored on the GFMBench-API re-implementation of TraitGym rather than on perturbation data.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-genomics-20261011-gfmbench-traitgym-mendelian-zs-embedding

field
links:assessed_by:lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding
value
lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding
relevance
proxy
endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Mendelian
rationale
Proxy, not direct: the protocol ranks candidate noncoding variants by zero-shot model scores, which matches the first step of choosing allele edits, but it gives no element-gene link and no effect direction, and it is scored on the GFMBench-API re-implementation of TraitGym rather than on perturbation data.
constraints
Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results.; Compare only with other rows of the same GFMBench-API table; TraitGym's own paper uses different context lengths and scoring.
limitations
Preprint, not peer reviewed.; Mendelian AUROCs reach 0.60-0.63 (Evo2 and DNABERT-2 L2 distance); no uncertainty is printed.; Curated Mendelian regulatory variants, not experimental allele effects.
citation locators
source id: lit-genomics-20261011-gfmbench-source-larey2026; locator: Table 3, task 'TraitGym Mendelian'; appendix task description
source locator
lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description
headline metric
auroc
reason
Add a relevance judgement for a genomic foundation model benchmark protocol from the 2026-10-11 genomics suite pass, reviewed in docs/reviews/2026-10-11-genomics-gfm-suites.md.
comparison group
gfmbench-traitgym
comparison title
GFMBench-API zero-shot scores on TraitGym
stratum label
TraitGym Mendelian, embedding distance scores
stratum order
1
reviewed evaluations
lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-mendelian-zs-embedding
review
method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs and output, the proxy rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-3, independent of the extractor lit-genomics; no human review claimed; reviewed at: 2026-10-11T14:40:00Z; note: Automated review; not human scientific review.
pins
record id: lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: d9f72273d01a90dd8ccff055d42a462c2ce02c01364a1134a0717f72a172ed30; record id: lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 093f0dc8575f4ea186473b7af7554e4527a7540ff4d7a2463f0a3cad06997e5d; record id: lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 55ea286a3a7189b81365417f2e9a75725c39e362cfa9dd694c841b188777a0d0; record id: lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 361d589a542d6739013fa60f519cadc5634daa9f0b19ee5049da3e3d423c284a; record id: lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 397d29ee00d5a7204646396dd4a7d729cfdba4499b8e51656c1a4cf73f1ef4b5; record id: lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding; fields: status; name; description; facets; source_ids; links; attributes; sha256: 27563f0e9ac8c3b1421003c8307f0a2c0d3e6a9175178603ba95a61c2f7e320b; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 54127ba2be121d743876e25732f408a5e442fa031dad07c1000a7a9a60cfc8c2; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 18dcc008efdc40b7fe11b04c511254395f3a6234f9c471a2761453c64006914c; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5937648877e8786fe31f925b3f1a93347841039548c63fc66e18387b62b50826; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5cb621ebe2ed0feeb76b2b779796f285949d29824b4334fd8fde4088799fbd41; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 87a9730652242f33bbba7ee88879043e138fc2e09a67d89b54b526ad4d22432b; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7ec717cf2e67f727d9fad14d093a24a3ea7af7acdbcbfe8d1509b2fa8e32e0cd; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: fa44ca4f2688e74f0b279f4781e199a8dcf8643229288e61556d71fcea642b55; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f879c2d3cf8c5954219708d269709bf88300189c3561e97b367325a2f6d1b1b3; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a362d7a0217d949872c23c118a363d2fdd0646a1bc51f0ae88ac2e5a83e9139c; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 40d1b8eae31c2be1b2c3f6ef8ca1b85490d7556cdaed7643a5117b56351d6756; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5cd3ba4c61b8648030f5fabd09129f6f0dcbbe0c78fe184340ea5b64b1e9317d; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0fa86afe5262ba03d713826e3b58588fe0bf2b0df67e56eedd257fa3202573a5; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 44785a3262fa4dbb54649cca8bf3cef1e2c7c33b927cb2f10d40f8bd46a568ae; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3c1b3d3eb898c783addf1e68042e5514e120be95d94c0cf3f37ab758946d8221; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b143ca95284e4ebef758e4ee6a1af085d5a00cbf824a101860b67e9da89f94e0; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 29628712f0e758040633d7ca894e52295845e0d0d910e154bd02f15f2683ea27; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 576cc699aa55f801c83b143aae47f611238b5fc313ae922888a925b363e1ffaf; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f89cbe81fe52d12a2496f87621a4fccf267c925f1cb9ab9b7f3911df320fce39; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eeed987c1e112ac87e0ed94ae8143d50789627232d2f708f8a816b9ed0f0ebde; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6d7634bc63f3aa95aedff5cc1b71a996d68edd0a9a128b89bed451e781ad152f; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3db080f61f7e265e22fc79dfbdd0b562afa057b1ea81f51877f1f7d1f3304df0; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d0f4ba9b343a7dc2176db919bbbaab36c1c3e33c88647a03fd84df79f675a2cc; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5eb53f7e4908518c8e84cf34c7f929e173b79a7fd949563390ddcc106f1512ed; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eb32bebfc5a3a27afb09eed5928455171dd8396df4270bca213c773c784e4de0; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e46ce4fa6fe4b95c75b29fa0961e8b55bcb1612382383259420d29424e0f73e2; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7dd6ffe0a37b00c4e52c9fcfccbf4351a706fac59f2f360d08070ee3d422affe; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 13437e9e44559d0400fb2ee4910dce378daa95aea64fa85cb43e2285ba44d124; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 35bc05b2c404ada49a2b6853d680c1f169b347b2c6fb60cb537ed0b31f6ec706; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: fb469cd9d330e49dec7cda42e63ac7f620bfc8e3118452cd817f34d74b909f0f; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2e0dcb5a7eb53797b674761d3bd81d61dd835f9d18a92dba15c545820182b854; record id: lit-genomics-20261011-gfmbench-source-larey2026; fields: status; attributes.artifact_sha256; sha256: a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc; record id: use-case-regulatory-variant-gene-follow-up; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 0071547e01ccaec30238aea9e4882dd92e6f51be9c967c7fb88bc61d2788b870
Related records

Suggest a correction