rewirebio.iobenchmarks
Evidence claim

Relevance of GFMBench-API TraitGym Complex, zs-embedding to "Select regulatory variants and genes for functional follow-up"

Use case
Select regulatory variants and genes for functional follow-up
Assessed by
GFMBench-API TraitGym Complex, zs-embedding
Relevance
Proxy evidence
Endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Complex

GFMBench-API zero-shot embedding distance scores scores on TraitGym Complex as evidence for choosing noncoding variants to edit.

5 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    lit-genomics-20261011-gfmbench-source-larey2026
    locator
    Table 3, task 'TraitGym Complex'; appendix task description
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
gfmbench-traitgym
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
GFMBench-API zero-shot scores on TraitGym
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
3 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Do not combine this mapping's evaluations with any other protocol's results.
  • Compare only with other rows of the same GFMBench-API table; TraitGym's own paper uses different context lengths and scoring.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Complex
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:lit-genomics-20261011-gfmbench-protocol-traitgym-complex-zs-embedding
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
auroc
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
3 values
  • Preprint, not peer reviewed.
  • Complex-trait AUROCs are all close to 0.5; differences between models are small and no uncertainty is printed.
  • Fine-mapping labels, not experimental allele effects.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
38 values
  • record id
    lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-complex-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    6c74ca0e56ebc2972bf7560ef6ddde062e2140eb64bb06b75ad102f0d17a1d1b
  • record id
    lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-complex-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    07b5669880ffcb9212c18c254f10c9dad277dc9ab9f1553c0f2d4a18b329dd0c
  • record id
    lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-complex-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    dcb87c568f3dfa69d86a8b92d912c4ef02ac4d2d7fb564168e10702c14dfa23c
  • record id
    lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-complex-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    6f842c2538c75c95f02ab924afdabdab6d446843b46478ff5092887a17b3a770
  • record id
    lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-complex-zs-embedding
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    29f7a30319120338494392648dc4d29b32511729380386d2b4effe9b5df51728
  • record id
    lit-genomics-20261011-gfmbench-protocol-traitgym-complex-zs-embedding
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    9396322276430768bc06971ab94c66227702bdcd446eb41b4bf6ce76e43f0e06
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    4246243905a2ebdf74738903ee0960235caf242fe16ab47371a649e6d0c2483a
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2ce3abc858f7ae8a734af0f77232f00816194884743f73602d23e268ae7389b1
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    cb108f584ddfce03cabfe309e3bac589ed0d6d5ee2442b3168a0379c47c2273f
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2e50684da75f0d34a9743e9cda794f66f8b1a7bf88dad653f5d724d220be71ba
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    121bbb72f86c25710fcd2a925b5bf841e29cebf199fec8c04568375d929e2b04
  • record id
    lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f66bb863b37b763e32060ff30edc69f23a8c213674f0cf9cf8ba82f11e99bdda
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b28a6f813d1370c0bb3f9d989105e5df3e43f254b4577cb7a605060e9450baf9
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7f01cdeb32a5e26f9b11d25457e2f1adc7a801f0504204df544518a7f71f8d60
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    58c73d05d0a41efa36312f8de7017ec777742c44bbe44ff5716455b22f5b1d94
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6db54039a2889ecab269342a3854f85ea1a2d81fc1c83dbb8d239b4ec477943c
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    4796625dc86e64e15732eedbde12811cd0b3a7f0e4942d2cf658ed3ce91b43e6
  • record id
    lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    438cc2d1e204d2142bc82d8b4a5d14a5cf1c66ad6940084ace5ae34f5b67ac09
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8e15cb4ffbff0efd2b2bd180a7650978320ca600ec278a29562e293e0bdcb14b
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    926efce12cb2264fd52787059eecee59c7c7aeda0a0503475c4007fba6730389
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    530650fda52a13a90e708309aa492ff6084986cc6156e308ae949770c401e4bf
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8eaf5006e45aea54dec1e40507ae85f8aef4b0da5be063df091b902afa1a7bcb
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    14cd8ea99cf98c59de99f56a0f8a2f8912965dcf0ff29e8b66279c2e85ef506f
  • record id
    lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    688ccde76735532761010bc39325199251a1aae43cb1de078f6af11478aa8e2b
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0bbd2c0580a00ae35cce0dcf474ea0a2e33779656f81b4a0a97fd250c3350f78
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f203665c652495a4b95ed689d6728768a6fecb2f71ebc230fa64ce4f953bbc7a
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a8ceb117bbb05aead465cd904fe416ed800cbe19eaacfa7d6359c15775abbf6f
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d937c0c0df86d4a7819b8a98926789222c2dba4cf04ea3b4dc8ebfdf2e3579b5
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    cf4cffe0a4fe4c592159595e21cee2ab64c9675b1c8bc2a033cf0db39bb408bd
  • record id
    lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    c566a5d18a6e963daa7e6ff8f77574fa1af8066c605b517028a4c68734a0fce4
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    db09874b3390f982adb7397afd78d67c2e965e9bd987204774313ff67672f5db
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2e0ef27f35fcdaac59f1ef5e40343b874b6f7a0205fa43ab4f7bb2b02eb43829
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-l2-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9559df84ef1e6a626acf0f6df23a4cf01f211e4d312bcf1f60538b9c40eff8bc
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-l2-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    bea7325c43dd266f57461600d6fe5e943ff80d311b5e082d72d9515c8c09cbd0
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    1c1e0dc855e13a185d764eaed9435a3a0ce1cb7971079aa5b4b68b47946df80c
  • record id
    lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    cddaa5abb8c21ba7df262af4cc9a96e93038a78c45afcc9567b148c41d284246
  • record id
    lit-genomics-20261011-gfmbench-source-larey2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc
  • record id
    use-case-regulatory-variant-gene-follow-up
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    0071547e01ccaec30238aea9e4882dd92e6f51be9c967c7fb88bc61d2788b870
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Proxy, not direct: the protocol ranks candidate noncoding variants by zero-shot model scores, which matches the first step of choosing allele edits, but it gives no element-gene link and no effect direction, and it is scored on the GFMBench-API re-implementation of TraitGym rather than on perturbation data.
Context-only references
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-genomics-20261011-gfmbench-traitgym-complex-zs-embedding

field
links:assessed_by:lit-genomics-20261011-gfmbench-protocol-traitgym-complex-zs-embedding
value
lit-genomics-20261011-gfmbench-protocol-traitgym-complex-zs-embedding
relevance
proxy
endpoint
Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Complex
rationale
Proxy, not direct: the protocol ranks candidate noncoding variants by zero-shot model scores, which matches the first step of choosing allele edits, but it gives no element-gene link and no effect direction, and it is scored on the GFMBench-API re-implementation of TraitGym rather than on perturbation data.
constraints
Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results.; Compare only with other rows of the same GFMBench-API table; TraitGym's own paper uses different context lengths and scoring.
limitations
Preprint, not peer reviewed.; Complex-trait AUROCs are all close to 0.5; differences between models are small and no uncertainty is printed.; Fine-mapping labels, not experimental allele effects.
citation locators
source id: lit-genomics-20261011-gfmbench-source-larey2026; locator: Table 3, task 'TraitGym Complex'; appendix task description
source locator
lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Complex'; appendix task description
headline metric
auroc
reason
Add a relevance judgement for a genomic foundation model benchmark protocol from the 2026-10-11 genomics suite pass, reviewed in docs/reviews/2026-10-11-genomics-gfm-suites.md.
comparison group
gfmbench-traitgym
comparison title
GFMBench-API zero-shot scores on TraitGym
stratum label
TraitGym Complex, embedding distance scores
stratum order
4
reviewed evaluations
lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-complex-zs-embedding; lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-complex-zs-embedding; lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-complex-zs-embedding; lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-complex-zs-embedding; lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-complex-zs-embedding
review
method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs and output, the proxy rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-3, independent of the extractor lit-genomics; no human review claimed; reviewed at: 2026-10-11T14:40:00Z; note: Automated review; not human scientific review.
pins
record id: lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-complex-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 6c74ca0e56ebc2972bf7560ef6ddde062e2140eb64bb06b75ad102f0d17a1d1b; record id: lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-complex-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 07b5669880ffcb9212c18c254f10c9dad277dc9ab9f1553c0f2d4a18b329dd0c; record id: lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-complex-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: dcb87c568f3dfa69d86a8b92d912c4ef02ac4d2d7fb564168e10702c14dfa23c; record id: lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-complex-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 6f842c2538c75c95f02ab924afdabdab6d446843b46478ff5092887a17b3a770; record id: lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-complex-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 29f7a30319120338494392648dc4d29b32511729380386d2b4effe9b5df51728; record id: lit-genomics-20261011-gfmbench-protocol-traitgym-complex-zs-embedding; fields: status; name; description; facets; source_ids; links; attributes; sha256: 9396322276430768bc06971ab94c66227702bdcd446eb41b4bf6ce76e43f0e06; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4246243905a2ebdf74738903ee0960235caf242fe16ab47371a649e6d0c2483a; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2ce3abc858f7ae8a734af0f77232f00816194884743f73602d23e268ae7389b1; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cb108f584ddfce03cabfe309e3bac589ed0d6d5ee2442b3168a0379c47c2273f; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2e50684da75f0d34a9743e9cda794f66f8b1a7bf88dad653f5d724d220be71ba; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 121bbb72f86c25710fcd2a925b5bf841e29cebf199fec8c04568375d929e2b04; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f66bb863b37b763e32060ff30edc69f23a8c213674f0cf9cf8ba82f11e99bdda; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b28a6f813d1370c0bb3f9d989105e5df3e43f254b4577cb7a605060e9450baf9; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7f01cdeb32a5e26f9b11d25457e2f1adc7a801f0504204df544518a7f71f8d60; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 58c73d05d0a41efa36312f8de7017ec777742c44bbe44ff5716455b22f5b1d94; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6db54039a2889ecab269342a3854f85ea1a2d81fc1c83dbb8d239b4ec477943c; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4796625dc86e64e15732eedbde12811cd0b3a7f0e4942d2cf658ed3ce91b43e6; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 438cc2d1e204d2142bc82d8b4a5d14a5cf1c66ad6940084ace5ae34f5b67ac09; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8e15cb4ffbff0efd2b2bd180a7650978320ca600ec278a29562e293e0bdcb14b; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 926efce12cb2264fd52787059eecee59c7c7aeda0a0503475c4007fba6730389; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 530650fda52a13a90e708309aa492ff6084986cc6156e308ae949770c401e4bf; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8eaf5006e45aea54dec1e40507ae85f8aef4b0da5be063df091b902afa1a7bcb; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 14cd8ea99cf98c59de99f56a0f8a2f8912965dcf0ff29e8b66279c2e85ef506f; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 688ccde76735532761010bc39325199251a1aae43cb1de078f6af11478aa8e2b; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0bbd2c0580a00ae35cce0dcf474ea0a2e33779656f81b4a0a97fd250c3350f78; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f203665c652495a4b95ed689d6728768a6fecb2f71ebc230fa64ce4f953bbc7a; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a8ceb117bbb05aead465cd904fe416ed800cbe19eaacfa7d6359c15775abbf6f; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d937c0c0df86d4a7819b8a98926789222c2dba4cf04ea3b4dc8ebfdf2e3579b5; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cf4cffe0a4fe4c592159595e21cee2ab64c9675b1c8bc2a033cf0db39bb408bd; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c566a5d18a6e963daa7e6ff8f77574fa1af8066c605b517028a4c68734a0fce4; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: db09874b3390f982adb7397afd78d67c2e965e9bd987204774313ff67672f5db; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2e0ef27f35fcdaac59f1ef5e40343b874b6f7a0205fa43ab4f7bb2b02eb43829; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9559df84ef1e6a626acf0f6df23a4cf01f211e4d312bcf1f60538b9c40eff8bc; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: bea7325c43dd266f57461600d6fe5e943ff80d311b5e082d72d9515c8c09cbd0; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 1c1e0dc855e13a185d764eaed9435a3a0ce1cb7971079aa5b4b68b47946df80c; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-complex-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cddaa5abb8c21ba7df262af4cc9a96e93038a78c45afcc9567b148c41d284246; record id: lit-genomics-20261011-gfmbench-source-larey2026; fields: status; attributes.artifact_sha256; sha256: a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc; record id: use-case-regulatory-variant-gene-follow-up; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 0071547e01ccaec30238aea9e4882dd92e6f51be9c967c7fb88bc61d2788b870
Related records

Suggest a correction