Relevance of GFMBench-API TraitGym Mendelian, zs-embedding to "Select regulatory variants and genes for functional follow-up"
- Use case
- Select regulatory variants and genes for functional follow-up
- Assessed by
- GFMBench-API TraitGym Mendelian, zs-embedding
- Relevance
- Proxy evidence
- Endpoint
- Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Mendelian
GFMBench-API zero-shot embedding distance scores scores on TraitGym Mendelian as evidence for choosing noncoding variants to edit.
5 reviewed evaluations
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
19 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group gfmbench-traitgym Context-only references | GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title GFMBench-API zero-shot scores on TraitGym Context-only references | GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints3 values
| GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Mendelian Context-only references | GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding Context-only references | GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric auroc Context-only references | GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations3 values
| GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records38 values
| GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale Proxy, not direct: the protocol ranks candidate noncoding variants by zero-shot model scores, which matches the first step of choosing allele edits, but it gives no element-gene link and no effect direction, and it is scored on the GFMBench-API re-implementation of TraitGym rather than on perturbation data. Context-only references | GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description Version: bioRxiv preprint version 2, posted 2026-08-11 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
1 source record and release history
- GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models · Original source · bioRxiv preprint version 2, posted 2026-08-11
Technical metadata and extraction receipts
Stable ID: use-case-mapping-lit-genomics-20261011-gfmbench-traitgym-mendelian-zs-embedding
- field
- links:assessed_by:lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding
- value
- lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding
- relevance
- proxy
- endpoint
- Zero-shot AUROC and AUPRC of embedding distance scores for genomic language models (DNA-BERT, DNABERT-2, NTv3, Caduceus-Ps, Evo2) on TraitGym Mendelian
- rationale
- Proxy, not direct: the protocol ranks candidate noncoding variants by zero-shot model scores, which matches the first step of choosing allele edits, but it gives no element-gene link and no effect direction, and it is scored on the GFMBench-API re-implementation of TraitGym rather than on perturbation data.
- constraints
- Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results.; Compare only with other rows of the same GFMBench-API table; TraitGym's own paper uses different context lengths and scoring.
- limitations
- Preprint, not peer reviewed.; Mendelian AUROCs reach 0.60-0.63 (Evo2 and DNABERT-2 L2 distance); no uncertainty is printed.; Curated Mendelian regulatory variants, not experimental allele effects.
- citation locators
- source id: lit-genomics-20261011-gfmbench-source-larey2026; locator: Table 3, task 'TraitGym Mendelian'; appendix task description
- source locator
- lit-genomics-20261011-gfmbench-source-larey2026: Table 3, task 'TraitGym Mendelian'; appendix task description
- headline metric
- auroc
- reason
- Add a relevance judgement for a genomic foundation model benchmark protocol from the 2026-10-11 genomics suite pass, reviewed in docs/reviews/2026-10-11-genomics-gfm-suites.md.
- comparison group
- gfmbench-traitgym
- comparison title
- GFMBench-API zero-shot scores on TraitGym
- stratum label
- TraitGym Mendelian, embedding distance scores
- stratum order
- 1
- reviewed evaluations
- lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-mendelian-zs-embedding; lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-mendelian-zs-embedding
- review
- method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs and output, the proxy rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-3, independent of the extractor lit-genomics; no human review claimed; reviewed at: 2026-10-11T14:40:00Z; note: Automated review; not human scientific review.
- pins
- record id: lit-genomics-20261011-gfmbench-eval-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: d9f72273d01a90dd8ccff055d42a462c2ce02c01364a1134a0717f72a172ed30; record id: lit-genomics-20261011-gfmbench-eval-dna-bert-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 093f0dc8575f4ea186473b7af7554e4527a7540ff4d7a2463f0a3cad06997e5d; record id: lit-genomics-20261011-gfmbench-eval-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 55ea286a3a7189b81365417f2e9a75725c39e362cfa9dd694c841b188777a0d0; record id: lit-genomics-20261011-gfmbench-eval-evo2-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 361d589a542d6739013fa60f519cadc5634daa9f0b19ee5049da3e3d423c284a; record id: lit-genomics-20261011-gfmbench-eval-ntv3-zero-shot-traitgym-mendelian-zs-embedding; fields: status; links; attributes.origin; attributes.comparison; sha256: 397d29ee00d5a7204646396dd4a7d729cfdba4499b8e51656c1a4cf73f1ef4b5; record id: lit-genomics-20261011-gfmbench-protocol-traitgym-mendelian-zs-embedding; fields: status; name; description; facets; source_ids; links; attributes; sha256: 27563f0e9ac8c3b1421003c8307f0a2c0d3e6a9175178603ba95a61c2f7e320b; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 54127ba2be121d743876e25732f408a5e442fa031dad07c1000a7a9a60cfc8c2; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 18dcc008efdc40b7fe11b04c511254395f3a6234f9c471a2761453c64006914c; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5937648877e8786fe31f925b3f1a93347841039548c63fc66e18387b62b50826; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5cb621ebe2ed0feeb76b2b779796f285949d29824b4334fd8fde4088799fbd41; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 87a9730652242f33bbba7ee88879043e138fc2e09a67d89b54b526ad4d22432b; record id: lit-genomics-20261011-gfmbench-result-caduceus-ps-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7ec717cf2e67f727d9fad14d093a24a3ea7af7acdbcbfe8d1509b2fa8e32e0cd; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: fa44ca4f2688e74f0b279f4781e199a8dcf8643229288e61556d71fcea642b55; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f879c2d3cf8c5954219708d269709bf88300189c3561e97b367325a2f6d1b1b3; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a362d7a0217d949872c23c118a363d2fdd0646a1bc51f0ae88ac2e5a83e9139c; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 40d1b8eae31c2be1b2c3f6ef8ca1b85490d7556cdaed7643a5117b56351d6756; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5cd3ba4c61b8648030f5fabd09129f6f0dcbbe0c78fe184340ea5b64b1e9317d; record id: lit-genomics-20261011-gfmbench-result-dna-bert-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0fa86afe5262ba03d713826e3b58588fe0bf2b0df67e56eedd257fa3202573a5; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 44785a3262fa4dbb54649cca8bf3cef1e2c7c33b927cb2f10d40f8bd46a568ae; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3c1b3d3eb898c783addf1e68042e5514e120be95d94c0cf3f37ab758946d8221; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b143ca95284e4ebef758e4ee6a1af085d5a00cbf824a101860b67e9da89f94e0; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 29628712f0e758040633d7ca894e52295845e0d0d910e154bd02f15f2683ea27; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 576cc699aa55f801c83b143aae47f611238b5fc313ae922888a925b363e1ffaf; record id: lit-genomics-20261011-gfmbench-result-dnabert-2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f89cbe81fe52d12a2496f87621a4fccf267c925f1cb9ab9b7f3911df320fce39; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eeed987c1e112ac87e0ed94ae8143d50789627232d2f708f8a816b9ed0f0ebde; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6d7634bc63f3aa95aedff5cc1b71a996d68edd0a9a128b89bed451e781ad152f; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3db080f61f7e265e22fc79dfbdd0b562afa057b1ea81f51877f1f7d1f3304df0; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d0f4ba9b343a7dc2176db919bbbaab36c1c3e33c88647a03fd84df79f675a2cc; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5eb53f7e4908518c8e84cf34c7f929e173b79a7fd949563390ddcc106f1512ed; record id: lit-genomics-20261011-gfmbench-result-evo2-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eb32bebfc5a3a27afb09eed5928455171dd8396df4270bca213c773c784e4de0; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e46ce4fa6fe4b95c75b29fa0961e8b55bcb1612382383259420d29424e0f73e2; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7dd6ffe0a37b00c4e52c9fcfccbf4351a706fac59f2f360d08070ee3d422affe; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-l2-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 13437e9e44559d0400fb2ee4910dce378daa95aea64fa85cb43e2285ba44d124; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-l2-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 35bc05b2c404ada49a2b6853d680c1f169b347b2c6fb60cb537ed0b31f6ec706; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auprc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: fb469cd9d330e49dec7cda42e63ac7f620bfc8e3118452cd817f34d74b909f0f; record id: lit-genomics-20261011-gfmbench-result-ntv3-zero-shot-traitgym-mendelian-zs-embedding-snv-cos-auroc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2e0dcb5a7eb53797b674761d3bd81d61dd835f9d18a92dba15c545820182b854; record id: lit-genomics-20261011-gfmbench-source-larey2026; fields: status; attributes.artifact_sha256; sha256: a461e55e4db9e70efca0dd40a686ae239e2e254757de72623099c721fd6eb0fc; record id: use-case-regulatory-variant-gene-follow-up; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 0071547e01ccaec30238aea9e4882dd92e6f51be9c967c7fb88bc61d2788b870