rewirebio.iobenchmarks
Source

GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

bioRxiv preprint version 2, posted 2026-08-11 · Preprint

DOI: 10.64898/2026.02.19.706811

What this source contributed

Records in this release that cite this source. A record can cite several sources.

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

ValueMetricModel or configurationDatasetEvaluation
0.1071 unitlessAUPRC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-embedding)
0.3782 unitlessAUPRC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-embedding)
0.3788 unitlessAUPRC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-embedding)
0.5446 unitlessAUROC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-embedding)
0.7724 unitlessAUROC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-embedding)
0.7798 unitlessAUROC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-embedding)
0.1397 unitlessAUPRC (zero-shot; masked log-likelihood ratio)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-mlm-llr)
0.6598 unitlessAUROC (zero-shot; masked log-likelihood ratio)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Disease test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Disease (zs-mlm-llr)
0.0775 unitlessAUPRC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-embedding)
0.0676 unitlessAUPRC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-embedding)
0.0692 unitlessAUPRC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-embedding)
0.5371 unitlessAUROC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-embedding)
0.4729 unitlessAUROC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-embedding)
0.4794 unitlessAUROC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-embedding)
0.0679 unitlessAUPRC (zero-shot; masked log-likelihood ratio)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-mlm-llr)
0.4886 unitlessAUROC (zero-shot; masked log-likelihood ratio)Caduceus-Ps (zero-shot, GFMBench-API)Bend Variant Effects Expression test set (GFMBench-API)Caduceus-Ps on GFMBench-API Bend Variant Effects Expression (zs-mlm-llr)
0.2025 unitlessAUPRC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-embedding)
0.2910 unitlessAUPRC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-embedding)
0.2986 unitlessAUPRC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-embedding)
0.4690 unitlessAUROC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-embedding)
0.5553 unitlessAUROC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-embedding)
0.5631 unitlessAUROC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-embedding)
0.2180 unitlessAUPRC (zero-shot; masked log-likelihood ratio)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-mlm-llr)
0.4832 unitlessAUROC (zero-shot; masked log-likelihood ratio)Caduceus-Ps (zero-shot, GFMBench-API)BRCA1 test set (GFMBench-API)Caduceus-Ps on GFMBench-API BRCA1 (zs-mlm-llr)
0.9581 unitlessAUPRC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)ClinVar Indel Pathogenicity test set (GFMBench-API)Caduceus-Ps on GFMBench-API ClinVar Indel Pathogenicity (zs-embedding)
0.9585 unitlessAUPRC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)ClinVar Indel Pathogenicity test set (GFMBench-API)Caduceus-Ps on GFMBench-API ClinVar Indel Pathogenicity (zs-embedding)
0.7080 unitlessAUROC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)ClinVar Indel Pathogenicity test set (GFMBench-API)Caduceus-Ps on GFMBench-API ClinVar Indel Pathogenicity (zs-embedding)
0.7100 unitlessAUROC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)ClinVar Indel Pathogenicity test set (GFMBench-API)Caduceus-Ps on GFMBench-API ClinVar Indel Pathogenicity (zs-embedding)
0.9012 unitlessAccuracy (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Promoter All test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Promoter All (supervised)
0.9601 unitlessAUPRC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Promoter All test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Promoter All (supervised)
0.9643 unitlessAUROC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Promoter All test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Promoter All (supervised)
0.8024 unitlessMCC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Promoter All test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Promoter All (supervised)
0.8963 unitlessAccuracy (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Splice Site test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Splice Site (supervised)
0.9174 unitlessAUPRC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Splice Site test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Splice Site (supervised)
0.9551 unitlessAUROC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Splice Site test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Splice Site (supervised)
0.8207 unitlessMCC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Splice Site test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Splice Site (supervised)
0.8056 unitlessAccuracy (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Transcription Factor test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Transcription Factor (supervised)
0.8875 unitlessAUPRC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Transcription Factor test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Transcription Factor (supervised)
0.8909 unitlessAUROC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Transcription Factor test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Transcription Factor (supervised)
0.6140 unitlessMCC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)GUE Transcription Factor test set (GFMBench-API)Caduceus-Ps on GFMBench-API GUE Transcription Factor (supervised)
0.0861 unitlessAUPRC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)LOL-EVE test set (GFMBench-API)Caduceus-Ps on GFMBench-API LOL-EVE (zs-embedding)
0.0863 unitlessAUPRC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)LOL-EVE test set (GFMBench-API)Caduceus-Ps on GFMBench-API LOL-EVE (zs-embedding)
0.5242 unitlessAUROC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)LOL-EVE test set (GFMBench-API)Caduceus-Ps on GFMBench-API LOL-EVE (zs-embedding)
0.5216 unitlessAUROC (zero-shot; L2 distance of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)LOL-EVE test set (GFMBench-API)Caduceus-Ps on GFMBench-API LOL-EVE (zs-embedding)
0.6286 unitlessAccuracy (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)LRB Variant Effect Causal eQTL test set (GFMBench-API)Caduceus-Ps on GFMBench-API LRB Variant Effect Causal eQTL (supervised)
0.6856 unitlessAUPRC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)LRB Variant Effect Causal eQTL test set (GFMBench-API)Caduceus-Ps on GFMBench-API LRB Variant Effect Causal eQTL (supervised)
0.6792 unitlessAUROC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)LRB Variant Effect Causal eQTL test set (GFMBench-API)Caduceus-Ps on GFMBench-API LRB Variant Effect Causal eQTL (supervised)
0.2838 unitlessMCC (test set)Caduceus-Ps (full fine-tuning, GFMBench-API)LRB Variant Effect Causal eQTL test set (GFMBench-API)Caduceus-Ps on GFMBench-API LRB Variant Effect Causal eQTL (supervised)
0.0022 unitlessAUPRC (zero-shot; cosine similarity of embeddings at the SNV position)Caduceus-Ps (zero-shot, GFMBench-API)LRB Variant Effect OMIM test set (GFMBench-API)Caduceus-Ps on GFMBench-API LRB Variant Effect OMIM (zs-embedding)
0.0019 unitlessAUPRC (zero-shot; cosine similarity of reference and alternate sequence embeddings)Caduceus-Ps (zero-shot, GFMBench-API)LRB Variant Effect OMIM test set (GFMBench-API)Caduceus-Ps on GFMBench-API LRB Variant Effect OMIM (zs-embedding)

Claims

Statements this source supports, each with where in the source it was checked.

Models, benchmarks and other records

Models (1)

Configurations (10)

Benchmarks (1)

Protocols (37)

Datasets (20)

Evaluations (139)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

23 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statementOriginal source and locationReview and provenance
attributes.artifact_sha256
df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.artifact_sha256

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Artifact url
https://www.biorxiv.org/content/10.64898/2026.02.19.706811v2.full.pdf
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.artifact_url

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Doi
10.64898/2026.02.19.706811
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.doi

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Licence
CC-BY-4.0
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.licence

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
2 values
  • Preprint, not peer reviewed.
  • Authors are at NVIDIA, which distributes the BioNeMo implementation of Evo2 used here; no model in the tables was developed by the authors.
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Media type
application/pdf
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.media_type

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Publication status
preprint
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.publication_status

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Retrieved at
2026-10-11T00:44:58Z
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.retrieved_at

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Review: date
2026-10-11
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.review.date

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Review: method note
Artifact re-retrieved or unpacked and SHA-256 matched.
Source metadata
GFMBench-API: A Standardized Interface for Benchmarking Genomic Foundation Models

Original source ↗

No field-specific location recorded

Version: bioRxiv preprint version 2, posted 2026-08-11
Retrieved: 2026-10-11T00:44:58Z

catalogued

No individual claim review recorded

Audit details

Field: attributes.review.method_note

Source artifact SHA-256: df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Primary source retrieved and hashed for the 2026-10-11 genomics literature pass.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)
Technical metadata and extraction receipts

Stable ID: lit-genomics-20261011-gfmbench-source-larey2026

areas
dna-genomes
contexts
research
url
https://doi.org/10.64898/2026.02.19.706811
artifact url
https://www.biorxiv.org/content/10.64898/2026.02.19.706811v2.full.pdf
version
bioRxiv preprint version 2, posted 2026-08-11
retrieved at
2026-10-11T00:44:58Z
artifact sha256
df9934c88ee245b1e19e155174169d6b6da57ae07c1a04bbea7115423facdf72
doi
10.64898/2026.02.19.706811
publication status
preprint
licence
CC-BY-4.0
media type
application/pdf
limitations
Preprint, not peer reviewed.; Authors are at NVIDIA, which distributes the BioNeMo implementation of Evo2 used here; no model in the tables was developed by the authors.
review
method: source-hash-verification; method note: Artifact re-retrieved or unpacked and SHA-256 matched.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-3, independent of the extractor lit-genomics; no human review claimed; date: 2026-10-11
Related records

    Suggest a correction