Source

GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models

arXiv 2406.01627 version 2 · Preprint

What this source contributed

Records in this release that cite this source. A record can cite several sources.

3 models
10 configurations
1 benchmark
52 protocols
52 datasets
350 evaluations
367 results

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Models, benchmarks and other records

Models (3)

Configurations (10)

Benchmarks (1)

GenBench

Protocols (52)

Datasets (52)

Evaluations (350)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

23 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
attributes.artifact_sha256 24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12 Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_sha256` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Artifact url https://arxiv.org/pdf/2406.01627v2 Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_url` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Limitations 7 values Preprint (2024), outside the search window; extracted for the suite completeness check. Tables report 'Top-1' scores; the paper does not define Top-1 (for classification it reads as ordinary top-1 accuracy). The NeurIPS checklist answers that error bars are reported, but no table prints any. Table 3 is called short-range in its caption and long-range in Section 4.2. The Table 8 caption names 3 models while the table has 5 columns of models. Table 4 rows for multi-dataset tasks are means of the Table 10 dataset rows. Many short-range datasets repeat GUE and Genomic Benchmarks tasks under GenBench's own fine-tuning. Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.limitations` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Media type application/pdf Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.media_type` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Missing metadata: licence: note No licence stated on the PDF; arXiv default distribution licence assumed only for reading; not archived Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.missing_metadata.licence.note` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Missing metadata: licence: reason unreported Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.missing_metadata.licence.reason` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Publication status preprint Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.publication_status` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieved at 2026-10-11T01:42:08Z Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.retrieved_at` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review: date 2026-10-11 Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.date` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review: method note Artifact re-retrieved or unpacked and SHA-256 matched. Source metadata	GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models Original source ↗ No field-specific location recorded Version: arXiv 2406.01627 version 2 Retrieved: 2026-10-11T01:42:08Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.method_note` Source artifact SHA-256: `24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Primary source retrieved and hashed for the 2026-10-11 genomics literature pass.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: lit-genomics-20261011-genbench-source-liu2024

areas: dna-genomes
contexts: research
url: https://arxiv.org/abs/2406.01627v2
artifact url: https://arxiv.org/pdf/2406.01627v2
version: arXiv 2406.01627 version 2
retrieved at: 2026-10-11T01:42:08Z
artifact sha256: 24105d26b25a6adf7509334b041d44e145d5006dfedf87e812e12138fa154f12
publication status: preprint
missing metadata: licence: reason: unreported; note: No licence stated on the PDF; arXiv default distribution licence assumed only for reading; not archived
media type: application/pdf
limitations: Preprint (2024), outside the search window; extracted for the suite completeness check.; Tables report 'Top-1' scores; the paper does not define Top-1 (for classification it reads as ordinary top-1 accuracy).; The NeurIPS checklist answers that error bars are reported, but no table prints any.; Table 3 is called short-range in its caption and long-range in Section 4.2.; The Table 8 caption names 3 models while the table has 5 columns of models.; Table 4 rows for multi-dataset tasks are means of the Table 10 dataset rows.; Many short-range datasets repeat GUE and Genomic Benchmarks tasks under GenBench's own fine-tuning.
review: method: source-hash-verification; method note: Artifact re-retrieved or unpacked and SHA-256 matched.; reviewer: claude; reviewer note: Claude review agent reviewer-gen-2, independent of the extractor lit-genomics; no human review claimed; date: 2026-10-11

Related records

Suggest a correction

Value	Metric	Model or configuration	Dataset	Evaluation
0.510 unitless	MSE	Caduceus (GenBench)	GenBench Bulk RNA expression	Caduceus on GenBench Bulk RNA expression
0.746 unitless	Spearman	Caduceus (GenBench)	GenBench Bulk RNA expression	Caduceus on GenBench Bulk RNA expression
0.9372 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Coding vs Intergenomic	Caduceus on GenBench Coding vs Intergenomic
0.443 unitless	Pearson	Caduceus (GenBench)	GenBench Drosophila enhancer activity (dev)	Caduceus on GenBench Drosophila enhancer activity (dev)
0.530 unitless	Pearson	Caduceus (GenBench)	GenBench Drosophila enhancer activity (hk)	Caduceus on GenBench Drosophila enhancer activity (hk)
0.486 unitless	Pearson (mean of dev and hk as printed)	Caduceus (GenBench)	GenBench Drosophila enhancer activity (Mean)	Caduceus on GenBench Drosophila enhancer activity (Mean)
0.0168 unitless	MSE	Caduceus (GenBench)	GenBench Genomic structure prediction (H1-ESC)	Caduceus on GenBench Genomic structure prediction (H1-ESC)
0.5024 unitless	Pearson	Caduceus (GenBench)	GenBench Genomic structure prediction (H1-ESC)	Caduceus on GenBench Genomic structure prediction (H1-ESC)
0.1047 unitless	MSE	Caduceus (GenBench)	GenBench Genomic structure prediction (HFF)	Caduceus on GenBench Genomic structure prediction (HFF)
0.3536 unitless	Pearson	Caduceus (GenBench)	GenBench Genomic structure prediction (HFF)	Caduceus on GenBench Genomic structure prediction (HFF)
0.8505 fraction	Accuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows)	Caduceus (GenBench)	GenBench Human Core Promoter Detection	Caduceus on GenBench Human Core Promoter Detection
0.8341 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Core Promoter Detection, dataset all	Caduceus on GenBench Human Core Promoter Detection, dataset all
0.8427 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Core Promoter Detection, dataset notata	Caduceus on GenBench Human Core Promoter Detection, dataset notata
0.8747 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Core Promoter Detection, dataset tata	Caduceus on GenBench Human Core Promoter Detection, dataset tata
0.7376 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human Enhancers Cohn	Caduceus on GenBench Human Enhancers Cohn
0.8448 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human Enhancers Ensembl	Caduceus on GenBench Human Enhancers Ensembl
0.7367 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human Ensembl Regulatory	Caduceus on GenBench Human Ensembl Regulatory
0.8885 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human Nontata Promoters	Caduceus on GenBench Human Nontata Promoters
0.8176 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human OCR Ensembl	Caduceus on GenBench Human OCR Ensembl
0.7322 fraction	Accuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows)	Caduceus (GenBench)	GenBench Human Promoter Detection	Caduceus on GenBench Human Promoter Detection
0.7833 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Promoter Detection, dataset all	Caduceus on GenBench Human Promoter Detection, dataset all
0.8279 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Promoter Detection, dataset notata	Caduceus on GenBench Human Promoter Detection, dataset notata
0.5854 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Promoter Detection, dataset tata	Caduceus on GenBench Human Promoter Detection, dataset tata
0.5674 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human Splice Site Detection	Caduceus on GenBench Human Splice Site Detection
0.6928 fraction	Accuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows)	Caduceus (GenBench)	GenBench Human Transcription Factor Prediction	Caduceus on GenBench Human Transcription Factor Prediction
0.7140 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Transcription Factor Prediction, dataset 0	Caduceus on GenBench Human Transcription Factor Prediction, dataset 0
0.7590 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Transcription Factor Prediction, dataset 1	Caduceus on GenBench Human Transcription Factor Prediction, dataset 1
0.6650 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Transcription Factor Prediction, dataset 2	Caduceus on GenBench Human Transcription Factor Prediction, dataset 2
0.6370 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Transcription Factor Prediction, dataset 3	Caduceus on GenBench Human Transcription Factor Prediction, dataset 3
0.6890 fraction	Accuracy	Caduceus (GenBench)	GenBench Human Transcription Factor Prediction, dataset 4	Caduceus on GenBench Human Transcription Factor Prediction, dataset 4
0.9557 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Human vs Worm	Caduceus on GenBench Human vs Worm
0.9302 fraction	Accuracy	Caduceus (GenBench)	GenBench Long-range Promoters Prediction	Caduceus on GenBench Long-range Promoters Prediction
0.7110 fraction	Accuracy	Caduceus (GenBench)	GenBench Long-range Species Classification	Caduceus on GenBench Long-range Species Classification
0.691 unitless	AUROC (mean of donor and acceptor as printed)	Caduceus (GenBench)	GenBench Long-range splice site prediction (Mean)	Caduceus on GenBench Long-range splice site prediction (Mean)
0.740 unitless	AUROC	Caduceus (GenBench)	GenBench Long-range splice site prediction (Splice acceptor)	Caduceus on GenBench Long-range splice site prediction (Splice acceptor)
0.642 unitless	AUROC	Caduceus (GenBench)	GenBench Long-range splice site prediction (Splice donar)	Caduceus on GenBench Long-range splice site prediction (Splice donar)
0.8163 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Mouse Enhancers	Caduceus on GenBench Mouse Enhancers
0.6519 fraction	Accuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows)	Caduceus (GenBench)	GenBench Mouse Transcription Factor Prediction	Caduceus on GenBench Mouse Transcription Factor Prediction
0.6012 fraction	Accuracy	Caduceus (GenBench)	GenBench Mouse Transcription Factor Prediction, dataset 0	Caduceus on GenBench Mouse Transcription Factor Prediction, dataset 0
0.7386 fraction	Accuracy	Caduceus (GenBench)	GenBench Mouse Transcription Factor Prediction, dataset 1	Caduceus on GenBench Mouse Transcription Factor Prediction, dataset 1
0.7485 fraction	Accuracy	Caduceus (GenBench)	GenBench Mouse Transcription Factor Prediction, dataset 2	Caduceus on GenBench Mouse Transcription Factor Prediction, dataset 2
0.5792 fraction	Accuracy	Caduceus (GenBench)	GenBench Mouse Transcription Factor Prediction, dataset 3	Caduceus on GenBench Mouse Transcription Factor Prediction, dataset 3
0.5920 fraction	Accuracy	Caduceus (GenBench)	GenBench Mouse Transcription Factor Prediction, dataset 4	Caduceus on GenBench Mouse Transcription Factor Prediction, dataset 4
0.3794 fraction	Accuracy (task-level value as printed in Table 4)	Caduceus (GenBench)	GenBench Virus Covid Variant Classification	Caduceus on GenBench Virus Covid Variant Classification
0.6378 fraction	Accuracy (task-level value as printed in Table 4; mean of the Table 10 dataset rows)	Caduceus (GenBench)	GenBench Yeast Epigenetic Marks Prediction	Caduceus on GenBench Yeast Epigenetic Marks Prediction
0.7147 fraction	Accuracy	Caduceus (GenBench)	GenBench Yeast Epigenetic Marks Prediction, dataset H3	Caduceus on GenBench Yeast Epigenetic Marks Prediction, dataset H3
0.6032 fraction	Accuracy	Caduceus (GenBench)	GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me1	Caduceus on GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me1
0.5938 fraction	Accuracy	Caduceus (GenBench)	GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me2	Caduceus on GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me2
0.5696 fraction	Accuracy	Caduceus (GenBench)	GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me3	Caduceus on GenBench Yeast Epigenetic Marks Prediction, dataset H3K4me3
0.6392 fraction	Accuracy	Caduceus (GenBench)	GenBench Yeast Epigenetic Marks Prediction, dataset H3K9ac	Caduceus on GenBench Yeast Epigenetic Marks Prediction, dataset H3K9ac