Genomic Benchmarks

Stable ID: discovery-benchmark-genomic-benchmarks

areas

genomics

entity level

suite

scope note

Specialist molecular or omics evaluation; protocol details require review before numerical comparison.

task

Genomic sequence classification

version

Not reported

benchmark research

review date: 2026-09-17; status: primary_comparison_table_screened; primary sources: expansion-p3-genomic-benchmarks; inspected locators: Table 2; Training models section; Dataset construction and Table 1; searched queries: Genomic Benchmarks collection genomic sequence classification PMC10150520; gaps: The two implementations are not distinct foundation-model families.; Dataset totals in Table 1 are not test-set denominators.; Table 2 does not enumerate dataset-version numbers, split file hashes or replicate uncertainty.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.

historical missing metadata

dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted

metadata review scope

historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.

entity classification

review date: 2026-09-17; rationale: The cited profile describes a collection of evaluation tasks or protocols; retain it as the top-level benchmark suite. Its datasets and individual protocols remain separate records.; source ids: src-discovery-ml-bioinfo-ceitec-genomic-benchmarks; source locator: Pinned README: repository purpose; info and download_dataset examples; ambiguities: None recorded

run documentation

record id: discovery-benchmark-genomic-benchmarks; source ids: run-doc-genomic-benchmarks-readme-md-605d8539; status: official_documentation_linked; summary: Official data-loading examples and linked TensorFlow/PyTorch training notebooks are available. Dataset download alone produces train/test sequence files rather than a model score. The optional dependency examples contain unquoted >= specifiers, so do not copy those lines directly into a shell without quoting the version requirement.; source locator: README.md lines 8–34 and 36–108 (Install and Usage)

run recipes

id: genomic-benchmarks-official; protocol id: discovery-benchmark-genomic-benchmarks; version: 605d8539830e16c85abe7826990958303ffc5e1c; title: Load a Genomic Benchmarks dataset; purpose: generate_and_evaluate; summary: Install the package and load one of the nine sequence classification datasets whose baseline scores this page records.; inputs: A sequence classifier you want to train and score.; outputs: Train and test splits as the package ships them.; requirements: data: Downloaded by the package; the splits are fixed by the release.; weights: None. The published baseline is a small convolutional network.; licence: Project licence: Apache-2.0. Upstream data licences are separate and unreported here.; software: Python with the genomic-benchmarks package from PyPI.; hardware: CPU is enough to load the data; training the baseline benefits from a GPU.; instructions: runtime: command_line; title: Install; code: pip install genomic-benchmarks; status: source_reviewed_not_executed; source ids: project-recipe-genomic-benchmarks-605d8539; source locator: README.md at 605d8539, Install, lines 13-13; runtime: python; title: List the datasets; code: >>> from genomic_benchmarks.data_check import list_datasets >>> >>> list_datasets() ['demo_coding_vs_intergenomic_seqs', 'demo_human_or_worm', 'dummy_mouse_enhancers_ensembl', 'human_enhancers_cohn', 'human_enhancers_ensembl', 'human_ensembl_regulatory', 'human_nontata_promoters', 'human_ocr_ensembl']; status: source_reviewed_not_executed; source ids: project-recipe-genomic-benchmarks-605d8539; source locator: README.md at 605d8539, Usage, lines 40-43; limitations: Quoted from the project's README and not executed by rewire, so the commands are evidence of what the project documents rather than a verified run.; The project may have changed since the pinned commit.; The scores here are the paper's own baseline in two frameworks, not a leaderboard.; source ids: project-recipe-genomic-benchmarks-605d8539; source locator: README.md at 605d8539; id: genomic-benchmarks-rewirebench; protocol id: genomic-benchmarks-v2; version: aecb9e79a2a5e83b59e482212b1a8b812dd16079; title: Score your own classifier with rewirebench; purpose: generate_and_evaluate; summary: Prepare a downloaded dataset and score an adapter on accuracy and F1 over the packaged test split.; inputs: The dataset directory the genomic-benchmarks package downloaded.; An adapter returning a class index, or a probability for a binary dataset.; outputs: A local report with accuracy, F1, the coverage and the digest of the sequences it read.; requirements: data: Whatever the package downloaded locally, read from its train and test directories.; weights: Whatever your own model needs; the runner supplies none.; licence: Runner code is MIT. The benchmark's own data terms are upstream and unreported here.; software: Python 3.11 with the pinned rewirebench environment.; hardware: CPU for scoring. Your own model decides what it needs.; instructions: runtime: command_line; title: Prepare a dataset; code: rewirebench prepare genomic-benchmarks-v2 \ --source ./genomic_benchmarks --options '{"dataset": "human_nontata_promoters"}' \ --output ./prepared-promoters; status: source_reviewed_not_executed; source ids: project-recipe-runner-genomic-benchmarks-genomic-benchmarks-md-aecb9e79; project-recipe-runner-genomic-benchmarks-genomic-benchmarks-py-aecb9e79; source locator: docs/genomic-benchmarks.md at aecb9e79, Prepare, run and score, lines 51-53; runtime: command_line; title: Run and score an adapter; code: rewirebench run --prepared ./prepared-promoters \ --adapter my_models.dna:MyAdapter \ --model-name 'my model' --training-overlap 'unreported' \ --output ./scored-promoters; status: source_reviewed_not_executed; source ids: project-recipe-runner-genomic-benchmarks-genomic-benchmarks-md-aecb9e79; project-recipe-runner-genomic-benchmarks-genomic-benchmarks-py-aecb9e79; source locator: docs/genomic-benchmarks.md at aecb9e79, Prepare, run and score, lines 61-64; limitations: Quoted from the runner's documentation and not executed by this repository.; Scoring follows the benchmark's own evaluator; running it does not by itself reproduce a published number.; Class labels are assigned from the sorted class name, because upstream takes them from filesystem order, which differs between machines. Compare class names, not indices.; For the one dataset with three classes the paper does not say which F1 averaging it used, so macro and weighted are both reported and neither is the paper's number.; Genomic Benchmarks v1 prepared artifacts exposed class information in row IDs and must be discarded. V2 uses opaque IDs and label-independent ordering; prepare the dataset again.; This scores a hashed local data copy; it does not certify the official cohort or reproduce a published experiment.; source ids: project-recipe-runner-genomic-benchmarks-genomic-benchmarks-md-aecb9e79; project-recipe-runner-genomic-benchmarks-genomic-benchmarks-py-aecb9e79; source locator: docs/genomic-benchmarks.md at aecb9e79

• applicable to: k-mer ridge regression
• part of: Genomic Benchmarks DEMO-CODING-VS-INTERGENOMIC-SEQS-ACCURACY: demo_coding_vs_intergenomic_seqs, Accuracy
• part of: Genomic Benchmarks DEMO-CODING-VS-INTERGENOMIC-SEQS-F1: demo_coding_vs_intergenomic_seqs, F1 score
• part of: Genomic Benchmarks DEMO-HUMAN-OR-WORM-ACCURACY: demo_human_or_worm, Accuracy
• part of: Genomic Benchmarks DEMO-HUMAN-OR-WORM-F1: demo_human_or_worm, F1 score
• part of: Genomic Benchmarks DROSOPHILA-ENHANCERS-STARK-ACCURACY: drosophila_enhancers_stark, Accuracy
• part of: Genomic Benchmarks DROSOPHILA-ENHANCERS-STARK-F1: drosophila_enhancers_stark, F1 score
• part of: Genomic Benchmarks DUMMY-MOUSE-ENHANCERS-ENSEMBL-ACCURACY: dummy_mouse_enhancers_ensembl, Accuracy
• part of: Genomic Benchmarks DUMMY-MOUSE-ENHANCERS-ENSEMBL-F1: dummy_mouse_enhancers_ensembl, F1 score
• part of: Genomic Benchmarks HUMAN-ENHANCERS-COHN-ACCURACY: human_enhancers_cohn, Accuracy
• part of: Genomic Benchmarks HUMAN-ENHANCERS-COHN-F1: human_enhancers_cohn, F1 score
• part of: Genomic Benchmarks HUMAN-ENHANCERS-ENSEMBL-ACCURACY: human_enhancers_ensembl, Accuracy
• part of: Genomic Benchmarks HUMAN-ENHANCERS-ENSEMBL-F1: human_enhancers_ensembl, F1 score
• part of: Genomic Benchmarks HUMAN-ENSEMBL-REGULATORY-ACCURACY: human_ensembl_regulatory, Accuracy
• part of: Genomic Benchmarks HUMAN-ENSEMBL-REGULATORY-F1: human_ensembl_regulatory, F1 score
• part of: Genomic Benchmarks HUMAN-NONTATA-PROMOTERS-ACCURACY: human_nontata_promoters, Accuracy
• part of: Genomic Benchmarks HUMAN-NONTATA-PROMOTERS-F1: human_nontata_promoters, F1 score
• part of: Genomic Benchmarks HUMAN-OCR-ENSEMBL-ACCURACY: human_ocr_ensembl, Accuracy
• part of: Genomic Benchmarks HUMAN-OCR-ENSEMBL-F1: human_ocr_ensembl, F1 score