Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop)
ICLR 2026 workshop (the workshop is not named in the PDF) · 2026 · OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) · Peer-reviewed
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- reported_finding: Saadat and Fellay 2026: Reported finding With LoRA fine-tuning on the Nucleotide Transformer benchmark promoter (300 bp) and enhancer (400 bp) tasks, the authors report that the efficient implementation substantially reduces training time and peak GPU memory, with speedups generally increasing with model size, and achieves broadly comparable accuracy to the baseline implementation. (Abstract; Section 3.4 'Efficiency' and 'Task performance' paragraphs (page 6); Figure 4 (page 7))
- reported_finding: Saadat and Fellay 2026: Reported finding Under 8-bit and 4-bit quantization, embedding cosine similarity to BF16 stays above 0.98 for GENA-LM 110M, GENA-LM 336M and NT 500M; for NT 2.5B quantization lowers it to roughly 0.9. (Section 3.5 'Fidelity' paragraph (page 6): 'All models except NT 2.5B maintain embedding similarity above 0.98. NT 2.5B is more sensitive: quantization lowers similarity to roughly 0.9.')
- tables_not_extracted: Saadat and Fellay 2026: Tables not extracted Not extracted as results: Figure 2 (inference peak memory and throughput against sequence length 256-4096 and batch size 1-32), the Figure 3 points not quoted in Section 3.3 (baseline and FlashAttention stages for all models; every stage for GENA-LM 336M and NT 500M), Figure 4 (fine-tuning accuracy, peak memory and training time for promoter and enhancer prediction) and Figure 5 apart from the NT 2.5B BF16 and 4-bit memory. These figures plot values without printing them, and the repository does not read numbers from plotted marks. (Figures 2-5 (pages 5-7))
Models, benchmarks and other records
Configurations (10)
- GENA-LM 110M, efficient inference implementation (Saadat and Fellay 2026)
- GENA-LM 110M, training with FlashAttention and checkpointing (Saadat and Fellay 2026)
- GENA-LM 110M, training with FlashAttention, checkpointing and ZeRO-2 offload (Saadat and Fellay 2026)
- GENA-LM 336M, efficient inference implementation (Saadat and Fellay 2026)
- NT 2.5B, 4-bit weights for embedding inference (Saadat and Fellay 2026)
- NT 2.5B, BF16 weights for embedding inference (Saadat and Fellay 2026)
- NT 2.5B, efficient inference implementation (Saadat and Fellay 2026)
- NT 2.5B, training with FlashAttention and checkpointing (Saadat and Fellay 2026)
- NT 2.5B, training with FlashAttention, checkpointing and ZeRO-2 offload (Saadat and Fellay 2026)
- NT 500M, efficient inference implementation (Saadat and Fellay 2026)
Protocols (3)
- Peak memory under 8-bit and 4-bit quantization relative to BF16 (Saadat and Fellay 2026 Figure 5)
- Pseudo-perplexity agreement between reference and efficient implementations (Saadat and Fellay 2026 Figure 1)
- Training peak GPU memory under progressive optimizations, sequence length 1000, batch size 16 (Saadat and Fellay 2026 Figure 3)
Datasets (1)
Evaluations (10)
- GENA-LM 110M efficient implementation, pseudo-perplexity agreement with the reference
- GENA-LM 110M training peak memory, stage + Checkpointing
- GENA-LM 110M training peak memory, stage + DeepSpeed
- GENA-LM 336M efficient implementation, pseudo-perplexity agreement with the reference
- NT 2.5B 4-bit inference memory
- NT 2.5B BF16 inference memory
- NT 2.5B efficient implementation, pseudo-perplexity agreement with the reference
- NT 2.5B training peak memory, stage + Checkpointing
- NT 2.5B training peak memory, stage + DeepSpeed
- NT 500M efficient implementation, pseudo-perplexity agreement with the reference
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
30 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| attributes.artifact_sha256 4bc2452110f280a7ba8baa4c7c28beac89769511019666bc77fc33af9b3bcd60 Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://openreview.net/pdf?id=NUleWh1GFx Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Media type application/pdf Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Missing metadata: doi: note OpenReview workshop paper without a DOI Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Missing metadata: doi: reason inapplicable Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Missing metadata: licence: note No licence is printed in the PDF and the OpenReview forum page could not be read by script. Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Missing metadata: licence: reason unavailable Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Publication status peer_reviewed Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieval note OpenReview answers scripted requests with a bot challenge, so the PDF was downloaded by hand in a browser by the repository owner on 2026-10-15 (file 49_Optimizing_Genomic_Language.pdf; the browser recorded the download URL as https://openreview.net/pdf?id=NUleWh1GFx). retrieved_at is the file's creation time. The OpenReview API also returned a challenge, so the forum metadata (workshop name, licence, code link) was not read. Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-10-15T10:56:26Z Source metadata | Optimizing genomic language models for efficient training, fine-tuning, and inference (Saadat and Fellay, ICLR 2026 workshop) No field-specific location recorded Version: OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09) | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
Workshop paper PDF retrieved from OpenReview and hashed for the issue #162 extraction.
Technical metadata and extraction receipts
Stable ID: saadat-fellay-20261015-source-paper
- areas
- dna-genomes
- contexts
- research
- artifact url
- https://openreview.net/pdf?id=NUleWh1GFx
- url
- https://openreview.net/forum?id=NUleWh1GFx
- retrieved at
- 2026-10-15T10:56:26Z
- artifact sha256
- 4bc2452110f280a7ba8baa4c7c28beac89769511019666bc77fc33af9b3bcd60
- publication status
- peer_reviewed
- version
- OpenReview PDF headed 'Published as a workshop paper at ICLR 2026' (11 pages; PDF CreationDate 2026-03-09)
- venue
- ICLR 2026 workshop (the workshop is not named in the PDF)
- year
- 2026
- media type
- application/pdf
- retrieval note
- OpenReview answers scripted requests with a bot challenge, so the PDF was downloaded by hand in a browser by the repository owner on 2026-10-15 (file 49_Optimizing_Genomic_Language.pdf; the browser recorded the download URL as https://openreview.net/pdf?id=NUleWh1GFx). retrieved_at is the file's creation time. The OpenReview API also returned a challenge, so the forum metadata (workshop name, licence, code link) was not read.
- missing metadata
- doi: reason: inapplicable; note: OpenReview workshop paper without a DOI; licence: reason: unavailable; note: No licence is printed in the PDF and the OpenReview forum page could not be read by script.
- review
- reviewer: claude; reviewer note: Claude review agent; no human review claimed. Independent Claude (Opus 5.5) review agent, separate from the extraction agent that built this batch. Automated review only. Transcription, not reproduction.; reviewed at: 2026-10-15T22:00:00Z; artifact sha256: 4bc2452110f280a7ba8baa4c7c28beac89769511019666bc77fc33af9b3bcd60; retrieval url: https://openreview.net/pdf?id=NUleWh1GFx; method: source-hash-verification; ai-assisted-source-review; method note: Checked the archived copy of the 2026-10-15 hand download against the pinned SHA-256; no fresh download (OpenReview answers scripted requests with a bot challenge). URL, version, venue and missing-metadata reasons checked against the PDF.; note: Hash matches. No source-level evidence concern; the quantization memory conflict is recorded on the two affected results.