Datasets
DNase-sensitivity QTLs and chromatin-accessibility QTLs used in DART-Eval.
Task
regulatory-variant scoring
Regulatory-variant scoring is assessed against DART-Eval quantitative-trait-locus effect annotations.
Sources
Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89Overview
Metrics
Predicted allele scores are compared with observed effects; correlation analysis is restricted to variants with significant observed effects.
Allowed inputs
Reference/alternate DNA sequences at the evaluated variants.
Sources
Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89Evaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
1 evaluation · 1 result. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Pipeline: ARSENAL+ChromBPNet | Task: regulatory-variant scoring Dataset: Yoruban LCL dsQTLs | 0.896 AUROC fraction · unknown Uncertainty: ±0.016 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceARSENAL+ChromBPNet: regulatory-variant scoring Supervised ChromBPNet variant scoring with ARSENAL motif-discovery regularization Aggregation: Not reported Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Table 1, Yoruban LCL dsQTLs section, ARSENAL+ChromBPNet row, AUROC column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
DNase-sensitivity QTLs and chromatin-accessibility QTLs used in DART-Eval. The paper describes chromosome-separated pretraining partitions; evaluation is zero-shot variant scoring. Predicted allele scores are compared with observed effects; correlation analysis is restricted to variants with significant observed effects.
Sources
Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89Recorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
No source-reviewed explanatory claims are recorded here yet.
Limitations and conditions
- Correlation is calculated only on significant-effect variants, changing the evaluated population. Zero-shot label use does not establish that their genomic contexts were absent from pretraining.
Sources
Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89
Profile review details
Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.
Stable record: reported-task-b9199a30a0bcb2Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | DNase-sensitivity QTLs and chromatin-accessibility QTLs used in DART-Eval.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Splits | The paper describes chromosome-separated pretraining partitions; evaluation is zero-shot variant scoring.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Metrics | Predicted allele scores are compared with observed effects; correlation analysis is restricted to variants with significant observed effects.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Baselines | Figure 4 compares with DNA-language-model scores taken from DART-EVAL and includes an ARSENAL No Prior ablation. These are imported zero-shot likelihood-scoring comparisons, separate from the paper’s supervised ChromBPNet experiments.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Results: zero-shot regulatory-QTL scoring, Fig.4 caption; Methods: Zero-shot Variant Effect Prediction |
| Leakage controls | The variant experiment uses no variant-annotation supervision and evaluates significant-effect QTLs. The zero-shot methods do not document removal of these QTL loci from sequence pretraining; chromosome splits described for pretraining and supervised models are not an explicit QTL-overlap audit. · Not reported in inspected sourcesSourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Zero-shot Variant Effect Prediction; pretraining partitions; Supervised Model Training; Fig.4 |
| Uncertainty | The zero-shot results and Figure 4 caption do not specify a QTL resampling unit, replicate count or confidence-interval procedure. Variability reported in the supervised ChromBPNet comparison does not establish uncertainty for these zero-shot correlations. · Not reported in inspected sourcesSourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Zero-shot regulatory-QTL results and Fig.4 caption; Methods: Zero-shot Variant Effect Prediction |
| Entity type | Paper-specific computational evaluation protocol.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Organisms | Human.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Assays | DNase-sensitivity and chromatin-accessibility QTL annotations.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Allowed inputs | Reference/alternate DNA sequences at the evaluated variants.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
| Adaptation | Zero-shot variant scoring; pretraining chromosome partitions are separate from the QTL evaluation.SourcesShort-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization | preprint version in PMC | Read source DOI: 10.64898/2026.02.05.703637 |
Historical gaps recorded on 2026-09-17
The catalogue now holds 1 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- These are trained hybrid models, not zero-shot ARSENAL scores.
- Keep QTL cohorts and Pearson/Spearman/AUROC separate; verify printed ± definition before plotting error bars.
Search and extraction details
primary comparison table screened
Searches
- "PMC12889687"
Evidence locations
- Table 1
- Methods: Pretraining Data; Supervised Model Training
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
17 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets DNase-sensitivity QTLs and chromatin-accessibility QTLs used in DART-Eval. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits The paper describes chromosome-separated pretraining partitions; evaluation is zero-shot variant scoring. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Zero-shot variant scoring; pretraining chromosome partitions are separate from the QTL evaluation. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Predicted allele scores are compared with observed effects; correlation analysis is restricted to variants with significant observed effects. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Figure 4 compares with DNA-language-model scores taken from DART-EVAL and includes an ARSENAL No Prior ablation. These are imported zero-shot likelihood-scoring comparisons, separate from the paper’s supervised ChromBPNet experiments. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Results: zero-shot regulatory-QTL scoring, Fig.4 caption; Methods: Zero-shot Variant Effect Prediction Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls The variant experiment uses no variant-annotation supervision and evaluates significant-effect QTLs. The zero-shot methods do not document removal of these QTL loci from sequence pretraining; chromosome splits described for pretraining and supervised models are not an explicit QTL-overlap audit. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Zero-shot Variant Effect Prediction; pretraining partitions; Supervised Model Training; Fig.4 Version: preprint version in PMC | unreported automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty The zero-shot results and Figure 4 caption do not specify a QTL resampling unit, replicate count or confidence-interval procedure. Variability reported in the supervised ChromBPNet comparison does not establish uncertainty for these zero-shot correlations. Individual claims | Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization Zero-shot regulatory-QTL results and Fig.4 caption; Methods: Zero-shot Variant Effect Prediction Version: preprint version in PMC | unreported automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
2 source records and release history
- Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Original source · preprint version in PMC
- Short-Context Regulatory DNA Language Models with Motif-Discovery Regularization · Original source · preprint version in PMC
Technical metadata and extraction receipts
Stable ID: reported-task-b9199a30a0bcb2
- areas
- dna-genomes
- tasks
- regulatory-variant scoring
- entity level
- task
- version
- Not reported
- task
- regulatory-variant scoring
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: primary_comparison_table_screened; primary sources: expansion-p3-arsenal-regulatory-dna-2026; inspected locators: Table 1; Methods: Pretraining Data; Supervised Model Training; searched queries: "PMC12889687"; gaps: These are trained hybrid models, not zero-shot ARSENAL scores.; Keep QTL cohorts and Pearson/Spearman/AUROC separate; verify printed ± definition before plotting error bars.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: arsenal-regulatory-dna-2026; source locator: Methods: Zero-Shot Variant Effect Prediction; Pretraining Data; cached text lines 50–52, 85–89; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.
Related records
- dataset: Yoruban LCL dsQTLs
- benchmark: ARSENAL+ChromBPNet: regulatory-variant scoring