Datasets
ClinVar variants meeting a review-status filter, with pathogenic/likely-pathogenic positives and benign/likely-benign negatives; the catalogue retains the 3-prime UTR subset.
Task
ClinVar 3-prime UTR variant classification
ClinVar variant classification assesses whether zero-shot allele scores distinguish pathogenic from benign annotations.
Sources
A Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56Overview
Metrics
ROC-based assessment for ClinVar; AUPRC described for the separate imbalanced OMIM comparison must not be imported automatically.
Allowed inputs
Reference/alternate genomic sequences for the 3-prime UTR subset.
Sources
A Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56Evaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
A Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
1 evaluation · 1 result. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: PhyloGPN | Task: ClinVar 3-prime UTR variant classification Dataset: ClinVar 3-prime UTR variants | 0.94 AUROC fraction · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourcePhyloGPN: ClinVar 3-prime UTR variant classification log-likelihood-ratio scoring Aggregation: Not reported A Phylogenetic Approach to Genomic Language Modeling · Table 1, 3-prime UTR row, PhyloGPN AUROC column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
ClinVar variants meeting a review-status filter, with pathogenic/likely-pathogenic positives and benign/likely-benign negatives; the catalogue retains the 3-prime UTR subset. Zero-shot evaluation, distinct from supervised embedding tasks elsewhere in the paper. ROC-based assessment for ClinVar; AUPRC described for the separate imbalanced OMIM comparison must not be imported automatically. Nucleotide Transformer, HyenaDNA, Caduceus and GPN-MSA comparisons. The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim.
Sources
A Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56Recorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
- The clinical evaluation uses an explicit review-status filter and separates clinical labels from zero-shot scoring.
Sources
A Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56
Limitations and conditions
- The main ClinVar result uses review-status-filtered variants; it is distinct from the held-out-chromosome ablation. Genome pretraining and downstream label use are separate sources of possible overlap.
Sources
A Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56
Profile review details
Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.
Stable record: reported-task-ed3dd3b83c4505Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | ClinVar variants meeting a review-status filter, with pathogenic/likely-pathogenic positives and benign/likely-benign negatives; the catalogue retains the 3-prime UTR subset.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Splits | Zero-shot evaluation, distinct from supervised embedding tasks elsewhere in the paper.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Metrics | ROC-based assessment for ClinVar; AUPRC described for the separate imbalanced OMIM comparison must not be imported automatically.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Baselines | Nucleotide Transformer, HyenaDNA, Caduceus and GPN-MSA comparisons.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Leakage controls | ClinVar variants are selected by review status and pathogenic/benign labels for zero-shot scoring. The main ClinVar experiment does not report exclusion of those loci from genome pretraining. The odd/even chromosome held-out experiment is a separate ablation, not the stated split of the main ClinVar result. · Not reported in inspected sourcesSourcesA Phylogenetic Approach to Genomic Language Modeling · Classifying ClinVar variants and Table 1; held-out-human-sequence ablation; cached paragraphs 36,51–53,72 |
| Uncertainty | The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. · Not reported in inspected sourcesSourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Entity type | Paper-specific computational evaluation protocol.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Organisms | Human ClinVar variants.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Assays | Clinical pathogenic/benign assertions meeting a review-status filter.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Allowed inputs | Reference/alternate genomic sequences for the 3-prime UTR subset.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
| Adaptation | Zero-shot variant scoring; supervised embedding experiments elsewhere are not this protocol.SourcesA Phylogenetic Approach to Genomic Language Modeling · Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
| Paper or primary resource | Version | Reference |
|---|---|---|
| A Phylogenetic Approach to Genomic Language Modeling | preprint version in PMC | Read source |
Historical gaps recorded on 2026-09-17
The catalogue now holds 1 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- Complete raw matrix acquired. Five methods for assignedUTRrow; othervariantclasses remain separate. Alignment-aware versus sequence-only inputs explicit; no architecture equivalence inferred. Structured extraction pending.
Search and extraction details
source found structured extraction pending
Searches
- A Phylogenetic Approach to Genomic Language Modeling primary paper benchmark results
Evidence locations
- Table1 ClinVar3-primeUTR row; likelihood-ratio evaluation
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
18 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets ClinVar variants meeting a review-status filter, with pathogenic/likely-pathogenic positives and benign/likely-benign negatives; the catalogue retains the 3-prime UTR subset. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Zero-shot evaluation, distinct from supervised embedding tasks elsewhere in the paper. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Zero-shot variant scoring; supervised embedding experiments elsewhere are not this protocol. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics ROC-based assessment for ClinVar; AUPRC described for the separate imbalanced OMIM comparison must not be imported automatically. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Nucleotide Transformer, HyenaDNA, Caduceus and GPN-MSA comparisons. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls ClinVar variants are selected by review status and pathogenic/benign labels for zero-shot scoring. The main ClinVar experiment does not report exclusion of those loci from genome pretraining. The odd/even chromosome held-out experiment is a separate ablation, not the stated split of the main ClinVar result. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Classifying ClinVar variants and Table 1; held-out-human-sequence ablation; cached paragraphs 36,51–53,72 Version: preprint version in PMC | unreported automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. Individual claims | A Phylogenetic Approach to Genomic Language Modeling Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56 Version: preprint version in PMC | unreported automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
2 source records and release history
- A Phylogenetic Approach to Genomic Language Modeling · Original source · preprint version in PMC
- A Phylogenetic Approach to Genomic Language Modeling · Original source · preprint version in PMC
Technical metadata and extraction receipts
Stable ID: reported-task-ed3dd3b83c4505
- areas
- dna-genomes
- tasks
- ClinVar 3-prime UTR variant classification
- entity level
- task
- version
- Not reported
- task
- ClinVar 3-prime UTR variant classification
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: source_found_structured_extraction_pending; primary sources: evidence-expansion-p2-phylogpn-2025-807f3a26cbfa; inspected locators: Table1 ClinVar3-primeUTR row; likelihood-ratio evaluation; searched queries: A Phylogenetic Approach to Genomic Language Modeling primary paper benchmark results; gaps: Complete raw matrix acquired. Five methods for assignedUTRrow; othervariantclasses remain separate. Alignment-aware versus sequence-only inputs explicit; no architecture equivalence inferred. Structured extraction pending.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: phylogpn-2025; source locator: Results: Log Likelihood Ratio Evaluation; ClinVar and OMIM tests; cached text lines 46–48, 52, 56; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.
Related records
- dataset: ClinVar 3-prime UTR variants
- benchmark: PhyloGPN: ClinVar 3-prime UTR variant classification