Task

Hierarchical metagenomic taxonomy classification

Hierarchical taxonomic classification evaluates sequence labels under separate in-distribution and out-of-distribution settings.

Sources

2 evaluations · 2 results

Overview

Datasets

BERTax-derived ID, OOD and Complete datasets spanning major taxonomic groups.

Metrics

Accuracy and macro average precision at superkingdom and phylum levels; micro/macro AUC appears in supplements.

Allowed inputs

DNA sequence fragments.

Sources

Evaluation procedure diagram

How it worksComputational evaluation flow

Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.

Sources

limited source coverage · Automated source review, 2026-09-16. All specifications and missing details

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

2 evaluations · 2 results. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: ICCTax	Task: Hierarchical metagenomic taxonomy classification Dataset: ICCTax Complete dataset	67.2 Genus macro AveP % · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Author-reported evaluation · Source checked Methods, coverage and source ICCTax: Hierarchical metagenomic taxonomy classification Macro average precision at genus rank on Complete dataset. Aggregation: Not reported ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Table 2, ICCTax row, Genus column
Configuration: Kraken2	Task: Hierarchical metagenomic taxonomy classification Dataset: ICCTax Complete dataset	70.6 Genus macro AveP % · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source Kraken2: Hierarchical metagenomic taxonomy classification Macro average precision at genus rank on Complete dataset. Aggregation: Not reported ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Table 2, Kraken2 row, Genus column

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

BERTax-derived ID, OOD and Complete datasets spanning major taxonomic groups. Accuracy and macro average precision at superkingdom and phylum levels; micro/macro AUC appears in supplements. MMseqs2, Minimap2, Kraken2, sourmash, MetaPhlAn4, CAT, DeepMicrobes and BERTax.

Sources

ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages; Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages; Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths and considerations

No source-reviewed explanatory claims are recorded here yet.

Limitations and conditions

ID and OOD scores use different taxonomic partition strategies. Reference-database coverage also varies between external comparison tools.
Sources
ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages

Profile review details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Stable record: reported-task-f4b1c9373f0929

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Datasets	BERTax-derived ID, OOD and Complete datasets spanning major taxonomic groups. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Splits	ID reserves 2,000 sequences per phylum for testing and uses the rest for training. OOD assigns every sequence in a genus to a single side of the train/test partition. The Complete-dataset classifier is a separate setting. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · §3.1 Datasets; Fig.2 caption (a–c)
Metrics	Accuracy and macro average precision at superkingdom and phylum levels; micro/macro AUC appears in supplements. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Baselines	MMseqs2, Minimap2, Kraken2, sourmash, MetaPhlAn4, CAT, DeepMicrobes and BERTax. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Leakage controls	Figure 2 defines ID testing by sampling sequences within phyla, while OOD places all sequences from a genus wholly in training or wholly in testing. The same genome collection is partitioned differently, so ID scores do not establish performance on unseen genera. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · §3.1 Datasets; full-XML Fig.2 caption (a–c)
Uncertainty	The paper gives bootstrap confidence intervals for accuracy and average precision on both ID and OOD datasets. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Entity type	Paper-specific computational evaluation protocol. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Organisms	Major taxonomic groups in BERTax-derived ID/OOD datasets. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Assays	Reference sequence/taxonomy annotations. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Allowed inputs	DNA sequence fragments. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages
Adaptation	Supervised hierarchical classification tested in in-distribution and out-of-distribution regimes. Sources ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.

Paper or primary resource	Version	Reference
ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model	version of record	Read source DOI: 10.1093/bioadv/vbaf257

Historical gaps recorded on 2026-09-17

The catalogue now holds 2 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

MetaPhlAn4 uses its official prebuilt database, unlike the common training database; preserve the asterisk.
CAT and MetaPhlAn4 do not supply probabilities, so AveP is inapplicable, not zero.
Bootstrap intervals exist in supplemental figures; main-table values lack printed intervals.

Search and extraction details

primary comparison table screened

Searches

"PMC12619997"

Evidence locations

Tables 1–2 and footnotes
Section 3.2.1
Supplement Figures S3–S4 (interval locations)

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

17 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram steps Input: DNA sequence fragments. Evaluation: Supervised hierarchical classification tested in in-distribution and out-of-distribution regimes. Readout: Accuracy and macro average precision at superkingdom and phylum levels; micro/macro AUC appears in supplements. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram title Computational evaluation flow Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Datasets BERTax-derived ID, OOD and Complete datasets spanning major taxonomic groups. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Splits ID reserves 2,000 sequences per phylum for testing and uses the rest for training. OOD assigns every sequence in a genus to a single side of the train/test partition. The Complete-dataset classifier is a separate setting. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ §3.1 Datasets; Fig.2 caption (a–c) Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Adaptation Supervised hierarchical classification tested in in-distribution and out-of-distribution regimes. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.10.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Metrics Accuracy and macro average precision at superkingdom and phylum levels; micro/macro AUC appears in supplements. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.2.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Baselines MMseqs2, Minimap2, Kraken2, sourmash, MetaPhlAn4, CAT, DeepMicrobes and BERTax. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.3.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Leakage controls Figure 2 defines ID testing by sampling sequences within phyla, while OOD places all sequences from a genus wholly in training or wholly in testing. The same genome collection is partitioned differently, so ID scores do not establish performance on unseen genera. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ §3.1 Datasets; full-XML Fig.2 caption (a–c) Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.4.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Uncertainty The paper gives bootstrap confidence intervals for accuracy and average precision on both ID and OOD datasets. Individual claims	ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model Original source ↗ Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages Version: version of record Retrieved: 2026-09-16T10:33:55.373Z	source checked automated source review · 2026-09-16 Audit details Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: `attributes.profile.facts.5.value` Source artifact SHA-256: `2ce0b48f1cde3aea7e561d92f4d7dc1525af7439ccd16f80bec0773e8812c8ec` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: needs review

2 source records and release history

ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Original source · version of record
ICCTax: a hierarchical taxonomic classifier for metagenomic sequences on a large language model · Original source · version of record

Download this release

Technical metadata and extraction receipts

Stable ID: reported-task-f4b1c9373f0929

areas: microbes-communities
tasks: Hierarchical metagenomic taxonomy classification
entity level: task
version: Not reported
task: Hierarchical metagenomic taxonomy classification
scope note: Paper-specific evaluation task; protocol completeness requires further extraction.
benchmark research: review date: 2026-09-17; status: primary_comparison_table_screened; primary sources: expansion-p3-icctax-2025; inspected locators: Tables 1–2 and footnotes; Section 3.2.1; Supplement Figures S3–S4 (interval locations); searched queries: "PMC12619997"; gaps: MetaPhlAn4 uses its official prebuilt database, unlike the common training database; preserve the asterisk.; CAT and MetaPhlAn4 do not supply probabilities, so AveP is inapplicable, not zero.; Bootstrap intervals exist in supplemental figures; main-table values lack printed intervals.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
historical missing metadata: protocol version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
legacy kinds: benchmark
entity classification: review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: icctax-2025; source locator: Results §§3.1–3.2.1; cached text lines 47–48, 51, 53–54; uncertainty/repeat-run/statistical-comparison passages; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.

Related records

dataset: ICCTax Complete dataset
benchmark: ICCTax: Hierarchical metagenomic taxonomy classification
benchmark: Kraken2: Hierarchical metagenomic taxonomy classification

Suggest a correction