rewirebio.iobenchmarks
Task

Metagenomic taxonomic classification

Metagenomic classification compares reference-based tools and examines dependence on reference-genome quality.

SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

2 evaluations · 2 results

Overview

Datasets

CAMI2 Toy Human Microbiome Project samples are among the benchmark collections.

Metrics

Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.

Allowed inputs

Sequencing reads and model-specific reference databases.

SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
Evaluation procedure diagram
How it worksComputational evaluation flow
Computational evaluation flow1. Input: Sequencing reads and model-specific reference databases.. Then: 2. Evaluation: Taxonomic classification with default comparator settings where applicable.. Then: 3. Readout: Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.Computational evaluation flow1. Input: Sequencing reads and model-specific reference databases.. Then: 2. Evaluation: Taxonomic classification with default comparator settings where applicable.. Then: 3. Readout: Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.Computational evaluation flow1. Input: Sequencing reads and model-specific reference databases.. Then: 2. Evaluation: Taxonomic classification with default comparator settings where applicable.. Then: 3. Readout: Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.

Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.

SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

limited source coverage · Automated source review, 2026-09-16. All specifications and missing details

2 recorded evaluations, 2 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

2 evaluations · 2 results. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: NABAS+Task: Metagenomic taxonomic classification
Dataset: CAMI II Toy human gastrooral sample19-new
0.719 F1
unitless · unknown

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

NABAS+: Metagenomic taxonomic classification

Newly generated sample19 used for classifier comparison.

Aggregation: Not reported

Advancing metagenomic classification with NABAS+: a novel alignment-based approach · Table 3, Sample19-new / NABAS+ row, F1 score column
Configuration: MetaPhlAn3Task: Metagenomic taxonomic classification
Dataset: CAMI II Toy human gastrooral sample19-new
0.753 F1
unitless · unknown

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

MetaPhlAn3: Metagenomic taxonomic classification

Newly generated sample19 used for classifier comparison.

Aggregation: Not reported

Advancing metagenomic classification with NABAS+: a novel alignment-based approach · Table 3, Sample19-new / MetaPhlAn3 row, F1 score column

Source checking is not independent reproduction. Release 2026-11-21-adc15377fab2.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

CAMI2 Toy Human Microbiome Project samples are among the benchmark collections. Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement. MetaPhlAn3, Kraken and GOTTCHA, using defaults where applicable. The source discusses reference assembly age and database coverage as factors affecting apparent accuracy. The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim.

SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths and considerations

No source-reviewed explanatory claims are recorded here yet.

Limitations and conditions

Profile review details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Stable record: reported-task-92137759a9e7b0

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
PropertyDescription and evidence
DatasetsCAMI2 Toy Human Microbiome Project samples are among the benchmark collections.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
SplitsThis is evaluation of reference-database classifiers on CAMI2 samples and Zymo mock communities, rather than a supervised train/validation/test split. Reference-database versions are specified separately; the regenerated CAMI sample19 is a distinct condition. · Not applicable
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: CAMI2 reference databases, Recreating CAMI sample19, Zymo community standards, Running classifiers
MetricsPrecision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
BaselinesMetaPhlAn3, Kraken and GOTTCHA, using defaults where applicable.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
Leakage controlsThe source discusses reference assembly age and database coverage as factors affecting apparent accuracy.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
UncertaintyThe cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. · Not reported in inspected sources
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
Entity typePaper-specific computational evaluation protocol.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
OrganismsCAMI2 human-microbiome community taxa.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
AssaysMetagenomic sequence mixtures with reference taxonomy.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
Allowed inputsSequencing reads and model-specific reference databases.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table
AdaptationTaxonomic classification with default comparator settings where applicable.
SourcesAdvancing metagenomic classification with NABAS+: a novel alignment-based approach · Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resourceVersionReference
Advancing metagenomic classification with NABAS+: a novel alignment-based approachPMC12231603.1Read source
DOI: 10.1093/nargab/lqaf092
Historical gaps recorded on 2026-09-17

The catalogue now holds 2 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

  • Complete raw tables acquired.19 new versusold reference experiments separate; Bray-Curtis lower-is-better unlike F1. Reference database state required before chart groups. Structured extraction pending.
Search and extraction details

source found structured extraction pending

Searches

  • Advancing metagenomic classification with NABAS+: a novel alignment-based approach primary paper benchmark results

Evidence locations

  • Tables3–4; new/old reference database and community datasets

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

17 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-adc15377fab2
Property and statementOriginal source and locationReview and provenance
Diagram caption
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Input: Sequencing reads and model-specific reference databases.
  • Evaluation: Taxonomic classification with default comparator settings where applicable.
  • Readout: Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram title
Computational evaluation flow
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.diagram.title

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Datasets
CAMI2 Toy Human Microbiome Project samples are among the benchmark collections.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Splits
This is evaluation of reference-database classifiers on CAMI2 samples and Zymo mock communities, rather than a supervised train/validation/test split. Reference-database versions are specified separately; the regenerated CAMI sample19 is a distinct condition.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: CAMI2 reference databases, Recreating CAMI sample19, Zymo community standards, Running classifiers

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

inapplicable

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Adaptation
Taxonomic classification with default comparator settings where applicable.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.10.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Metrics
Precision, recall, F1, Jaccard distance and Bray–Curtis distance distinguish detection from abundance agreement.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.2.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Baselines
MetaPhlAn3, Kraken and GOTTCHA, using defaults where applicable.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.3.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Leakage controls
The source discusses reference assembly age and database coverage as factors affecting apparent accuracy.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

source checked

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.4.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Uncertainty
The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim.
Individual claims
Advancing metagenomic classification with NABAS+: a novel alignment-based approach

Original source ↗

Methods: reference classifiers; Discussion: CAMI evaluation; cached text lines 13–15, 108–109; task metric definitions and corresponding results table

Version: PMC12231603.1
Retrieved: 2026-09-16T10:41:16.546107+00:00

unreported

automated source review · 2026-09-16

Audit details

Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.

Field: attributes.profile.facts.5.value

Source artifact SHA-256: 49903e751beb86f6744825d2fdb3ea2fbe327b52b1ce68c323bfa8b66dae71ec

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-adc15377fab2 · Record review: needs review

2 source records and release historyDownload this release (gzip)
Technical metadata and record identifiers

Technical metadata for this record is tracked under stable ID reported-task-92137759a9e7b0.

Related records

Suggest a correction