Datasets
Queries derived from a reference index of human-gut bacterial genomes; Table 5 reports four strain-level query sets.
Task
Strain-level abundance quantification
Strain-level taxonomic abundance estimation is compared against known compositions in simulated human-gut read collections.
Sources
Strain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57Overview
Metrics
Correctly identified strains and L1/L2 distances between predicted and true relative abundance vectors.
Allowed inputs
Metagenomic reads and a reference genome index.
Sources
Strain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57Evaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
Strain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
2 evaluations · 2 results. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: CAMMiQ | Task: Strain-level abundance quantification Dataset: HumanGut-all strain-level query | 0.0517 L1 abundance error unitless · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceCAMMiQ: Strain-level abundance quantification Strain-level quantification on the HumanGut-all synthetic query. Aggregation: Not reported Strain level microbial detection and quantification with applications to single cell metagenomics · Table 5, B. L1 Err. / HumanGut-all row, CAMMiQ L1 Err. column |
| Configuration: Kraken2 | Task: Strain-level abundance quantification Dataset: HumanGut-all strain-level query | 0.284 L1 abundance error unitless · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceKraken2: Strain-level abundance quantification Strain-level quantification on the HumanGut-all synthetic query. Aggregation: Not reported Strain level microbial detection and quantification with applications to single cell metagenomics · Table 5, B. L1 Err. / HumanGut-all row, Kraken2 L1 Err. column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
Queries derived from a reference index of human-gut bacterial genomes; Table 5 reports four strain-level query sets. An index-and-query benchmark rather than a supervised train/test split; the relevant evaluation is the strain-level collection. Correctly identified strains and L1/L2 distances between predicted and true relative abundance vectors. Kraken2, KrakenUniq, CLARK, Centrifuge and MetaPhlAn2. The four strain-level queries are simulated from the same 614-genome reference collection used to build the custom indices. A genome without unique or doubly unique 100-mers is excluded from query generation. This is identification and abundance estimation against an indexed collection, not a strain-held-out generalization test.
Sources (2)
Strain level microbial detection and quantification with applications to single cell metagenomics; cammiq-2022__41467_2022_33869_MOESM1_ESM.pdf · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57; Supplementary Notes 5.2 Strain-Level Index Dataset and 5.5 Strain-Level QueriesRecorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
- Known query composition provides an explicit reference for strain-abundance error.
Sources
Strain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57
Limitations and conditions
- Reference-index inclusion and simulated query composition define the scope; no unseen-strain generalization guarantee is inferred.
Sources
Strain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57
Profile review details
Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.
Stable record: reported-task-571f0a2e7faed3Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | Queries derived from a reference index of human-gut bacterial genomes; Table 5 reports four strain-level query sets.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Splits | An index-and-query benchmark rather than a supervised train/test split; the relevant evaluation is the strain-level collection.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Metrics | Correctly identified strains and L1/L2 distances between predicted and true relative abundance vectors.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Baselines | Kraken2, KrakenUniq, CLARK, Centrifuge and MetaPhlAn2.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Leakage controls | The four strain-level queries are simulated from the same 614-genome reference collection used to build the custom indices. A genome without unique or doubly unique 100-mers is excluded from query generation. This is identification and abundance estimation against an indexed collection, not a strain-held-out generalization test.Sources (2)Strain level microbial detection and quantification with applications to single cell metagenomics; cammiq-2022__41467_2022_33869_MOESM1_ESM.pdf · Supplementary Notes 5.2 Strain-Level Index Dataset and 5.5 Strain-Level Queries |
| Uncertainty | Table 5 reports one identification count and L1/L2 abundance error for each of four different query constructions. Supplementary Note 5.5 does not define these four constructions as replicate draws for a confidence interval, and no uncertainty interval accompanies the table values. · Not reported in inspected sourcesSources (2)Strain level microbial detection and quantification with applications to single cell metagenomics; cammiq-2022__41467_2022_33869_MOESM1_ESM.pdf · Table 5; Supplementary Note 5.5 Strain-Level Queries |
| Entity type | Paper-specific computational evaluation protocol.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Organisms | Human-gut bacterial strains.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Assays | Synthetic sequence queries with known strain abundances.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Allowed inputs | Metagenomic reads and a reference genome index.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
| Adaptation | Reference-index construction and query classification; no assay-label supervised fitting is prescribed.SourcesStrain level microbial detection and quantification with applications to single cell metagenomics · Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Strain level microbial detection and quantification with applications to single cell metagenomics | version of record | Read source DOI: 10.1038/s41467-022-33869-7 |
Historical gaps recorded on 2026-09-17
The catalogue now holds 2 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- Complete raw table acquired. Identified-strain counts/ratios and L1/L2 abundance errors must remain different metrics; MetaPhlAn detection bounds are not exact counts. Structured extraction pending.
Search and extraction details
source found structured extraction pending
Searches
- Strain level microbial detection and quantification with applications to single cell metagenomics primary paper benchmark results
Evidence locations
- Table5, human-gut mixtures and six methods
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
21 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets Queries derived from a reference index of human-gut bacterial genomes; Table 5 reports four strain-level query sets. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits An index-and-query benchmark rather than a supervised train/test split; the relevant evaluation is the strain-level collection. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Reference-index construction and query classification; no assay-label supervised fitting is prescribed. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Correctly identified strains and L1/L2 distances between predicted and true relative abundance vectors. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Kraken2, KrakenUniq, CLARK, Centrifuge and MetaPhlAn2. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls The four strain-level queries are simulated from the same 614-genome reference collection used to build the custom indices. A genome without unique or doubly unique 100-mers is excluded from query generation. This is identification and abundance estimation against an indexed collection, not a strain-held-out generalization test. Individual claims | Strain level microbial detection and quantification with applications to single cell metagenomics Supplementary Notes 5.2 Strain-Level Index Dataset and 5.5 Strain-Level Queries Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls The four strain-level queries are simulated from the same 614-genome reference collection used to build the custom indices. A genome without unique or doubly unique 100-mers is excluded from query generation. This is identification and abundance estimation against an indexed collection, not a strain-held-out generalization test. Individual claims | cammiq-2022__41467_2022_33869_MOESM1_ESM.pdf Supplementary Notes 5.2 Strain-Level Index Dataset and 5.5 Strain-Level Queries Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved 2026-09-16; sha256:910aed130f3b4648b0758bdcc6b82d1e2d38ddb320ea80d670c98c565930610b | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: 41467_2022_33869_MOESM1_ESM.pdf |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
3 source records and release history
- Strain level microbial detection and quantification with applications to single cell metagenomics · Original source · version of record
- Strain level microbial detection and quantification with applications to single cell metagenomics · Original source · version of record
- cammiq-2022__41467_2022_33869_MOESM1_ESM.pdf · Original source · Retrieved 2026-09-16; sha256:910aed130f3b4648b0758bdcc6b82d1e2d38ddb320ea80d670c98c565930610b
Technical metadata and extraction receipts
Stable ID: reported-task-571f0a2e7faed3
- areas
- microbes-communities
- tasks
- Strain-level abundance quantification
- entity level
- task
- version
- Not reported
- task
- Strain-level abundance quantification
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: source_found_structured_extraction_pending; primary sources: evidence-expansion-p2-cammiq-2022-f0939647ed3d; inspected locators: Table5, human-gut mixtures and six methods; searched queries: Strain level microbial detection and quantification with applications to single cell metagenomics primary paper benchmark results; gaps: Complete raw table acquired. Identified-strain counts/ratios and L1/L2 abundance errors must remain different metrics; MetaPhlAn detection bounds are not exact counts. Structured extraction pending.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: cammiq-2022; source locator: Results: Datasets; Performance of CAMMiQ at the strain level; Table 5; cached text lines 19–22, 55–57; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.
Related records
- dataset: HumanGut-all strain-level query
- benchmark: CAMMiQ: Strain-level abundance quantification
- benchmark: Kraken2: Strain-level abundance quantification