Datasets
CAMI II community datasets and a separately sequenced mock community with known reference genomes.
Task
Mock-community MAG taxonomy classification
Metagenome-assembled-genome taxonomy is evaluated on simulated and experimentally sequenced mock communities.
Sources
kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42Overview
Metrics
Sensitivity, precision, false-discovery rate, balanced accuracy and F1.
Allowed inputs
Metagenome-assembled genomes and taxonomic reference resources.
Sources
kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42Evaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
2 evaluations · 2 results. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: kMetaShot | Task: Mock-community MAG taxonomy classification Dataset: Real mock community MAGs | 95.8 Genus-level F1 % · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourcekMetaShot: Mock-community MAG taxonomy classification Genus classification of MAGs from MegaHIT contigs; uncorrected kMetaShot. Aggregation: Not reported kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Table 2, F1-score % row, Genus kMS column |
| Configuration: GTDB-Tk | Task: Mock-community MAG taxonomy classification Dataset: Real mock community MAGs | 89.8 Genus-level F1 % · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceGTDB-Tk: Mock-community MAG taxonomy classification Genus classification of the same MAG set. Aggregation: Not reported kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Table 2, F1-score % row, Genus Gtk column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
CAMI II community datasets and a separately sequenced mock community with known reference genomes. Community-specific evaluations compare inferred MAG classifications against reference labels; these are not supervised train/test partitions. Sensitivity, precision, false-discovery rate, balanced accuracy and F1. GTDBtk and CAMITAX; two assembly methods are also assessed for the real mock community. The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim.
Sources
kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42Recorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
No source-reviewed explanatory claims are recorded here yet.
Limitations and conditions
- Strain-level labels are refined using reference-genome similarity and assessed before/after refinement. Refined labels and original labels are distinct ground-truth conditions.
Sources
kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42
Profile review details
Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.
Stable record: reported-task-8406b6aabfb8c0Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | CAMI II community datasets and a separately sequenced mock community with known reference genomes.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Splits | Community-specific evaluations compare inferred MAG classifications against reference labels; these are not supervised train/test partitions.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Metrics | Sensitivity, precision, false-discovery rate, balanced accuracy and F1.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Baselines | GTDBtk and CAMITAX; two assembly methods are also assessed for the real mock community.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Leakage controls | HMP genomes are used to tune the assignment threshold, then separate CAMI and mock-community datasets are evaluated. HMP includes both reference-represented and unrepresented taxa, so the tuning comparison is not wholly out of reference; the paper reports these groups separately.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: HMP testing and mock communities; Results: HMP benchmarking; cached paragraphs 31–42, 54–59 |
| Uncertainty | The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. · Not reported in inspected sourcesSourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Entity type | Paper-specific computational evaluation protocol.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Organisms | Microbial communities with known reference genomes.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Assays | Simulated CAMI data and a measured mock community.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Allowed inputs | Metagenome-assembled genomes and taxonomic reference resources.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
| Adaptation | Genome classification against references; community evaluation is separate from supervised training splits.SourceskMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Methods: CAMI datasets; real dataset; cached text lines 36–42 |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
| Paper or primary resource | Version | Reference |
|---|---|---|
| kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes | PMC archival version PMC11695915.1 | Read source DOI: 10.1093/bib/bbae680 |
Historical gaps recorded on 2026-09-17
The catalogue now holds 2 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- Do not pool assemblies, ranks or raw/corrected taxonomic labels.
- Table entries such as '9 + 33' and '8 + 35' are printed decompositions, not unambiguous single numerical observations.
- Total MAGs and metric confusion-matrix denominators differ; retain both.
Search and extraction details
primary comparison table screened
Searches
- "PMC11695915"
Evidence locations
- Tables 2–3 and captions
- Results: real mock sequencing data
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
17 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets CAMI II community datasets and a separately sequenced mock community with known reference genomes. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Community-specific evaluations compare inferred MAG classifications against reference labels; these are not supervised train/test partitions. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Genome classification against references; community evaluation is separate from supervised training splits. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Sensitivity, precision, false-discovery rate, balanced accuracy and F1. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines GTDBtk and CAMITAX; two assembly methods are also assessed for the real mock community. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls HMP genomes are used to tune the assignment threshold, then separate CAMI and mock-community datasets are evaluated. HMP includes both reference-represented and unrepresented taxa, so the tuning comparison is not wholly out of reference; the paper reports these groups separately. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: HMP testing and mock communities; Results: HMP benchmarking; cached paragraphs 31–42, 54–59 Version: PMC archival version PMC11695915.1 | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. Individual claims | kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes Methods: CAMI datasets; real dataset; cached text lines 36–42 Version: PMC archival version PMC11695915.1 | unreported automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
2 source records and release history
- kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Original source · PMC archival version PMC11695915.1
- kMetaShot: a fast and reliable taxonomy classifier for metagenome-assembled genomes · Original source · PMC archival version PMC11695915.1
Technical metadata and extraction receipts
Stable ID: reported-task-8406b6aabfb8c0
- areas
- microbes-communities
- tasks
- Mock-community MAG taxonomy classification
- entity level
- task
- version
- Not reported
- task
- Mock-community MAG taxonomy classification
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: primary_comparison_table_screened; primary sources: expansion-p3-kmetashot-2025; inspected locators: Tables 2–3 and captions; Results: real mock sequencing data; searched queries: "PMC11695915"; gaps: Do not pool assemblies, ranks or raw/corrected taxonomic labels.; Table entries such as '9 + 33' and '8 + 35' are printed decompositions, not unambiguous single numerical observations.; Total MAGs and metric confusion-matrix denominators differ; retain both.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: kmetashot-2025; source locator: Methods: CAMI datasets; real dataset; cached text lines 36–42; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.