rewirebio.iobenchmarks
Task

metagenomic genus classification

Genus classification compares sequence representations on separate simulated amplicon and shotgun datasets.

SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

1 evaluation · 1 result

Overview

Evaluation procedure diagram
How it worksComputational evaluation flow
Computational evaluation flow1. Input: Sequence-derived PC-mer or alternative feature encodings.. Then: 2. Evaluation: Ten-fold cross-validation is reported for the benchmark experiments.. Then: 3. Readout: Classification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes.Computational evaluation flow1. Input: Sequence-derived PC-mer or alternative feature encodings.. Then: 2. Evaluation: Ten-fold cross-validation is reported for the benchmark experiments.. Then: 3. Readout: Classification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes.Computational evaluation flow1. Input: Sequence-derived PC-mer or alternative feature encodings.. Then: 2. Evaluation: Ten-fold cross-validation is reported for the benchmark experiments.. Then: 3. Readout: Classification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes.

Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.

SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

limited source coverage · Automated source review, 2026-09-16. All specifications and missing details

1 recorded evaluation, 1 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

1 evaluation · 1 result. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Pipeline: PC-mer + LRTask: metagenomic genus classification
Dataset: AMP
97% Accuracy
percent · unknown

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

PC-mer + LR: metagenomic genus classification

k=8 PC-mer feature extraction with logistic regression on AMP genus-classification dataset

Aggregation: Not reported

PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Table 3, AMP section, PC-mer + LR k=8 row, Accuracy (%) column

Source checking is not independent reproduction. Release 2026-11-21-adc15377fab2.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

Amplicon and shotgun collections derived from RDP version 11 update 5 (30 September 2016), with balanced taxonomic subsets and a separate unbalanced Qiita collection. The taxonomic label level and sequence-generation method must remain attached to each evaluation. Ten-fold cross-validation is reported for the benchmark experiments. Classification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes. Conventional classifiers using PC-mer are compared with CNN/DBN approaches using alternative encodings. The study evaluates supervised classifiers with ten-fold cross-validation across the amplicon and shotgun collections. Sections 2.2 and 3.1–3.5 do not specify genome-, species- or homology-grouped fold assignment; ten-fold evaluation alone does not establish unseen-species generalization.

SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Results 3.1 Datasets; Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58; Methods 2.2 Learning unit; Results 3.1 Datasets and 3.5.1 Training and testing procedure

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths and considerations

No source-reviewed explanatory claims are recorded here yet.

Profile review details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Stable record: reported-task-4420dcdfe8338d

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
PropertyDescription and evidence
DatasetsAmplicon and shotgun collections derived from RDP version 11 update 5 (30 September 2016), with balanced taxonomic subsets and a separate unbalanced Qiita collection. The taxonomic label level and sequence-generation method must remain attached to each evaluation.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Results 3.1 Datasets
SplitsTen-fold cross-validation is reported for the benchmark experiments.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
MetricsClassification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
BaselinesConventional classifiers using PC-mer are compared with CNN/DBN approaches using alternative encodings.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
Leakage controlsThe study evaluates supervised classifiers with ten-fold cross-validation across the amplicon and shotgun collections. Sections 2.2 and 3.1–3.5 do not specify genome-, species- or homology-grouped fold assignment; ten-fold evaluation alone does not establish unseen-species generalization. · Not reported in inspected sources
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods 2.2 Learning unit; Results 3.1 Datasets and 3.5.1 Training and testing procedure
UncertaintyThe cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. · Not reported in inspected sources
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
Entity typePaper-specific computational evaluation protocol.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
OrganismsSimulated microbial amplicon and shotgun communities.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
AssaysGenus-level taxonomic labels.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
Allowed inputsSequence-derived PC-mer or alternative feature encodings.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58
AdaptationSupervised classifiers evaluated through ten-fold cross-validation.
SourcesPC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification · Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resourceVersionReference
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classificationversion of recordRead source
DOI: 10.1371/journal.pone.0307279
Historical gaps recorded on 2026-09-17

The catalogue now holds 1 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

  • Complete raw table acquired. Dataset and k-mer row spans must be propagated; CNN/DBN/RDP/PC-mer-LR/WalkIm rows retained. Structured extraction pending; do not infer equivalent preprocessing.
Search and extraction details

source found structured extraction pending

Searches

  • PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification primary paper benchmark results

Evidence locations

  • Table3 AMP/SG datasets and k-mer configurations

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

17 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-adc15377fab2
Property and statementOriginal source and locationReview and provenance
Diagram caption
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Input: Sequence-derived PC-mer or alternative feature encodings.
  • Evaluation: Ten-fold cross-validation is reported for the benchmark experiments.
  • Readout: Classification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram title
Computational evaluation flow
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.diagram.title

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Datasets
Amplicon and shotgun collections derived from RDP version 11 update 5 (30 September 2016), with balanced taxonomic subsets and a separate unbalanced Qiita collection. The taxonomic label level and sequence-generation method must remain attached to each evaluation.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Results 3.1 Datasets

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Splits
Ten-fold cross-validation is reported for the benchmark experiments.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Adaptation
Supervised classifiers evaluated through ten-fold cross-validation.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.10.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Metrics
Classification accuracy at multiple ranks; the genus task must be distinguished from easier higher-rank outcomes.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.2.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Baselines
Conventional classifiers using PC-mer are compared with CNN/DBN approaches using alternative encodings.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

source checked

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.3.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Leakage controls
The study evaluates supervised classifiers with ten-fold cross-validation across the amplicon and shotgun collections. Sections 2.2 and 3.1–3.5 do not specify genome-, species- or homology-grouped fold assignment; ten-fold evaluation alone does not establish unseen-species generalization.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods 2.2 Learning unit; Results 3.1 Datasets and 3.5.1 Training and testing procedure

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

unreported

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.4.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Uncertainty
The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim.
Individual claims
PC-mer: An Ultra-fast memory-efficient tool for metagenomics profiling and classification

Original source ↗

Methods §3.1; Results/Conclusion; cached text lines 23–24, 48, 58

Version: version of record
Retrieved: 2026-09-16T10:41:06Z

unreported

automated source review · 2026-09-16

Audit details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Field: attributes.profile.facts.5.value

Source artifact SHA-256: 0b0a225fa6f5f3ba41dffc7b320c91f47738301cf45835260eecaa03b704a097

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-adc15377fab2 · Record review: needs review

2 source records and release historyDownload this release (gzip)
Technical metadata and record identifiers

Technical metadata for this record is tracked under stable ID reported-task-4420dcdfe8338d.

Related records

Suggest a correction