Datasets
MetaShot simulated reads with origin labels mapped through NCBI taxonomy into a CAMI reference profile.
Task
Simulated metagenome virus-taxon retrieval
Simulated metagenome evaluation measures recovery of known viral taxa at multiple taxonomic levels.
Sources
Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37Overview
Metrics
OPAL-based precision, recall and F1 for viral taxa and all predicted taxa, with genus/species levels distinguished.
Allowed inputs
Metagenomic reads and reference resources.
Sources
Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37Evaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37limited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
2 evaluations · 2 results. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Pipeline: Lazypipe-nt | Task: Simulated metagenome virus-taxon retrieval Dataset: Simulated viral metagenome | 0.932 Genus-level F1 unitless · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceLazypipe-nt: Simulated metagenome virus-taxon retrieval Genus-rank viral taxon retrieval. Aggregation: Not reported Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Table 1, Lazypipe-nt / Genus row, F column |
| Configuration: Kraken2 | Task: Simulated metagenome virus-taxon retrieval Dataset: Simulated viral metagenome | 0.627 Genus-level F1 unitless · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceKraken2: Simulated metagenome virus-taxon retrieval Genus-rank viral taxon retrieval. Aggregation: Not reported Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Table 1, Kraken2 / Genus row, F column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
MetaShot simulated reads with origin labels mapped through NCBI taxonomy into a CAMI reference profile. An index/query evaluation against a fixed known mixture; the paper separately evaluates mock-community data. OPAL-based precision, recall and F1 for viral taxa and all predicted taxa, with genus/species levels distinguished. Lazypipe variants, Centrifuge, MetaPhlAn2 and Kraken2 are discussed in the comparison. The MetaShot simulation is scored against accession-derived viral and bacterial truth using reference-based classifiers. Methods 2.3 does not report removing the simulated source genomes from the classifiers’ reference databases; this is not established as an unseen-genome test.
Sources
Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37; Methods 2.3 Benchmarking performanceRecorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
No source-reviewed explanatory claims are recorded here yet.
Limitations and conditions
- Taxonomic rank changes the assessed outcome. A simulation drawn from known reference organisms does not establish performance on viruses absent from the reference databases.
Sources
Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods 2.3 Benchmarking performance
Profile review details
Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.
Stable record: reported-task-369dcfef14c4a9Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | MetaShot simulated reads with origin labels mapped through NCBI taxonomy into a CAMI reference profile.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Splits | An index/query evaluation against a fixed known mixture; the paper separately evaluates mock-community data.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Metrics | OPAL-based precision, recall and F1 for viral taxa and all predicted taxa, with genus/species levels distinguished.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Baselines | Lazypipe variants, Centrifuge, MetaPhlAn2 and Kraken2 are discussed in the comparison.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Leakage controls | The MetaShot simulation is scored against accession-derived viral and bacterial truth using reference-based classifiers. Methods 2.3 does not report removing the simulated source genomes from the classifiers’ reference databases; this is not established as an unseen-genome test. · Not reported in inspected sourcesSourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods 2.3 Benchmarking performance |
| Uncertainty | The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. · Not reported in inspected sourcesSourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Entity type | Paper-specific computational evaluation protocol.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Organisms | Simulated viral taxa and background community members.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Assays | MetaShot reads with known origin taxonomy.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Allowed inputs | Metagenomic reads and reference resources.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
| Adaptation | Index/query analysis of a known mixture; mock-community data are a separate test.SourcesNovel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types | version of record | Read source |
Historical gaps recorded on 2026-09-17
The catalogue now holds 2 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- complete numerical transcription and independent cell review: Full primary artifact and table inventory preserved; no new numeric row is published from this audit alone.
- exact checkpoint hashes and per-method scored denominators: Table labels alone do not establish these fields; do not infer checkpoint or scored count from model name or dataset size.
Search and extraction details
primary comparison tables located
Searches
- Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types 10.1093/ve/veaa091
Evidence locations
- Table 1.; XML table veaa091-T1
- Table 2.; XML table veaa091-T2
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
17 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets MetaShot simulated reads with origin labels mapped through NCBI taxonomy into a CAMI reference profile. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits An index/query evaluation against a fixed known mixture; the paper separately evaluates mock-community data. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Index/query analysis of a known mixture; mock-community data are a separate test. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics OPAL-based precision, recall and F1 for viral taxa and all predicted taxa, with genus/species levels distinguished. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Lazypipe variants, Centrifuge, MetaPhlAn2 and Kraken2 are discussed in the comparison. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | source checked automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls The MetaShot simulation is scored against accession-derived viral and bacterial truth using reference-based classifiers. Methods 2.3 does not report removing the simulated source genomes from the classifiers’ reference databases; this is not established as an unseen-genome test. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods 2.3 Benchmarking performance Version: version of record | unreported automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty The cited text-accessible evaluation sections give no confidence-interval, resampling or repeat-run error-bar specification. Image-only tables and uninspected supplements are outside this absence claim. Individual claims | Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types Methods §2.3; Results §3.1; cached text lines 27–29, 35–37 Version: version of record | unreported automated source review · 2026-09-16 Audit detailsTargeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
2 source records and release history
- Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Original source · version of record
- Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types · Original source · version of record
Technical metadata and extraction receipts
Stable ID: reported-task-369dcfef14c4a9
- areas
- microbes-communities
- tasks
- Simulated metagenome virus-taxon retrieval
- entity level
- task
- version
- Not reported
- task
- Simulated metagenome virus-taxon retrieval
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: primary_comparison_tables_located; primary sources: evidence-expansion-lazypipe-2020-77842d8e; inspected locators: Table 1.; XML table veaa091-T1; Table 2.; XML table veaa091-T2; searched queries: Novel NGS pipeline for virus discovery from a wide spectrum of hosts and sample types 10.1093/ve/veaa091; gaps: complete numerical transcription and independent cell review: Full primary artifact and table inventory preserved; no new numeric row is published from this audit alone.; exact checkpoint hashes and per-method scored denominators: Table labels alone do not establish these fields; do not infer checkpoint or scored count from model name or dataset size.; claim scope: Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: lazypipe-2020; source locator: Methods §2.3; Results §3.1; cached text lines 27–29, 35–37; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.