Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses) of Kraken2 (Distantly related dataset, Mock et al. 2022) on BERTax Distantly related dataset (Mock et al. 2022)
19.5% Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)
Methods
- Tested configuration
- Kraken2 (Distantly related dataset, Mock et al. 2022)
- Protocol
- BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)
- Dataset
- BERTax Distantly related dataset (Mock et al. 2022)
- Procedure
- BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)
- Evaluation
- Kraken2 on the BERTax distantly related dataset
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Not reported by the source
- Evidence
- Author-reported evaluation · source checkedTaxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) · Table 1, row "Kraken2", column group "Distantly related", column "Phyl." (PNAS PDF page 4)
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Per phylum, one or more whole genera (the subset closest to 2,000 samples) form the test set, undersampled to 1,780 samples per phylum; no test sample shares a genus with the training set.
- Adaptation
- Reference or training data restricted to the dataset's training partition
- Scoring implementation
- Top prediction only; unclassified samples treated as random guesses for AveP
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 19.50 Individual claims | Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) Table 1, row "Kraken2", column group "Distantly related", column "Phyl." (PNAS PDF page 4) Version: PNAS 119(35):e2122636119 version-of-record PDF (10 pages) | source checked ["pdf-text-parse","independent-cell-check"] · 2026-10-12 author reported Audit detailsThe pinned PNAS PDF (SHA-256 7977424613ab5e5f4986745b527a270fd2c0427dbab40471813903eda85dcaec) could not be re-fetched: www.pnas.org returns HTTP 403 to command-line clients and the reviewer did not use a browser. The cell was checked against the PMC open-access copy of the same version-of-record PDF (different bytes, SHA-256 above), whose page-4 Table 1 is a text layer, with a separate pdftotext parser; all 63 Table 1 cells match it and the extractor's archived text-layer excerpt. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
3 source records and release history
- Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) · Original source · PNAS 119(35):e2122636119 version-of-record PDF (10 pages)
- Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks · Original source · PNAS 119(35):e2122636119, published 2022-08-26; PMC9436379 full-text XML
- BERTax SI Appendix (pnas.2122636119.sapp.pdf) · Original source · SI Appendix as distributed with PMC9436379 (member pnas.2122636119.sapp.pdf of the Europe PMC supplementaryFiles ZIP)
Technical metadata and extraction receipts
Stable ID: dna-pathogen-20261012-bertax-result-kraken2-distantly-phylum-avep
- metric
- macro-average-precision
- metric direction
- higher
- unit
- percent
- printed value
- 19.50
- numeric value
- 19.50
- metric qualifier
- phylum rank; macro average over classes; unclassified samples treated as random guesses
- source locator
- Table 1, row "Kraken2", column group "Distantly related", column "Phyl." (PNAS PDF page 4)
- missing metadata
- uncertainty: reason: unreported
- review
- method: pdf-text-parse; independent-cell-check; reviewer: claude; reviewer note: Separate Claude review agent (rev-mlvb), independent of the extracting agent; no human review claimed; date: 2026-10-12; artifact sha256: 725c99ba8bb95d9e5e73997cbbe45d3a889185163749f494673a084de534c835; retrieval url: https://pmc-oa-opendata.s3.amazonaws.com/PMC9436379.1/PMC9436379.1.pdf; method note: Read from the PDF text layer (pdf.js 4.0.379) of the pinned PNAS PDF; extract/extract_bertax.py asserts the title, column headers, row labels and seven two-decimal values per row.; note: The pinned PNAS PDF (SHA-256 7977424613ab5e5f4986745b527a270fd2c0427dbab40471813903eda85dcaec) could not be re-fetched: www.pnas.org returns HTTP 403 to command-line clients and the reviewer did not use a browser. The cell was checked against the PMC open-access copy of the same version-of-record PDF (different bytes, SHA-256 above), whose page-4 Table 1 is a text layer, with a separate pdftotext parser; all 63 Table 1 cells match it and the extractor's archived text-layer excerpt.
Related records
- evaluation: Kraken2 on the BERTax distantly related dataset