Proportion (phylum rank; share of test samples the tool returned a prediction for) of sourmash (Closely related dataset, Mock et al. 2022) on BERTax Closely related dataset (Mock et al. 2022)
6.9% Proportion (phylum rank; share of test samples the tool returned a prediction for)
Methods
- Tested configuration
- sourmash (Closely related dataset, Mock et al. 2022)
- Protocol
- BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)
- Dataset
- BERTax Closely related dataset (Mock et al. 2022)
- Procedure
- BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)
- Evaluation
- sourmash on the BERTax closely related dataset
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Not reported by the source
- Evidence
- Author-reported evaluation · source checkedBERTax SI Appendix (pnas.2122636119.sapp.pdf) · SI Table S2, row "sourmash", column group "closely related dataset", "phylum", column "prop." (SI page S15)
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Test set of 2,000 randomly selected samples per phylum; 95% of the remaining samples train and 5% validate; test and training samples may share a genus.
- Adaptation
- Reference or training data restricted to the dataset's training partition
- Scoring implementation
- Top prediction only; unclassified samples treated as random guesses for AveP
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 6.90 Individual claims | BERTax SI Appendix (pnas.2122636119.sapp.pdf) SI Table S2, row "sourmash", column group "closely related dataset", "phylum", column "prop." (SI page S15) Version: SI Appendix as distributed with PMC9436379 (member pnas.2122636119.sapp.pdf of the Europe PMC supplementaryFiles ZIP) | source checked ["source-hash-verification","pdf-text-parse","independent-cell-check"] · 2026-10-12 author reported Audit detailsRe-downloaded the SI PDF from the PMC open-access copy and matched its pinned SHA-256; compared every Table S2 and S5 cell with a separate pdftotext parser written for this review (row and column labels asserted). Reviewer corrections: accuracy qualifier states the classified-samples denominator; proportion classified has direction unknown, because classifying more samples is not better by itself. Field: Source artifact SHA-256: Hash scope: SHA-256 of the PDF member, not of the ZIP (the ZIP is rebuilt per request) Extraction artifact SHA-256: |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
3 source records and release history
- BERTax SI Appendix (pnas.2122636119.sapp.pdf) · Original source · SI Appendix as distributed with PMC9436379 (member pnas.2122636119.sapp.pdf of the Europe PMC supplementaryFiles ZIP)
- Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks · Original source · PNAS 119(35):e2122636119, published 2022-08-26; PMC9436379 full-text XML
- Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) · Original source · PNAS 119(35):e2122636119 version-of-record PDF (10 pages)
Technical metadata and extraction receipts
Stable ID: dna-pathogen-20261012-bertax-result-sourmash-closely-phylum-proportion-classified
- metric
- proportion
- metric direction
- unknown
- unit
- percent
- printed value
- 6.90
- numeric value
- 6.90
- metric qualifier
- phylum rank; share of test samples the tool returned a prediction for
- source locator
- SI Table S2, row "sourmash", column group "closely related dataset", "phylum", column "prop." (SI page S15)
- missing metadata
- uncertainty: reason: unreported
- review
- method: source-hash-verification; pdf-text-parse; independent-cell-check; reviewer: claude; reviewer note: Separate Claude review agent (rev-mlvb), independent of the extracting agent; no human review claimed; date: 2026-10-12; artifact sha256: 89a2f86f66cbb2ff17886d413cd89577285086fedd1009e1d9daf87b29fbc1c0; retrieval url: https://pmc-oa-opendata.s3.amazonaws.com/PMC9436379.1/pnas.2122636119.sapp.pdf; method note: Read from `pdftotext -layout` output of the pinned SI PDF; extract/extract_bertax.py asserts the caption, the three header lines, the nine tool labels and eight values per row.; note: Re-downloaded the SI PDF from the PMC open-access copy and matched its pinned SHA-256; compared every Table S2 and S5 cell with a separate pdftotext parser written for this review (row and column labels asserted). Reviewer corrections: accuracy qualifier states the classified-samples denominator; proportion classified has direction unknown, because classifying more samples is not better by itself.
Related records
- evaluation: sourmash on the BERTax closely related dataset