Contested result

Kraken2 and sourmash: Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) ranks Kraken2 above sourmash in 7 of 9 comparisons; other studies rank sourmash above Kraken2 in 24 of 29.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

Evaluation of taxonomic classification… ranks sourmash above Kraken2 in 24 of its 48 comparisons and Kraken2 higher in 5; 19 are ties. Taxonomic classification of DNA… ranks Kraken2 above sourmash in 7 of its 11 comparisons and sourmash higher in 2; 2 are ties.

Rank sourmash higher (1): Evaluation of taxonomic classification…
Rank Kraken2 higher (1): Taxonomic classification of DNA… (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

sourmash higherTieKraken2 higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks sourmash higher	Ranks Kraken2 higher	Ties
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF) and 1 more record · DOI	Disagrees	2	7	2
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets · DOI	Agrees	24	5	19

Comparisons in Taxonomic classification of DNA… (11)

Protocol	Dataset	Metric	sourmash	Kraken2	Higher
BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Closely related dataset (Mock et al. 2022)	Accuracy (phylum rank; among the samples the tool classified; SI Table S2 'acc.'), percent	99.8	98.8	Tie
BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Closely related dataset (Mock et al. 2022)	Accuracy (superkingdom rank; among the samples the tool classified; SI Table S2 'acc.'), percent	100	99.8	Tie
BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Closely related dataset (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses), percent	9.04	76	Kraken2
BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Closely related dataset (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses), percent	30.7	86.3	Kraken2
BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Distantly related dataset (Mock et al. 2022)	Accuracy (phylum rank; among the samples the tool classified; SI Table S2 'acc.'), percent	91.7	68.3	sourmash
BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Distantly related dataset (Mock et al. 2022)	Accuracy (superkingdom rank; among the samples the tool classified; SI Table S2 'acc.'), percent	99.6	90.7	sourmash
BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Distantly related dataset (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses), percent	3.48	19.5	Kraken2
BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	BERTax Distantly related dataset (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses), percent	25.1	44.4	Kraken2
BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	BERTax Final model dataset (Mock et al. 2022)	Macro average precision (genus rank; macro average over classes; unclassified samples treated as random guesses), percent	3.07	70.6	Kraken2
BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	BERTax Final model dataset (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses), percent	8	87.1	Kraken2
BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	BERTax Final model dataset (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses), percent	31	93.7	Kraken2

Comparisons in Evaluation of taxonomic classification… (48)

Protocol	Dataset	Metric	sourmash	Kraken2	Higher
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other'), percent	67	50.7	Kraken2
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	F-beta (F0.5 (beta = 0.5)), fraction	0.89	0.21	sourmash
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	F1, fraction	0.93	0.29	sourmash
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	False negatives, count	0	0	Tie
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	False positives, count	3	96	sourmash
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	Precision, fraction	0.87	0.17	sourmash
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	Recall, fraction	1	1	Tie
HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi)	True positives, count	20	20	Tie
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other'), percent	52	51.9	Tie
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	F-beta (F0.5 (beta = 0.5)), fraction	0.93	0.08	sourmash
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	F1, fraction	0.93	0.13	sourmash
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	False negatives, count	1	1	Tie
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	False positives, count	1	196	sourmash
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	Precision, fraction	0.93	0.07	sourmash
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	Recall, fraction	0.93	0.93	Tie
HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi)	True positives, count	14	14	Tie
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other'), percent	55.4	44.8	Kraken2
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	F-beta (F0.5 (beta = 0.5)), fraction	0.83	0.24	sourmash
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	F1, fraction	0.89	0.34	sourmash
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	False negatives, count	0	0	Tie
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	False positives, count	5	77	sourmash
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	Precision, fraction	0.8	0.21	sourmash
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	Recall, fraction	1	1	Tie
Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ATCC MSA-1003 mock community (HiSeq 2500)	True positives, count	20	20	Tie
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other'), percent	82.3	59.7	Kraken2
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	F-beta (F0.5 (beta = 0.5)), fraction	0.86	0.17	sourmash
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	F1, fraction	0.91	0.24	sourmash
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	False negatives, count	0	0	Tie
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	False positives, count	2	62	sourmash
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	Precision, fraction	0.83	0.14	sourmash
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	Recall, fraction	1	1	Tie
Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled	True positives, count	10	10	Tie
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other'), percent	41.5	16.7	Kraken2
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	F-beta (F0.5 (beta = 0.5)), fraction	0.76	0.07	sourmash
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	F1, fraction	0.83	0.11	sourmash
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	False negatives, count	0	0	Tie
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	False positives, count	4	166	sourmash
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	Precision, fraction	0.71	0.06	sourmash
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	Recall, fraction	1	1	Tie
ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION)	True positives, count	10	10	Tie
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other'), percent	28.3	22.3	Kraken2
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	F-beta (F0.5 (beta = 0.5)), fraction	0.81	0.07	sourmash
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	F1, fraction	0.87	0.11	sourmash
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	False negatives, count	0	0	Tie
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	False positives, count	3	156	sourmash
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	Precision, fraction	0.76	0.06	sourmash
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	Recall, fraction	1	1	Tie
ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION)	True positives, count	10	10	Tie

What disagrees

Other studies: rank sourmash above Kraken2 in 24 of 29 comparisons
This study: ranks Kraken2 above sourmash in 7 of 9 comparisons

Compared: Kraken2, sourmash
Study that disagrees: Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF), BERTax SI Appendix (pnas.2122636119.sapp.pdf)
Other studies: Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (18)
Study	Tool	Protocol or benchmark	Metric	Value
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Closely related dataset, Mock et al. 2022)	BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)	76
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Closely related dataset, Mock et al. 2022)	BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses)	86.3
BERTax SI Appendix (pnas.2122636119.sapp.pdf); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF)	Kraken2 (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Accuracy (phylum rank; among the samples the tool classified; SI Table S2 'acc.')	68.3
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)	19.5
BERTax SI Appendix (pnas.2122636119.sapp.pdf); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF)	Kraken2 (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Accuracy (superkingdom rank; among the samples the tool classified; SI Table S2 'acc.')	90.7
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses)	44.4
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Final model dataset, Mock et al. 2022)	BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	Macro average precision (genus rank; macro average over classes; unclassified samples treated as random guesses)	70.6
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Final model dataset, Mock et al. 2022)	BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)	87.1
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	Kraken2 (Final model dataset, Mock et al. 2022)	BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses)	93.7
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Closely related dataset, Mock et al. 2022)	BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)	9.04
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Closely related dataset, Mock et al. 2022)	BERTax Closely related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses)	30.7
BERTax SI Appendix (pnas.2122636119.sapp.pdf); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF)	sourmash (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Accuracy (phylum rank; among the samples the tool classified; SI Table S2 'acc.')	91.7
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)	3.48
BERTax SI Appendix (pnas.2122636119.sapp.pdf); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF)	sourmash (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Accuracy (superkingdom rank; among the samples the tool classified; SI Table S2 'acc.')	99.6
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Distantly related dataset, Mock et al. 2022)	BERTax Distantly related dataset: taxonomic classification at superkingdom and phylum (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses)	25.1
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Final model dataset, Mock et al. 2022)	BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	Macro average precision (genus rank; macro average over classes; unclassified samples treated as random guesses)	3.07
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Final model dataset, Mock et al. 2022)	BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	Macro average precision (phylum rank; macro average over classes; unclassified samples treated as random guesses)	8
Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks (PNAS PDF); Taxonomic classification of DNA sequences beyond sequence similarity using deep neural networks; BERTax SI Appendix (pnas.2122636119.sapp.pdf)	sourmash (Final model dataset, Mock et al. 2022)	BERTax Final model dataset: taxonomic classification at superkingdom and phylum, and genus (Mock et al. 2022)	Macro average precision (superkingdom rank; macro average over classes; unclassified samples treated as random guesses)	31

In the other studies (63 results)

In the other studies
Study	Tool	Protocol or benchmark	Metric	Value
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.21
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.29
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	96
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	50.7
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.17
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.89
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.93
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	3
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	67
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	HiFi ATCC MSA-1003 mock community (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.87
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.08
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.13
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	196
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.07
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.93
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.93
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	1
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	HiFi ZymoBIOMICS D6331 gut microbiome standard (PacBio Sequel II HiFi): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.93
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.24
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.34
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	77
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	44.8
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.21
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.83
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.89
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	5
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	55.4
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ATCC MSA-1003 mock community (HiSeq 2500): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.8
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.17
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.24
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	62
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	59.7
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.14
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.86
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.91
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	2
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.83
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	2
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	Illumina ZymoBIOMICS D6300 standard (NovaSeq 6000), subsampled: species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	82.3
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.07
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.11
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	166
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	16.7
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.06
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.76
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.83
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	4
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k31 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.71
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.76
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.83
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	4
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	41.5
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT Q20 ZymoBIOMICS D6300 standard, length-filtered (PromethION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.71
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.07
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.11
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	156
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	22.3
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Kraken2 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.06
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F-beta (F0.5 (beta = 0.5))	0.81
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	F1	0.87
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	False positives	3
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Composition L1 distance (species-level relative abundance, false positives pooled as 'Other')	28.3
Evaluation of taxonomic classification and profiling methods for long-read shotgun metagenomic sequencing datasets	Sourmash-k51 (Portik et al. 2022)	ONT R10.3 ZymoBIOMICS D6300 standard, length-filtered (GridION): species detection at 0.001% of total reads (Portik et al. 2022 Table 4)	Precision	0.76

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-20839b2c55d8defe · All contested results · How findings are made and followed up