Use caseDNA and genomesResearch and clinical research
Select an RNA pathogen-detection workflow for diagnostic testing
Which RNA sequencing workflow detects RNA pathogens reliably in the intended specimen and distinguishes real detections from host/background contamination?
What the evidence shows
The single-assay results are a UCSF respiratory RNA metagenomic assay at 93.6% (103 of 110) sensitivity against original clinical respiratory panel testing, as reported by its developers, and the developers' own prospective validation of an Oxford nanopore workflow on 344 nasopharyngeal swabs (preprint).
With thresholds fixed on a separate derivation set, the nanopore workflow detected 51% (133 of 260) of PCR-positive targets in the samples passing quality control, from 85% for RSV to 19% for rhinovirus/enterovirus, and 99.8% of PCR-negative targets were not detected; criteria chosen after the validation raised sensitivity to 58%, and the authors attributed all nine false positives to barcode crossover. Its swabs were selected mostly as PCR-positive at Ct 30 or below, with about one negative per eight positives, and reads were matched only to the 23 panel pathogens, so it does not test detection outside the panel. Three comparisons of several workflows add proxy evidence.
On 88 nasal washings scored against 1144 respiratory virus PCR results (Carbo et al. 2022), all five classifiers detected 22 or 23 of the 24 PCR-positive results at species level at a read-count cut-off chosen on the same data, but 31% to 58% of their positive calls were PCR-negative. When 13 European clinical virology laboratories each ran their own pipeline on the same 13 clinical datasets (de Vries et al. 2021, preprint), 10 to 13 samples were correctly called positive, and the three pipelines that found every sample reported 6, 2 and 0 additional PCR-negative viruses. In the CAMI II blood metagenome challenge (Meyer et al. 2022), 4 of 10 submissions listed the causal virus.
In a developer benchmark of five virus profilers on one shared database (Tisza et al. 2026, preprint), the model-estimated probability of reporting the correct family for reads from seven divergent viruses at 10 read pairs was 0.974 for EsViritu and 0.9998 for Metabuli but 0.045 for Kraken2, 0.012 for Centrifuger and 0.0009 for Sylph at the authors' headline settings, and Metabuli reported a median of 18 false species in seven pathogen-free libraries against 0 for EsViritu. None of this is clinical validation: the cohorts are small or single-site, reference databases and reporting rules differ, and the benchmark data are mostly synthetic.
You bring
Respiratory specimen RNA (DNase-treated), reverse-transcribed to cDNA libraries
The original clinical reference assay(s) available for sensitivity comparison
You want
A sourced sensitivity-against-original-clinical-testing figure for one respiratory mNGS assay; no general RNA-virus-only diagnostic accuracy claim.
Evidence last checked 11 October 2026 · re-checked monthly, next due 11 November 2026
Executive Takeaway
For "Select an RNA pathogen-detection workflow for diagnostic testing", on Five classifiers against a respiratory PCR panel (Carbo et al. 2022), Species level, read-count cut-off 0 (Recall, a proxy protocol), Kaiju and Kaiju and Kaiju share the best value, 95.83%, reported by Carbo et al. medRxiv preprint v1, Supplementary Tables 1 and 2.
Best reported valueKaijuCarbo et al. medRxiv preprint v1, Supplementary Tables 1…
Recall95.83%Five classifiers against a respiratory PCR panel (Carbo et al. 2022), Species level, read-count cut-off 0
Baseline on this protocolNone scoredNo margin is computed
Ranked tools1515 from independent studies
15 tools have a value on this protocol; 15 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.
Ranking
Ranked on Five classifiers against a respiratory PCR panel (Carbo et al. 2022), Species level, read-count cut-off 0 (Recall, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.
5 tools in 15 rows, from 1 source. 15 of 15 results (100%) come from an independent study.
Rank▲
Model
Recall (higher is better)
Evidence
Coverage
Reported by
License
#1
Kaijuhuman reads removed (Bowtie2 to GRCh38) before classification
Kraken2 and Centrifuge. Kraken2 (k-mer matching) and Centrifuge (FM-index matching) classify reads against reference sequences without a trained model, and are the conventional classifiers scored in the linked virus benchmarks. The clinical workflows that align reads to NCBI nt, such as SURPI+ and CZ ID, have no method record, so they cannot be named.
No linked comparison scores a machine-learning method against the baseline on the same data yet, so this evidence cannot say whether machine learning beats it.
Species level, read-count cut-off 0: Sensitivity, selectivity (specificity), PPV, NPV, AUC and ROC distance against 1144 respiratory PCR results for Centrifuge, CLARK, Kaiju, Kraken2 and Genome Detective, each with human reads kept, removed, or removed with genome-length normalisation; species level, read-count cut-off 0
Species level, read-count cut-off 10: Sensitivity, selectivity (specificity), PPV, NPV, AUC and ROC distance against 1144 respiratory PCR results for Centrifuge, CLARK, Kaiju, Kraken2 and Genome Detective, each with human reads kept, removed, or removed with genome-length normalisation; species level, read-count cut-off 10
Genus level, read-count cut-off 0: Sensitivity, selectivity (specificity), PPV, NPV, AUC and ROC distance against 1144 respiratory PCR results for Centrifuge, CLARK, Kaiju, Kraken2 and Genome Detective, each with human reads kept, removed, or removed with genome-length normalisation; genus level, read-count cut-off 0
Family level, read-count cut-off 0: Sensitivity, selectivity (specificity), PPV, NPV, AUC and ROC distance against 1144 respiratory PCR results for Centrifuge, CLARK, Kaiju, Kraken2 and Genome Detective, each with human reads kept, removed, or removed with genome-length normalisation; family level, read-count cut-off 0
Single cohort of COPD nasal washings; 24 PCR-positive results.
The column labelled 'Informedness' is the ROC distance (Section 2.7).
The article is the published version (Pathogens 2022), but its supplement could not be retrieved; the values come from the medRxiv preprint version 1 supplement with the same table titles, and may differ from the published supplement.
Sensitivity, selectivity, PPV and NPV are at the operating point chosen from the ROC over read-count cut-offs on the same 1144 PCR results (columns marked '(ROC)', Section 2.7), so they describe the best cut-off for this cohort and are optimistic for a new one. AUC does not depend on that choice.
Libraries were prepared with a protocol that processes RNA and DNA together in one tube (Section 2.2), not the DNase-treated RNA input the use case describes.
2 cells print 10000 for a quantity bounded by 0 and 1, probably 1.0000; their numeric values are left null.
6 cells print 10000 for a quantity bounded by 0 and 1, probably 1.0000; their numeric values are left null.
Primary benchmark evaluation shown (Five classifiers against a respiratory PCR panel (Carbo et al. 2022)). 5 additional study protocols (Thirteen laboratory pipelines on 13 clinical datasets (ENNGS, de Vries et al. 2021), Causal RNA virus in a blood metagenome (CAMI II, Meyer et al. 2022), Nanopore RNA metagenomics on nasopharyngeal swabs against multiplex PCR (Sanderson et al. 2026) and others) are cited in individual tool drawer rows and under Question, sources and review.
Not covered yet
A positive-specimen BAL/swab count disagreement between the source's Results and Methods sections (104+6 vs 103+7) is unresolved and preserved.
No confidence interval is extracted for this sensitivity figure in the inspected text.
No independent comparison of CZ ID (IDseq), SURPI+, DRAGEN or VirMAP against other workflows on RNA specimens with printed per-tool values was found in this pass.
Carbo et al. 2022's own supplement could not be retrieved (Europe PMC returned a truncated zip, MDPI HTTP 403, PMC a browser challenge); the values come from the medRxiv preprint supplement of the same tables.
The de Vries et al. 2021 published version could not be retrieved (Europe PMC HTTP 500, publisher and three repository copies HTTP 403 or blocked); the preprint and its supplement were used, and its Tables 1 and 2 are images that were not transcribed.
Across the ENNGS benchmark each laboratory used its own reference database and reporting criteria, so tool and database effects cannot be separated.
The COMPARE in silico proficiency test (Brinkmann et al. 2019) scores two highly divergent RNA viruses (measles at 82% and a bornavirus at 55% nucleotide identity) per participant, but participants are anonymised and not linked to named tools, so it was not extracted.
No RNA virus spike-in or reference-material comparison of workflows was found in this pass.
No machine-learning method has been compared with Kraken2 or Centrifuge on the same data in the linked evidence.
Select an RNA pathogen-detection workflow for diagnostic testing. Rewire biological model benchmark database, release 2026-11-21-c8bea25e5abb. https://benchmarks.rewirebio.io/use-cases/diagnostic-rna-pathogen-detection/
Release 2026-11-21-c8bea25e5abb. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.
The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence
Question, sources and review
Who this is for, setting and exclusions
Inspect the UCSF respiratory RNA mNGS sensitivity-against-original-testing evidence before selecting a workflow and specimen-prep protocol for the intended respiratory-target population.
Who this is for
Clinical researchers scoping the available diagnostic-genomics evidence
Published comparisons cover one assay's sensitivity against original clinical respiratory panel testing (UCSF SURPI+, 103 of 110), five classifiers on the same 88 respiratory specimens against a 1144-result PCR panel (Carbo et al. 2022), thirteen clinical virology laboratory pipelines on the same 13 clinical datasets, five of them respiratory (de Vries et al. 2021, preprint), and ten submissions on one blood metagenome in the CAMI II pathogen challenge (Meyer et al. 2022). None of the sources found in this bounded search compares workflows prospectively on a large cohort with an adjudicated composite reference standard.
Clinical research scope
Clinical applicability is bounded to agreement with the original clinical panel on one residual-sample cohort; this is not a mixed DNA/RNA sample-prep comparison and not an RNA-virus-only subgroup score, and is not validated prospectively.
Outside this use case
A composite-PPA figure (98.7%, 110.5/113) that conflicts with the source's own printed numbers and is excluded from this intake
Non-respiratory specimen types, except in a comparison of several workflows on the same data that is recorded as proxy evidence and names its specimen types
Evidence collection plan
Evidence collection planCollecting evidence
Already on record: 6 comparisons shown · 28 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.
Comparison question
Which RNA sequencing workflow detects RNA pathogens reliably in the intended specimen and distinguishes real detections from host/background contamination?
Baselines, outcomes and validation requirements
Baselines to include
The conventional/author-introduced workflow measured in the linked primary source(s).
Outcomes to measure
The declared endpoint in this use case's active mapping(s); see evidence_gaps for what remains open.
Validation requirements
Independent held-out population matched to the intended clinical setting.
Qualified human scientific review before any clinical-validation claim.
Next collection task
A dedicated rewire-benchmarks protocol/run task with a prospective RNA-pathogen cohort and an adjudicated composite reference standard.
Sources, review and downloads
Automated source review · 2026-10-07 · Claude Sonnet AMP-integration worker, bounded transcription of Codex-checked primary values; independently reviewed by Codex (workbench/amp-supervision/primary-review.md, integration-review-corrections.md)
Bounded primary-source transcription, independently Codex-checked. No new model execution, independent experimental reproduction, qualified human scientific review or clinical validation.