rewirebio.iobenchmarks

Use caseDNA and genomesResearch and clinical research

Select an RNA pathogen-detection workflow for diagnostic testing

Which RNA sequencing workflow detects RNA pathogens reliably in the intended specimen and distinguishes real detections from host/background contamination?

In this release 1 evaluated endpoint · 1 tested configuration · Proxy evidence only · 2 recorded evidence gaps

Question and applicability

Inspect the UCSF respiratory RNA mNGS sensitivity-against-original-testing evidence before selecting a workflow and specimen-prep protocol for the intended respiratory-target population.

Who this is for
  • Clinical researchers scoping the available diagnostic-genomics evidence
  • Computational researchers comparing exact evaluated configurations
Research setting

Research and clinical research. See Clinical research scope for what the evidence does not establish.

Biological setting
A UCSF respiratory RNA metagenomic next-generation sequencing (mNGS) assay (RNA extraction, DNase treatment, cDNA synthesis; SURPI+ pipeline) achieves 93.6% (103 of 110) sensitivity against original clinical respiratory-virus-panel testing in a residual pre-DTCA mixed respiratory-target cohort (adenovirus transcripts included via transcription).

Outside this use case

  • A composite-PPA figure (98.7%, 110.5/113) that conflicts with the source's own printed numbers and is excluded from this intake
  • Non-respiratory specimen types

Inputs and expected output

Inputs you need

  • Respiratory specimen RNA (DNase-treated), reverse-transcribed to cDNA libraries
  • The original clinical reference assay(s) available for sensitivity comparison

Expected output

A sourced sensitivity-against-original-clinical-testing figure for one respiratory mNGS assay; no general RNA-virus-only diagnostic accuracy claim.

Clinical research scope

Clinical applicability is bounded to agreement with the original clinical panel on one residual-sample cohort; this is not a mixed DNA/RNA sample-prep comparison and not an RNA-virus-only subgroup score, and is not validated prospectively.

Evaluated evidence

Evidence is grouped by its protocol. Relevance refers to the stated endpoint and context; it is separate from clinical validation and from the review method. Limits specific to each evaluation are listed with it.

UCSF respiratory RNA mNGS original-testing sensitivity protocol

Current source-reviewed mapping

Proxy evidence: transfer to this question is limited

Sensitivity against original clinical RVP testing (RNA extraction, DNase treatment, cDNA synthesis; mixed respiratory-virus target panel including transcriptionally detected adenovirus) is a proxy for the declared RNA-pathogen diagnostic-accuracy decision: the specimen prep is an RNA workflow, not a mixed DNA/RNA sample-prep comparison, and this is not an RNA-virus-only subgroup score. The conflicting composite PPA figure (98.7%, 110.5/113) is excluded.

Assessed endpoint
Sensitivity against original clinical respiratory-virus-panel testing (103/110, 93.6%) for RNA mNGS on a residual-sample pre-DTCA mixed respiratory-target cohort, adenovirus transcripts included
Evaluation protocol
UCSF respiratory RNA mNGS original-testing sensitivity protocol
Computational task
A reviewed task relationship is not recorded for this protocol.
Input and population constraints
  • Inspect every linked evaluation's source locator and preserved conflicts before citing a result.
  • Do not combine this mapping's evaluations with any other protocol's results.

Limits on interpretation

  • RNA-only specimen preparation supports RNA pathogen-detection workflow; tested target mix includes adenovirus, a DNA virus detected via transcription. Do not describe the 93.6% as a pure RNA-virus-only subgroup score.
  • Multiple detected targets are weighted so each specimen contributes one observation; out-of-panel mNGS positive calls are not counted as false positives.
  • Positive-specimen BAL/swab count disagreement in Results vs Methods remains unresolved; total 110 positives and 81 negatives agree.
  • No confidence interval extracted for original sensitivity. After selective discrepancy adjudication, report PPA/NPA rather than sensitivity; DTCA measurements must remain separate.
  • No foundation model or independent external replication; clinical residual-sample validation study supplies conventional baseline evidence.
  • Post-DTCA printed PPA98.7% (110.5/113) is arithmetically inconsistent; no DTCA endpoint selected or corrected.

Automated source review · 2026-10-07 · Claude Sonnet AMP-integration worker, bounded transcription of Codex-checked primary values; independently reviewed by Codex (workbench/amp-supervision/primary-review.md, integration-review-corrections.md)

Bounded primary-source transcription, independently Codex-checked. No new model execution, independent experimental reproduction, qualified human scientific review or clinical validation.

Evaluated configurations

Each configuration below belongs to this protocol. Inspect its inputs, population and scoring conditions before comparing it with another evaluation.

UCSF respiratory RNA mNGS original-testing sensitivity tested configuration

Author-reported evaluation · Source checked

UCSF respiratory RNA mNGS original-testing sensitivity; bounded primary-source candidate.

Inspect results, conditions and reproduction (1 recorded result)
Population and split
191 residual UCSF clinical samples: 110 RVP-virus-positive, 81 negative. Results says positive 104 upper respiratory swabs + 6 BAL; Methods instead 103 + 7, retained unresolved. · Residual clinical accuracy cohort; no model training/test split reported
Inputs and adaptation
Respiratory RNA (DNase-treated), reverse-transcribed cDNA libraries; original clinical multiplex RT-PCR reference · Conventional sequence alignment pipeline; no foundation-model adaptation
Evaluation budget
Not reported
Runtime and memory

Runtime and memory measurements are not reported in this evaluation. A study budget is not a runtime or memory measurement.

Recorded results for this configuration
MetricValueCoverageUncertainty and source
sensitivity_against_original_rvp_testing93.6% (103 of 110)

percent · higher

Not reported scored / Not reported eligible

Not reported

Result provenance

Uncertainty: Not reported/extracted for this scoped endpoint

Evaluation methods, evidence and reproduction

No execution recipe has been verified for this exact configuration and evaluation. Inspect its methods and original run documentation before attempting reproduction.

Open the protocol's results and comparison checks →

Mapping sources and review metadata

Mapping use-case-mapping-amp-20261007-issue14 · revision 1

Add Codex-checked primary-source protocol evidence from the bounded AMP intake (rewire.it#365).

Reviewed evidence fingerprint 4424bab553c78aafbbcf4a181468877e47b00bfc9ab02554c879483ef5f392cc

Limitations and missing evidence

These gaps apply to the question as a whole. Absence of evidence is not a zero score.

  • A positive-specimen BAL/swab count disagreement between the source's Results and Methods sections (104+6 vs 103+7) is unresolved and preserved.
  • No confidence interval is extracted for this sensitivity figure in the inspected text.

Planned work

These plans do not contribute measured results or evaluated winners above.

Contribute evidence or propose a correction

Evidence collection plan

Collecting evidence

Mapped evidence already covers 1 evaluated endpoint, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.

Comparison question

Which RNA sequencing workflow detects RNA pathogens reliably in the intended specimen and distinguishes real detections from host/background contamination?

Baselines, outcomes and validation requirements

Baselines to include

  • The conventional/author-introduced workflow measured in the linked primary source(s).

Outcomes to measure

  • The declared endpoint in this use case's active mapping(s); see evidence_gaps for what remains open.

Validation requirements

  • Independent held-out population matched to the intended clinical setting.
  • Qualified human scientific review before any clinical-validation claim.

Next collection task

A dedicated rewire-benchmarks protocol/run task with a prospective RNA-pathogen cohort and an adjudicated composite reference standard.

Sources and review

Automated source review · 2026-10-07 · Claude Sonnet AMP-integration worker, bounded transcription of Codex-checked primary values; independently reviewed by Codex (workbench/amp-supervision/primary-review.md, integration-review-corrections.md)

Bounded primary-source transcription, independently Codex-checked. No new model execution, independent experimental reproduction, qualified human scientific review or clinical validation.

Release provenance and downloads

Release 2026-10-07-1448159e6a81

Use-case input digest d60fd7f669bfec7bd34ec6e5080d8e1cb4e8186f8286ead60888848f1e20e001

Download questions, collection plans and review metadata (JSON) · Verify release checksums

Question use-case-diagnostic-rna-pathogen-detection. Any numerical results on this page come from this release's existing evaluation records.