rewirebio.iobenchmarks

Use caseDNA and genomesResearch and clinical research

Select an execution workflow for large-scale diagnostic genomics

Which eligible model configuration and execution workflow can meet a diagnostic genomics team's resource, throughput and reproducibility constraints?

What the evidence shows

Eleven published execution benchmarks are linked, alongside the DRAGEN 4.2 developer total runtime for HG002 (1,838.5 seconds); four are withheld pending source concerns (see the end of this summary), and pretraining compute for NTv3 and HyenaDNA is linked only as outside scope.

Of the seven active benchmarks, five measure conventional variant-calling pipelines and two measure genomic foundation models outside any diagnostic workflow: inference on microbial genomes, and fine-tuning and test-set scoring on human eQTL variants. Only the foundation-model benchmarks print repeated runs (five trials for GenomeOcean, three seeds for the Nucleotide Transformer); none reruns a variant-calling pipeline to measure concordance of the calls, none prints variant-calling accuracy in the same table as runtime, and hardware differs within most comparisons, so differences mix software and hardware. On ten WGS samples run by a University of Oslo and Oslo University Hospital group with no declared competing interests (Samarakoon et al. 2025), read mapping plus HaplotypeCaller for NA12878 (47.55x) took 2,317.7 minutes on a CPU-only GATK pipeline, 123.45 minutes with Parabricks on four L4 GPUs, 78.12 on four A100 GPUs, 54.22 on eight H100 GPUs and 60.29 with DRAGEN v4.2.

On the same H100 machine, Parabricks DeepVariant took 14.58 minutes for NA12878 with the version the pipeline used (printed 4.1.0-1; Methods say 4.3.0-1) and 16.05 minutes with the updated version (printed 4.3.0-1; Discussion says 4.3.1-1). On one HG002 30x genome on Google Cloud (O'Connell et al. 2023; the authors' employer funded the study and is an NVIDIA, AWS and Google alliance partner), HaplotypeCaller took 38.8 hours and cost 67.9 USD on a 32-vCPU machine and 0.59 hours and 17.5 USD on eight A100 GPUs, while the somatic caller LoFreq cost 8.1 USD on the CPU machine and 19 to 30.1 USD on 2 to 8 A100 GPUs. On Google Cloud machines of similar hourly cost (Franzoso et al. 2025, no declared conflicts), Sentieon DNASeq took 0:14:00 to 0:16:00 per exome at 0.82 to 1.03 USD and 3:00:00 to 3:50:00 per genome at 8.02 to 10.7 USD, and Parabricks Germline on one T4 GPU took 0:10:00 to 0:14:00 per exome at 0.71 to 0.93 USD and 4:06:00 to 4:40:00 per genome at 8.13 to 10.6 USD; costs exclude the Sentieon licence, and two genome runtime cells that cannot be genome runtimes are disputed and left out of these ranges.

On human WGS at five read depths on the HPC nodes of a regional genome project (Marangoni et al. 2026; the GeNePi developers, no declared competing interests; one unnamed sample per depth), a 30x genome took 17h with the CPU GATK equivalent and 1h 36m, 42 m and 30m with Parabricks v4.3.0 on two V100, two A100 and four A100 GPUs, and a 100x genome 60h, 6h 30m, 2h 30m and 2h 06m. On one 120-vCPU Azure instance (Hu et al. 2026; all authors employed by Sentieon), DNAscope took 18.9 minutes for 30x Illumina, DNAscope LongRead 41.2 minutes for 30x PacBio HiFi and DNAscope Hybrid 85.8 minutes for 10x PacBio HiFi plus 35x Illumina, at 1.1, 2.5 and 5.2 USD on demand excluding the licence; each pipeline had different input data, so the three are not a ranking. For GenomeOcean on one H100 (Yu et al. 2026; two authors are at the DOE Joint Genome Institute, which develops GenomeOcean; GTDB bacterial genomes; mean and standard deviation over five trials), embedding extraction with the 4B model ran at 21,386 ± 126 input tokens per second with 72.14 GB peak GPU memory in BF16 and 22,401 ± 204 with 67.91 GB with FP8 weights, and generation ran at 1818.79 ± 7.95 generated tokens per second in BF16 and 2169.04 ± 24.30 with FP8 weights, activations and KV cache.

On a human eQTL variant classification task (Hegde et al. 2025; Kingston University authors, independent of the model developers; 8,862 test variants), LoRA fine-tuning of Nucleotide Transformer 500M for 5 epochs took 7.53, 7.50 and 7.48 hours over three seeds and scoring the test split 8.49, 8.51 and 8.49 minutes, on GPUs the Methods give as two TITAN Xp; fine-tuning to convergence took approximately 19 hours for the Nucleotide Transformer and around 1 hour per pruned DNABERT-2 model on two Quadro RTX 8000. No active linked result measures variant-calling accuracy against a truth set; the GenomeOcean ARI values score metagenomic binning against GTDB species labels. The four withheld studies are Zheng et al. 2026 (Clair3 developers; Clair3 and DeepVariant on HG003 long reads), whose supplementary runtimes, DeepVariant version and DeepVariant accuracy differ from its Results text and whose accuracy uses one GIAB sample's v4.2.1 confident regions, which leave out the hardest regions; Aspidova et al. 2026 (ModernGENA and GENA-LM developers; four DNA language model families on one A100), whose printed tokens per second are far too low for its batch sizes; Pearce et al. 2026 (EVEE developers, three of whom co-authored Evo 2; Evo 2 embedding extraction for all ClinVar variants), whose source carries a concern about another figure; and Dalla-Torre et al. 2025 (Nucleotide Transformer developers; IA3 fine-tuning time), whose Methods fine-tuning times conflict with its Discussion.

Apart from the five GenomeOcean trials and the three Nucleotide Transformer seeds, each comparison prints one value per sample and set-up on at most ten samples, and costs are list prices at the time of each study.

You bring
  • The diagnostic genomics team's available hardware configuration
  • The required per-sample turnaround and batch throughput
You want
Sourced per-sample runtime, cost, speedup, cost-saving, throughput and peak-memory values for exact pipeline, model, version and hardware configurations, with the limits of each comparison; no foundation-model execution benchmark inside a diagnostic workflow, no reproducibility measurement and no clinical turnaround claim.
Evidence in this release
11 comparisons shown · 9 held for review · 11 tools · 39 configurations
Kept up to date
Evidence last checked 11 October 2026 · re-checked monthly, next due 11 November 2026
Executive Takeaway

For "Select an execution workflow for large-scale diagnostic genomics", on WGS germline pipeline stages on CPU, GPU and DRAGEN (Samarakoon et al. 2025), Read mapping (Runtime), DRAGEN scored 14.08 min, reported by Benchmarking accelerated next-generation sequencing analysis pipelines.

Best reported valueDRAGENBenchmarking accelerated next-generation sequencing…
Runtime14.08 minWGS germline pipeline stages on CPU, GPU and DRAGEN (Samarakoon et al. 2025), Read mapping
Baseline on this protocolCPU334.20 min
Ranked tools55 from independent studies

5 tools have a value on this protocol; 5 of them come from studies whose authors did not develop the tool. The use case's baseline, CPU, scored 334.20 min on the same protocol.

Ranking

Ranked on WGS germline pipeline stages on CPU, GPU and DRAGEN (Samarakoon et al. 2025), Read mapping (Runtime, lower is better). Only results on this protocol are ranked; other tools are listed below the table.

3 tools in 5 rows, from 2 sources. 5 of 5 results (100%) come from an independent study.

Rank▲ModelRuntime (lower is better)vs CPUEvidenceCoverageReported byLicense
#114.08 min+95.8%Source-checked Independent study1/11Benchmarking accelerated next-generation…Not recorded
#214.15 min+95.8%Source-checked Independent study6/11Benchmarking accelerated next-generation…Not recorded
#314.98 min+95.5%Source-checked Independent study6/11Benchmarking accelerated next-generation…Not recorded
#426.02 min+92.2%Source-checked Independent study6/11Benchmarking accelerated next-generation…Not recorded
#5
CPU[-] Baseline
334.20 minSource-checked Independent study2/11Benchmarking accelerated next-generation…Not recorded
8 tools are evaluated elsewhere in this use case but not on this protocol, so not ranked

Baseline

GATK Best Practices on CPU. The CPU GATK best-practice germline pipeline is the reference that accelerated workflows such as Parabricks and DRAGEN are timed and costed against (Samarakoon et al. 2025). Its configurations run GATK HaplotypeCaller on CPUs, or DeepVariant on the DeepVariant stage, so it is a hardware reference rather than a pre-machine-learning method.

No linked comparison scores a machine-learning method against the baseline on the same data yet, so this evidence cannot say whether machine learning beats it.

Tools compared

Comparisons

Unified model comparison for this use case. Compare tools against protocol baselines in a single view.

WGS germline pipeline stages on CPU, GPU and DRAGEN (Samarakoon et al. 2025)

Direct evidenceIndependent study: Reported in a study whose authors did not develop the evaluated tool.

Runtime (lower is better), with the other metrics below each value. The best value in each column is highlighted.

ToolRead mappingHaplotypeCaller or GSVC callingDeepVariant callingMapping plus HaplotypeCaller or GSVC
DRAGENconventional
14.08 min (best in column)NA12778 (9.82x)7.84 minNA12778 (9.82x)Not reported21.92 minNA12778 (9.82x)
H1003 sweepsconventional
14.15 minNA12778 (9.82x)4.87 min (best in column)NA12778 (9.82x)6.08 minNA12778 (9.82x)19.02 min (best in column)NA12778 (9.82x)
CPUBaselineconventional
334.20 minNA12778 (9.82x)613.88 minNA12778 (9.82x)337.75 minNA12778 (9.82x)948.08 minNA12778 (9.82x)
Caveats and method

What each column measures:

  • Read mapping: Wall-clock minutes for the read mapping stage on ten WGS samples for CPU-only GATK, Parabricks on L4, A100 and H100 GPUs, and DRAGEN v4.2
  • HaplotypeCaller or GSVC calling: Wall-clock minutes for the HaplotypeCaller or DRAGEN GSVC calling stage on ten WGS samples for CPU-only GATK, Parabricks on L4, A100 and H100 GPUs, and DRAGEN v4.2
  • DeepVariant calling: Wall-clock minutes for the DeepVariant calling stage on ten WGS samples for CPU-only DeepVariant and Parabricks DeepVariant on L4, A100 and H100 GPUs
  • Mapping plus HaplotypeCaller or GSVC: Wall-clock minutes for the total of read mapping and HaplotypeCaller or DRAGEN GSVC on ten WGS samples for CPU-only GATK, Parabricks on L4, A100 and H100 GPUs, and DRAGEN v4.2
  • One run per sample and configuration; no repeats or variance printed.
  • Hardware differs between columns (CPU cluster, two GCP VMs, an HPC GPU node and a DRAGEN server), so differences mix software and hardware.
  • CPU-only nodes were shared with other users; GCP VMs ran in a multi-tenant environment (Table S2 notes).
  • DRAGEN mapping time was assembled from three process logs because its processes run concurrently (Methods 2.4).
  • No linked source repeats runs, measures run-to-run concordance of the calls, or reports accuracy next to runtime, so this evidence bears on the resource and throughput part of the question only, not on reproducibility.
  • Hardware differs between columns (CPU cluster, two GCP VMs and an HPC GPU node), so differences mix software and hardware.
  • DRAGEN has no DeepVariant stage; its column prints NA and has no evaluation.
  • Total is the sum of read mapping and HaplotypeCaller or GSVC stages; DeepVariant is excluded. The NA12778 L4 total (117.59) does not equal its stage values (26.02 + 5.92 = 31.94); that result is disputed and not shown.

Protocols: WGS germline pipeline wall-clock, read mapping stage (Samarakoon et al. 2025 Table S3a), WGS germline pipeline wall-clock, HaplotypeCaller or DRAGEN GSVC calling stage (Samarakoon et al. 2025 Table S3b.1), WGS germline pipeline wall-clock, DeepVariant calling stage (Samarakoon et al. 2025 Table S3b.2), WGS germline pipeline wall-clock, total of read mapping and HaplotypeCaller or DRAGEN GSVC (Samarakoon et al. 2025 Table S3c)

Primary benchmark evaluation shown (WGS germline pipeline stages on CPU, GPU and DRAGEN (Samarakoon et al. 2025)). 10 additional study protocols (Fine-tuning time to convergence on the Enformer eQTL task (Hegde et al. 2025), Sentieon versus Parabricks on GCP machines of similar hourly cost (Franzoso et al. 2025), Germline callers on CPU and GPU cloud machines (O'Connell et al. 2023) and others) are cited in individual tool drawer rows and under Question, sources and review.

Held for review

Recorded but not shown as evidence until the reason below is resolved.

Evo 2 7B embedding extraction for all ClinVar variants (Pearce et al. 2026)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (gfm-guide-20261014-eval-evee-embedding-extraction-compute); no evaluation on the protocol is eligible evidence.

Pretraining GPUs and runtime for short-range models (HyenaDNA Table A.2)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked.

Nucleotide Transformer IA3 fine-tuning time on one 8 × A100 node (Dalla-Torre et al. 2025)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (gfm-guide-20261014-eval-nt-2-5b-ia3-fine-tuning-runtime, gfm-guide-20261014-eval-nt-500m-ia3-fine-tuning-runtime); no evaluation on the protocol is eligible evidence.

DNA language model inference throughput on one A100 (Aspidova et al. 2026)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (model-execution-20261011-eval-aspidova2026-dnabert-2, model-execution-20261011-eval-aspidova2026-gena-base, model-execution-20261011-eval-aspidova2026-gena-large, model-execution-20261011-eval-aspidova2026-moderngena-base, model-execution-20261011-eval-aspidova2026-moderngena-base-fa2-bf16); no evaluation on the protocol is eligible evidence.

Clair3 versus DeepVariant small-variant accuracy on HG003 (Zheng et al. 2026) (ONT)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (model-execution-20261011-eval-zheng2026-clair3-accuracy-hg003-accuracy-ont, model-execution-20261011-eval-zheng2026-clair3-pileup-accuracy-hg003-accuracy-ont, model-execution-20261011-eval-zheng2026-deepvariant-accuracy-hg003-accuracy-ont); no evaluation on the protocol is eligible evidence.

Clair3 versus DeepVariant small-variant accuracy on HG003 (Zheng et al. 2026) (PacBio Revio)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (model-execution-20261011-eval-zheng2026-clair3-accuracy-hg003-accuracy-pacbio, model-execution-20261011-eval-zheng2026-clair3-pileup-accuracy-hg003-accuracy-pacbio, model-execution-20261011-eval-zheng2026-deepvariant-accuracy-hg003-accuracy-pacbio); no evaluation on the protocol is eligible evidence.

Clair3 GPU workflow stage runtimes (Zheng et al. 2026)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (model-execution-20261011-eval-zheng2026-clair3-1-2-0-gpu-hg003-clair3-stage-runtime-ont, model-execution-20261011-eval-zheng2026-clair3-1-2-0-gpu-hg003-clair3-stage-runtime-pacbio); no evaluation on the protocol is eligible evidence.

Clair3 versus DeepVariant in GPU mode, runtime and memory on HG003 (Zheng et al. 2026) (ONT)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (model-execution-20261011-eval-zheng2026-clair3-1-2-0-gpu-hg003-resource-ont, model-execution-20261011-eval-zheng2026-deepvariant-1-10-beta-gpu-hg003-resource-ont); no evaluation on the protocol is eligible evidence.

Clair3 versus DeepVariant in GPU mode, runtime and memory on HG003 (Zheng et al. 2026) (PacBio Revio)

Withheld: a cited source is unchecked or has evidence concerns; protocol evidence is disputed or unchecked; a reviewed evaluation is no longer eligible (model-execution-20261011-eval-zheng2026-clair3-1-2-0-gpu-hg003-resource-pacbio, model-execution-20261011-eval-zheng2026-deepvariant-1-10-beta-gpu-hg003-resource-pacbio); no evaluation on the protocol is eligible evidence.

Not covered yet

  • Measured peak memory is printed only for GenomeOcean inference on one H100 (Yu et al. 2026); no active linked variant-calling study prints it (Samarakoon et al. 2025 and Franzoso et al. 2025 show memory use only in figures), and the Clair3 and DeepVariant memory table (Zheng et al. 2026) is withheld.
  • No linked study reruns the same pipeline on the same input and prints concordance of the calls, so run-to-run reproducibility of variant calls is not measured; Yu et al. 2026 print spread over five trials of throughput only, Hegde et al. 2025 print three seeds of fine-tuning and scoring time with identical aggregate accuracy but no per-variant concordance, Samarakoon et al. 2025 compare Parabricks on two A100 clusters only in a figure, and O'Connell et al. 2023 repeated DGX runs but print only the final run.
  • No active linked study reports variant-calling accuracy in the same printed table as runtime; Samarakoon et al. 2025 show accuracy and CPU concordance in Figure 5 only, and the Clair3 and DeepVariant accuracy table (Zheng et al. 2026) is withheld and not stated to come from the same runs as its runtime table.
  • Two active foundation-model studies are linked, GenomeOcean inference on microbial genomes (Yu et al. 2026) and Nucleotide Transformer and DNABERT-2 fine-tuning and test-set scoring time on human eQTL variants on older GPUs (Hegde et al. 2025); none measures a genomic foundation model inside a diagnostic workflow on human data. The four-family A100 throughput table (Aspidova et al. 2026) is withheld because its printed tokens per second are inconsistent with its batch sizes, and the Evo 2 ClinVar embedding extraction compute (Pearce et al. 2026) and the Nucleotide Transformer IA3 fine-tuning time (Dalla-Torre et al. 2025) are withheld pending source concerns.
  • Marangoni et al. 2026, Hu et al. 2026 and Zheng et al. 2026 do not state how many runs lie behind each printed runtime.
  • Costs are cloud list prices or authors' estimates at the time of each study and exclude storage and software licences unless stated.
  • A separate ~2-hour joint-aggregation figure over 3,202 genomes at concurrency 200 is a different operation and is not assigned to this single-sample runtime.
  • No machine-learning method has been compared with the CPU GATK pipeline on the same data in the linked evidence. On the Samarakoon 2025 DeepVariant stage, the CPU and GPU runs use the same DeepVariant model, so they compare hardware, not methods.

Contribute evidence or propose a correction

How to cite

Select an execution workflow for large-scale diagnostic genomics. Rewire biological model benchmark database, release 2026-11-21-c8bea25e5abb. https://benchmarks.rewirebio.io/use-cases/diagnostic-genomics-model-execution/

Release 2026-11-21-c8bea25e5abb. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.

The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence

Question, sources and review

Who this is for, setting and exclusions

Compare published per-sample runtime and cost of CPU, GPU and DRAGEN server execution set-ups that resemble the team's hardware and sequencing type when budgeting a workflow; most are conventional variant-calling pipelines, the two active foundation-model benchmarks (inference on microbial genomes; fine-tuning and test-set scoring on human eQTL variants) are outside any diagnostic workflow, and none measures run-to-run concordance of variant calls.

Who this is for
  • Clinical researchers scoping the available diagnostic-genomics evidence
  • Computational researchers comparing exact evaluated configurations
Biological setting
Published execution benchmarks, each on stated inputs: CPU-only GATK, Parabricks on L4, A100 and H100 GPUs and DRAGEN v4.2 on ten germline WGS samples (Samarakoon et al. 2025); two germline and four somatic callers on CPU and 2, 4 and 8 GPU cloud and DGX machines on one HG002 30x genome (O'Connell et al. 2023); Sentieon DNASeq and Parabricks Germline on Google Cloud machines of similar hourly cost, five exomes and five genomes (Franzoso et al. 2025); the DRAGEN 4.2 developer total runtime for HG002, 1,838.5 seconds on one hardware configuration and 5,521.21 seconds on an AWS f1.4xlarge configuration, whose stage times cannot be summed because of concurrency; Parabricks v4.3.0 on V100 and A100 nodes against the CPU GATK equivalent at five read depths of human WGS (Marangoni et al. 2026); three Sentieon DNAscope pipelines on one Azure instance (Hu et al. 2026); GenomeOcean inference in BF16 and FP8 on one H100 with microbial genomes (Yu et al. 2026); and LoRA fine-tuning and test-set scoring time of Nucleotide Transformer 500M on 2 × TITAN Xp, and fine-tuning time of pruned DNABERT-2 on 2 × Quadro RTX 8000, on a human eQTL variant classification task (Hegde et al. 2025). Four further linked studies are withheld pending source concerns: Clair3 and DeepVariant on HG003 long reads (Zheng et al. 2026), DNA language model inference on one A100 (Aspidova et al. 2026), Evo 2 embedding extraction for all ClinVar variants (Pearce et al. 2026) and Nucleotide Transformer IA3 fine-tuning time (Dalla-Torre et al. 2025). Pretraining compute for NTv3 and HyenaDNA is linked as outside scope. Hardware differs within most comparisons, so differences mix software and hardware.
Clinical research scope
Clinical applicability is not established: the linked evidence is published runtime, cost and resource use of conventional variant-calling pipelines on a few samples and two foundation-model benchmarks outside any diagnostic workflow (inference on microbial genomes; fine-tuning and test-set scoring time on human eQTL variants), not a foundation-model execution benchmark inside a diagnostic workflow, a reproducibility study or a clinical validation.

Outside this use case

  • Foundation-model execution inside a diagnostic workflow on human data (none found in the 2026-10-11 pass)
  • Clinical-reporting turnaround time
Evidence collection plan
Evidence collection planCollecting evidence

Already on record: 11 comparisons shown · 9 held for review · 11 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.

Comparison question

Which eligible model configuration and execution workflow can meet a diagnostic genomics team's resource, throughput and reproducibility constraints?

Baselines, outcomes and validation requirements

Baselines to include

  • The conventional/author-introduced workflow measured in the linked primary source(s).

Outcomes to measure

  • The declared endpoint in this use case's active mapping(s); see evidence_gaps for what remains open.

Validation requirements

  • Independent held-out population matched to the intended clinical setting.
  • Qualified human scientific review before any clinical-validation claim.

Next collection task

A dedicated rewire-benchmarks protocol/run task that actually executes an eligible foundation-model configuration and measures its resource/throughput budget.

Sources, review and downloads

Automated source review · 2026-10-07 · Claude Sonnet AMP-integration worker, bounded transcription of Codex-checked primary values; independently reviewed by Codex (workbench/amp-supervision/primary-review.md, integration-review-corrections.md)

Bounded primary-source transcription, independently Codex-checked. No new model execution, independent experimental reproduction, qualified human scientific review or clinical validation.

Release 2026-11-21-c8bea25e5abb · question use-case-diagnostic-genomics-model-execution

Use-case input digest bfbb7ab9303becad1e3cb395568c26f41e72ddb67cd1a89a06d9eaa0f76bd95f

Download questions and review metadata (JSON) (gzip) · Verify release checksums (gzip)