Contested result

DeepSomatic and Majority-vote consensus of somatic callers: Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling reverses the order

Low severity
Not yet reviewed
Ordering reversal
Different tasks across studies

Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling ranks Majority-vote consensus of somatic callers above DeepSomatic in 20 of 20 comparisons; other studies rank DeepSomatic above Majority-vote consensus of somatic callers in 8 of 10.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

A benchmarking study of individual… ranks DeepSomatic above Majority-vote consensus of somatic callers in 8 of its 12 comparisons and Majority-vote consensus of somatic callers higher in 2; 2 are ties. Prospectively frozen external… ranks Majority-vote consensus of somatic callers above DeepSomatic in all 20 comparisons that are not ties (4 are ties).

Rank DeepSomatic higher (1): A benchmarking study of individual…
Rank Majority-vote consensus of somatic callers higher (1): Prospectively frozen external… (disagrees)
Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

DeepSomatic higherTieMajority-vote consensus of somatic callers higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.

Papers compared (2)
Paper	Role	Ranks DeepSomatic higher	Ranks Majority-vote consensus of somatic callers higher	Ties
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling and 1 more record · DOI	Disagrees	0	20	4
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing and 1 more record · DOI	Agrees	8	2	2

Comparisons in Prospectively frozen external… (24)

Protocol	Dataset	Metric	DeepSomatic	Majority-vote consensus of somatic callers	Higher
HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	F1 (SNVs and indels, HG008 all-benchmark region), fraction	0.776	0.794	Majority-vote consensus of somatic callers
HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False negatives (SNVs and indels, HG008 all-benchmark region), count	6590	4710	Majority-vote consensus of somatic callers
HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False positives (SNVs and indels, HG008 all-benchmark region), count	133	7	Majority-vote consensus of somatic callers
HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Precision (SNVs and indels, HG008 all-benchmark region), fraction	0.989	0.999	Majority-vote consensus of somatic callers
HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Recall (SNVs and indels, HG008 all-benchmark region), fraction	0.638	0.741	Majority-vote consensus of somatic callers
HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	True positives (SNVs and indels, HG008 all-benchmark region), count	11600	13500	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	F1 (indel only, HG008 primary region), fraction	0.69	0.809	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False negatives (indel only, HG008 primary region), count	2320	1080	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False positives (indel only, HG008 primary region), count	28	2	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Precision (indel only, HG008 primary region), fraction	0.989	0.999	Tie
HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Recall (indel only, HG008 primary region), fraction	0.529	0.781	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	True positives (indel only, HG008 primary region), count	2610	3860	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	F1 (SNVs and indels, HG008 primary region), fraction	0.897	0.914	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False negatives (SNVs and indels, HG008 primary region), count	2510	1190	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False positives (SNVs and indels, HG008 primary region), count	124	5	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Precision (SNVs and indels, HG008 primary region), fraction	0.989	1	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Recall (SNVs and indels, HG008 primary region), fraction	0.821	0.915	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	True positives (SNVs and indels, HG008 primary region), count	11500	12800	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	F1 (SNV only, HG008 primary region), fraction	0.984	0.985	Tie
HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False negatives (SNV only, HG008 primary region), count	186	113	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	False positives (SNV only, HG008 primary region), count	96	3	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Precision (SNV only, HG008 primary region), fraction	0.989	1	Majority-vote consensus of somatic callers
HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	Recall (SNV only, HG008 primary region), fraction	0.98	0.988	Tie
HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	NIST/GIAB HG008 tumour-normal WGS (HG008-T pancreatic cancer cell line, matched normal), HG008-T draft somatic small-variant benchmark v0.3	True positives (SNV only, HG008 primary region), count	8890	8970	Tie

Comparisons in A benchmarking study of individual… (12)

Protocol	Dataset	Metric	DeepSomatic	Majority-vote consensus of somatic callers	Higher
SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	F1 (indel only), fraction	0.812	0.752	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	False positives (indel only), count	10	9	Majority-vote consensus of somatic callers
SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	False positive rate (indel only), fraction	6.64e-8	5.98e-8	Majority-vote consensus of somatic callers
SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	Precision (indel only), fraction	0.804	0.795	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	Recall (indel only), fraction	0.82	0.76	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	True positives (indel only), count	41	38	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	F1 (SNV only), fraction	0.901	0.881	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	False positives (SNV only), count	34	58	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	False positive rate (SNV only), fraction	2.26e-7	3.85e-7	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	Precision (SNV only), fraction	0.966	0.943	DeepSomatic
SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	Recall (SNV only), fraction	0.843	0.838	Tie
SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	SEQC2 HCC1395/HCC1395BL WES, Fudan replicate (WES_FD_T_1), high-confidence regions	True positives (SNV only), count	978	972	Tie

What disagrees

Other studies: rank DeepSomatic above Majority-vote consensus of somatic callers in 8 of 10 comparisons
This study: ranks Majority-vote consensus of somatic callers above DeepSomatic in 20 of 20 comparisons

Compared: DeepSomatic, Majority-vote consensus of somatic callers
Study that disagrees: Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling, AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)
Other studies: A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing, Guille et al. 2025, Supplementary Table S7 (validation dataset)
Rule: Ordering reversal; different tasks across studies
Model identity: Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (40)
Study	Tool	Protocol or benchmark	Metric	Value
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	F1 (SNVs and indels, HG008 all-benchmark region)	0.776
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	False negatives (SNVs and indels, HG008 all-benchmark region)	6590
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	False positives (SNVs and indels, HG008 all-benchmark region)	133
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	Precision (SNVs and indels, HG008 all-benchmark region)	0.989
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	Recall (SNVs and indels, HG008 all-benchmark region)	0.638
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	True positives (SNVs and indels, HG008 all-benchmark region)	11600
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	F1 (indel only, HG008 primary region)	0.69
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	False negatives (indel only, HG008 primary region)	2320
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	False positives (indel only, HG008 primary region)	28
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	Recall (indel only, HG008 primary region)	0.529
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	True positives (indel only, HG008 primary region)	2610
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	F1 (SNVs and indels, HG008 primary region)	0.897
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	False negatives (SNVs and indels, HG008 primary region)	2510
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	False positives (SNVs and indels, HG008 primary region)	124
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	Precision (SNVs and indels, HG008 primary region)	0.989
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	Recall (SNVs and indels, HG008 primary region)	0.821
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	True positives (SNVs and indels, HG008 primary region)	11500
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	False negatives (SNV only, HG008 primary region)	186
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	False positives (SNV only, HG008 primary region)	96
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	DeepSomatic 1.10.0, bundled WGS model (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	Precision (SNV only, HG008 primary region)	0.989
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Majority of Mutect2, Strelka2 and DeepSomatic (at least two callers) (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	F1 (SNVs and indels, HG008 all-benchmark region)	0.794
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Majority of Mutect2, Strelka2 and DeepSomatic (at least two callers) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	F1 (SNVs and indels, HG008 primary region)	0.914
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	False positives (SNVs and indels, HG008 all-benchmark region)	7
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	Precision (SNVs and indels, HG008 all-benchmark region)	0.999
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	False positives (indel only, HG008 primary region)	2
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	False positives (SNVs and indels, HG008 primary region)	5
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	Precision (SNVs and indels, HG008 primary region)	1
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	False positives (SNV only, HG008 primary region)	3
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Strict intersection of Mutect2, Strelka2 and DeepSomatic (all three callers) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	Precision (SNV only, HG008 primary region)	1
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	False negatives (SNVs and indels, HG008 all-benchmark region)	4710
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	Recall (SNVs and indels, HG008 all-benchmark region)	0.741
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 all-benchmark region (sensitivity analysis), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 4)	True positives (SNVs and indels, HG008 all-benchmark region)	13500
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	F1 (indel only, HG008 primary region)	0.809
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	False negatives (indel only, HG008 primary region)	1080
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	Recall (indel only, HG008 primary region)	0.781
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), indels, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'INDEL')	True positives (indel only, HG008 primary region)	3860
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	False negatives (SNVs and indels, HG008 primary region)	1190
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	Recall (SNVs and indels, HG008 primary region)	0.915
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs and indels combined, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 1)	True positives (SNVs and indels, HG008 primary region)	12800
Prospectively frozen external benchmarking reveals benchmark-context dependence in somatic variant calling; AmbioSpire P003/T001 publication source data and reproducibility package v1.0.0 (Zenodo)	Union of Mutect2, Strelka2 and DeepSomatic (at least one caller) (Hasan 2026)	HG008 primary region (nogermlineinterference), SNVs, Mutect2, Strelka2, DeepSomatic and ensembles (Hasan 2026 Table 2, Class 'SNV')	False negatives (SNV only, HG008 primary region)	113

In the other studies (20)
Study	Tool	Protocol or benchmark	Metric	Value
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	F1 (indel only)	0.812
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	False positives (indel only)	10
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	False positive rate (indel only)	6.64e-8
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	Precision (indel only)	0.804
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	Recall (indel only)	0.82
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	True positives (indel only)	41
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	F1 (SNV only)	0.901
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	False positives (SNV only)	34
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	False positive rate (SNV only)	2.26e-7
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	DeepSomatic-WES 1.7.0 (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	Precision (SNV only)	0.966
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Mutect2, Strelka, Varscan2, at least 2 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	False positives (indel only)	9
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Mutect2, Strelka, Varscan2, at least 2 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	False positive rate (indel only)	5.98e-8
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Mutect2, Strelka, Varscan2, at least 2 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	Precision (indel only)	0.795
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Mutect2, Strelka, Pindel, Varscan2, at least 2 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	F1 (indel only)	0.752
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Mutect2, Strelka, Pindel, Varscan2, at least 2 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	Recall (indel only)	0.76
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Mutect2, Strelka, Pindel, Varscan2, at least 2 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)	True positives (indel only)	38
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Lofreq, Muse, Mutect2, SomaticSniper, Strelka, Lancet, at least 3 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	F1 (SNV only)	0.881
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Lofreq, Muse, Mutect2, SomaticSniper, Strelka, Lancet, at least 3 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	False positives (SNV only)	58
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Lofreq, Muse, Mutect2, SomaticSniper, Strelka, Lancet, at least 3 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	False positive rate (SNV only)	3.85e-7
A benchmarking study of individual somatic variant callers and voting-based ensembles for whole-exome sequencing; Guille et al. 2025, Supplementary Table S7 (validation dataset)	Vote ensemble Lofreq, Muse, Mutect2, SomaticSniper, Strelka, Lancet, at least 3 agree (Guille et al. 2025)	SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7)	Precision (SNV only)	0.943

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-a44eb80c7595e31f · All contested results · How findings are made and followed up