SEQC2 HCC1395 WES validation sample (Guille et al. 2025)
Direct evidenceDeveloper-reported: Reported by the developers of the evaluated tool.Independent study: Reported in a study whose authors did not develop the evaluated tool.F1 (higher is better), with the other metrics below each value. Showing the top 10 of 17 rows. The best value in each column is highlighted.
| Tool | SNVs | Indels |
|---|---|---|
| 88.1% (best in column) | Not reported | |
Mutectconventional | 85.3% | Not reported |
Lofreqconventional | 84.7% | 32.65% |
VarNetmachine learning | 84.0% | 65.88% |
Museconventional | 84.0% | Not reported |
Virmidconventional | 83.5% | Not reported |
| 83.3% | 65.04% | |
Seuratconventional | 77.2% | 15.30% |
Vardictconventional | 75.4% | 20.43% |
Varscan2conventional | 73.3% | 41.43% |
SomaticSniperconventional | 71.7% | Not reported |
Lancetconventional | 65.9% | 46.25% |
| 63.9% | 53.66% | |
FreeBayesconventional | 59.6% | 17.82% |
Shimmerconventional | 33.1% | Not reported |
Pindelconventional | Not reported | 46.15% |
Scalpelconventional | Not reported | 5.30% |
Show all 17 rowsShow the top 10 rows only
R recall, P precision.
Caveats and method
What each column measures:
- SNVs: TP, FP, recall, false-positive rate, precision and F1 for somatic SNVs on a held-out SEQC2 HCC1395 WES sample for 16 individual callers and 2 retained voting ensembles
- Indels: TP, FP, recall, false-positive rate, precision and F1 for somatic indels on a held-out SEQC2 HCC1395 WES sample for 13 individual callers and 2 retained voting ensembles
- Single WES sample of one cell line; no replicate or interval estimates.
- Truth limited to SEQC2 high-confidence regions within the exome (1,160 SNVs).
- Not fully independent of development: the ensembles were selected on four datasets that include another WES replicate of the same HCC1395 pair with the same truth set (Table 2 row 'SEQC2').
- Post-alignment procedure for Table S7 is not stated (Tables S1-S2 compare four procedures on the development datasets).
- DeepSomatic and NeuSomatic pre-trained models were built with SEQC2 HCC1395 data (Discussion paragraph 5; supplementary methods checkpoint name), so their rows are not held out.
- The authors' 'best individual tool' comparison in Results 'Validation' covers the classic callers only (gains of 2.7 and 10.2 points over Mutect and Mutect2); it excludes the deep-learning callers, as the ensemble search did (Methods 'Ensemble approach').
- Small truth set: 50 indels in SEQC2 high-confidence regions, so one variant moves recall by 0.02.
Protocols: SEQC2 HCC1395 WES validation sample, somatic SNVs (Guille et al. 2025 Table S7), SEQC2 HCC1395 WES validation sample, somatic indels (Guille et al. 2025 Table S7)