rewirebio.iobenchmarks
Source

Genome-wide somatic variant calling using localized colored de Bruijn graphs

Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00 · Peer-reviewed

DOI: 10.1038/s42003-018-0023-9

What this source contributed

Records in this release that cite this source. A record can cite several sources.

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Model or configurationMetricValueDatasetEvaluation
LancetF1 (indel only)0.81 fractionNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetF1 (SNV only)0.85 fractionNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetFalse negatives (indel only)1359 countNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetFalse negatives (SNV only)7744 countNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetFalse positives (indel only)305 countNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetFalse positives (SNV only)565 countNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetPrecision (indel only)0.92 fractionNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetPrecision (SNV only)0.98 fractionNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetRecall (indel only)0.72 fractionNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetRecall (SNV only)0.75 fractionNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetTrue positives (indel only)3586 countNA12892/NA12891 virtual tumor and matched normalLancet evaluation
LancetTrue positives (SNV only)23,848 countNA12892/NA12891 virtual tumor and matched normalLancet evaluation
Strelka2F1 (indel only)0.77 fractionNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2F1 (SNV only)0.85 fractionNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2False negatives (indel only)1298 countNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2False negatives (SNV only)7460 countNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2False positives (indel only)867 countNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2False positives (SNV only)1117 countNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2Precision (indel only)0.81 fractionNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2Precision (SNV only)0.96 fractionNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2Recall (indel only)0.73 fractionNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2Recall (SNV only)0.76 fractionNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2True positives (indel only)3647 countNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation
Strelka2True positives (SNV only)24,132 countNA12892/NA12891 virtual tumor and matched normalStrelka2 evaluation

Claims

Statements this source supports, each with where in the source it was checked.

  • Evidence summary: tumour DNA somatic SNV and indel calling: Summary Eight studies are linked, and their values cannot be pooled: truth sets, sequencing, caller versions and scoring scripts differ. Values are rounded to two decimals, except where the source prints percentages. In the Lancet developers' own virtual-tumour test, Lancet and Strelka2 both reached SNV F1 0.85, with indel F1 0.81 and 0.77. Wang et al. 2020 ran eight callers, NeuSomatic and their own majority-vote ensembles on eight whole-genome tumour-normal pairs, with older caller versions (for example MuTect2 from GATK 3.7). An ensemble had the highest SNV F1 on seven of the eight pairs; on DREAM set 4, NeuSomatic, whose model was trained on DREAM set 3, was highest at 0.86. The best single configuration reached SNV F1 0.92 to 0.96 on the simulated DREAM sets 1 to 3 but 0.70 to 0.81 on the CLL, AML and MB tumours, where most callers had precision below 0.6 against curated truth sets that the authors say may be incomplete, so precision there is a lower bound. Indel F1 on the CLL and MB tumours and the COLO829 cell line (AML has no indel truth set) was at most 0.72 for every configuration. Guille et al. 2025, who did not develop the callers, scored them on a WES replicate of the SEQC2 HCC1395 cell line (1,160 truth SNVs, 50 truth indels); another replicate with the same truth set was used to choose their ensembles. Leaving out DeepSomatic and NeuSomatic, whose pre-trained models were built with HCC1395 data, the best single caller reached SNV F1 0.85 (Mutect) and indel F1 0.66 (VarNet), and the authors' ensembles reached 0.88 for SNVs and 0.75 and 0.74 for indels. Strelka reached SNV F1 0.64 there, with precision 0.51, against 0.85 for Strelka2 in the Lancet test, which shows how much results depend on the study. Sahraeian et al. 2022, the NeuSomatic developers, scored callers on SEQC2 HCC1395 whole-genome data in the half of the genome held out from their model training: 21 replicate pairs from six centres, 47 tumour purity and coverage mixtures, and six library preparations. Their NeuSomatic and NeuSomatic-S models trained on SEQC2 data (SEQC-WGS-Spike, SEQC-WGS-GT-50 and SEQC-WGS-GT50-SpikeWGS10) and Octopus-RF, whose random forest is not identified, are left out because they were built from the same cell line or may have been. Among the rest, mean F1 over the 21 replicates was 92.3% for DRAGEN, 91.8% for NeuSomatic with its DREAM3 model and 89.0% for Strelka2 for SNVs, and 81.7%, 79.3% and 72.8% for indels. Over the purity and coverage mixtures (10x to 300x, 5% to 100% tumour), mean SNV F1 was at most 67.7% (Octopus with hard filters). With 5% tumour in an 80x normal, SNV F1 for a pure tumour fell from 88.6% to 43.8% for MuTect2 and from 93.3% to 47.8% for Lancet, but only from 92.4% to 90.9% for Strelka2. Hasan 2026, a single-author preprint run under the author's own company by someone who developed none of the callers, scored Mutect2, Strelka2, DeepSomatic and three ensembles defined by the author on the GIAB HG008 tumour-normal pair against a draft truth set of clonal and truncal variants; the original scoring code was not retained. In the primary region (14,014 truth variants, excluding sites with germline interference), F1 was 0.91 for a two-of-three majority vote, 0.90 for Mutect2 and DeepSomatic and 0.78 for Strelka2; for indels Mutect2 reached 0.82 and DeepSomatic and Strelka2 0.69. Over the whole draft benchmark, F1 was 0.70 to 0.79 for all six. Two developer papers tested tumour-only calling, without the matched normal this use case assumes, so they are linked as proxy evidence. On Illumina COLO829 at 50x, ClairS-TO's developers report SNV F1 0.77 for ClairS-TO against 0.62 for DeepSomatic, 0.58 for Mutect2, 0.40 for Octopus and 0.22 for Pisces, with indel F1 at most 0.29; each F1 is at the threshold that gave that caller its best F1 on the test data, and truth variants with fewer than three supporting reads or VAF below 0.05 in the tested data were not scored. The VarNet-T developers report SNV AUPRC 0.77 for VarNet-T on SEQC2 and 0.66 on COLO829, against 0.54 and 0.27 for ClairS-TO and 0.08 and 0.07 for Mutect2; on three low-burden tumours every tumour-only caller had AUPRC at most 0.11. The two papers used different COLO829 sequencing data and truth sets: the same ClairS-TO model (Illumina SSRS, v0.4.0) had COLO829 SNV AUPRC 0.65 in its developers' paper and 0.27 in the VarNet-T paper. On a synthetic WES pair, where inserted SNVs make precision near 1 by construction, Mutect2 recalled 0.63 of inserted SNVs and Strelka2 0.46 with default nf-core/sarek settings. Only the HG008 preprint reports interval estimates; no linked study covers FFPE tissue, ctDNA or targeted panels, and none establishes clinical performance. (amp-oncology-rna-20261007-source-pmc6123722: Lancet virtual-tumour results; somatic-20261009-source-wang2020-tables: Tables S1-S2; somatic-20261009-source-guille2025-table-s7: Table S7; somatic-20261009-source-wang2020 and somatic-20261009-source-guille2025: Methods, Results and Discussion; somatic-neusomatic-20261010-source-sahraeian2022-additional-file-2: Tables S1-S4; somatic-neusomatic-20261010-source-sahraeian2022: Methods; somatic-clairs-to-20261011-source-chen2025: see claims.csv of its batch; somatic-clairs-to-20261011-source-chen2025-source-data: see claims.csv of its batch; somatic-clairs-to-20261011-source-chen2025-supplementary: see claims.csv of its batch; somatic-varnet-t-20261011-source-krishnamachari2026: see claims.csv of its batch; somatic-varnet-t-20261011-source-krishnamachari2026-source-data: see claims.csv of its batch; somatic-varnet-t-20261011-source-krishnamachari2026-supplementary: see claims.csv of its batch; somatic-hg008-20261011-source-hasan2026: see claims.csv of its batch; somatic-hg008-20261011-source-hasan2026-zenodo: see claims.csv of its batch; somatic-wes-synthetic-20261011-source-lopezcade2025: see claims.csv of its batch)
  • Relevance of Lancet paper virtual-tumor SNV/indel benchmark to "Select a tumour DNA somatic variant-calling workflow": Assessed by Lancet paper virtual-tumor SNV/indel benchmark (amp-oncology-rna-20261007-source-pmc6123722: See clinical/claims.csv and clinical/sources.md in data/omics/use-case-coverage-amp-20261007/)

Models, benchmarks and other records

Configurations (2)

Protocols (1)

Datasets (1)

Evaluations (2)

Use cases (1)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

21 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-adc15377fab2
Property and statementOriginal source and locationReview and provenance
Archive sha256
9600b1ecd13df1daba2390439ad8e4b5b7142db87bbc1b4ad0b99a7cb9ebeb53
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.archive_sha256

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Artifact SHA-256
08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.artifact_sha256

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Artifact url
https://www.ebi.ac.uk/europepmc/webservices/rest/PMC6123722/fullTextXML
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.artifact_url

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Doi
10.1038/s42003-018-0023-9
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.doi

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Licence
https://creativecommons.org/licenses/by/4.0/
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.licence

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Media type
application/xml
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.media_type

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Original artifact sha256
08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.original_artifact_sha256

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Publication status
peer_reviewed
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.publication_status

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Retrieved at
2026-10-07T12:24:27.822425+00:00
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.retrieved_at

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Review artifact format
original_source_artifact
Source metadata
Genome-wide somatic variant calling using localized colored de Bruijn graphs

Original source ↗

No field-specific location recorded

Version: Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Retrieved: 2026-10-07T12:24:27.822425+00:00

catalogued

No individual claim review recorded

Audit details

Field: attributes.review_artifact_format

Source artifact SHA-256: 08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-adc15377fab2 · Record review: source checked

Primary source for the AMP oncology-RNA bounded intake. Automated review, not independent reproduction.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)
Technical metadata and extraction receipts

Stable ID: amp-oncology-rna-20261007-source-pmc6123722

archive sha256
9600b1ecd13df1daba2390439ad8e4b5b7142db87bbc1b4ad0b99a7cb9ebeb53
artifact sha256
08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5
artifact url
https://www.ebi.ac.uk/europepmc/webservices/rest/PMC6123722/fullTextXML
doi
10.1038/s42003-018-0023-9
licence
https://creativecommons.org/licenses/by/4.0/
media type
application/xml
original artifact sha256
08fa271d01e0583f570c97dd0d260073caf8f48877fa9f6d6eabc858b6e518d5
publication status
peer_reviewed
retrieved at
2026-10-07T12:24:27.822425+00:00
review artifact
data/omics/amp-coverage-20261007/oncology-rna/artifacts/PMC6123722.xml.gz
review artifact format
original_source_artifact
review artifact matches original artifact sha256
true
url
https://doi.org/10.1038/s42003-018-0023-9
version
Europe PMC fullTextXML retrieved 2026-10-07T12:24:27.822425+00:00
Related records

    Suggest a correction