0.898 auroc
lee2026-mutscore-tsg auroc (CGC missense, tumour suppressors (loss of function))
- Tested configuration
- MutScore rank score, dbNSFP 5.3.1a (Lee 2026)
- Protocol
- CGC missense oncogenicity, tumour suppressors (loss of function) (Lee 2026, OncoCal tool_performance.tsv)
- Dataset
- Missense variants in 768 Cancer Gene Census genes with open oncogenicity labels
- Procedure
- somatic-oncogenicity-20261009-protocol-lee2026-tsg
- Evaluation
- MutScore on CGC missense, tumour suppressors (loss of function) (Lee 2026)
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Not reported by the source
- Evidence
- Independent external evaluation · source checkedAn openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers; OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role) · OncoCal tables/tool_performance.tsv at commit 40b7770f2a76, row tool 'MutScore', group 'TSG', column 'AUROC'
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Whole labelled set
- Adaptation
- Not reported
- Scoring implementation
- scikit-learn roc_auc_score and average_precision_score on dbNSFP 5.3.1a rank scores; tools with fewer than 50 scored variants or one class are skipped (src/evaluation/part_b.py, b1_tool_performance, at commit 40b7770f)
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
2 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 0.8976676529684575 Individual claims | An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers OncoCal tables/tool_performance.tsv at commit 40b7770f2a76, row tool 'MutScore', group 'TSG', column 'AUROC' Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: bioRxiv 2026.07.16.739080 v1, posted 2026-07-23; full-text HTML page | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-09T21:06:32Z independent paper Audit detailsExtracted by deterministic parse of the pinned TSV (extract/extract_somatic_oncogenicity.py), with the header, the three group labels and 49 tools per group asserted. printed_value is the field text as written in the file. Independent review 2026-10-09: value, metric, unit, direction, locator and configuration and protocol identity match the source. Field: Source artifact SHA-256: Hash scope: SHA-256 of one retrieval of the full-text HTML page, which is mutable (site chrome, metrics). The PDF request returned HTTP 429. Extraction artifact SHA-256: |
| Reported result 0.8976676529684575 Individual claims | OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role) OncoCal tables/tool_performance.tsv at commit 40b7770f2a76, row tool 'MutScore', group 'TSG', column 'AUROC' Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: commit 40b7770f2a768ba800c4f4c6b48e2bfe967ed14a (2026-07-11) | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-09T21:06:32Z independent paper Audit detailsExtracted by deterministic parse of the pinned TSV (extract/extract_somatic_oncogenicity.py), with the header, the three group labels and 49 tools per group asserted. printed_value is the field text as written in the file. Independent review 2026-10-09: value, metric, unit, direction, locator and configuration and protocol identity match the source. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
Sources and history
Release 2026-10-10-6e93f504adfc · Record review: source checked
2 source records and release history
- An openly licensed benchmark and per-gene calibration map for missense pathogenicity predictors on activating cancer drivers · Original source · bioRxiv 2026.07.16.739080 v1, posted 2026-07-23; full-text HTML page
- OncoCal repository, tables/tool_performance.tsv (per-tool AUROC and AUPRC by gene role) · Original source · commit 40b7770f2a768ba800c4f4c6b48e2bfe967ed14a (2026-07-11)
Technical metadata and extraction receipts
Stable ID: somatic-oncogenicity-20261009-result-lee2026-mutscore-tsg-auroc
- metric
- auroc
- metric direction
- higher
- unit
- unitless
- metric qualifier
- CGC missense, tumour suppressors (loss of function)
- printed value
- 0.8976676529684575
- numeric value
- 0.8976676529684575
- source locator
- OncoCal tables/tool_performance.tsv at commit 40b7770f2a76, row tool 'MutScore', group 'TSG', column 'AUROC'
- review
- method: source-hash-verification; deterministic-table-parse; independent-cell-check; method note: Re-downloaded the TSV from the pinned commit and matched its SHA-256. Split on tabs with a separate script written for this review (the extractor's scripts were not imported or run), asserting the header and 147 rows. Checked printed and numeric value (field text), metric, direction, locator (tool, group, column), and the linked evaluation's configuration, protocol, n and positives. Read the generating code at the pinned commit (src/evaluation/part_b.py).; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-09T21:06:32Z; artifact sha256: 8f5a81078482c8010b83bba6225e9707980a3a7cebbb2ca00ed8b155ad34c3b5; retrieval url: https://raw.githubusercontent.com/tjdrnjsqpf/oncocal/40b7770f2a768ba800c4f4c6b48e2bfe967ed14a/tables/tool_performance.tsv; note: Extracted by deterministic parse of the pinned TSV (extract/extract_somatic_oncogenicity.py), with the header, the three group labels and 49 tools per group asserted. printed_value is the field text as written in the file. Independent review 2026-10-09: value, metric, unit, direction, locator and configuration and protocol identity match the source.
- missing metadata
- uncertainty: reason: unreported