AUROC (ROC-AUC, pathogenic versus benign ClinVar deletion-insertions; zero-shot) of ESM1b absolute-valued PLLR indel score (Brandes et al. 2023) on ClinVar in-frame deletion-insertions (Brandes et al. 2023)
0.887 AUROC (ROC-AUC, pathogenic versus benign ClinVar deletion-insertions; zero-shot)
Methods
- Tested configuration
- ESM1b absolute-valued PLLR indel score (Brandes et al. 2023)
- Protocol
- ClinVar in-frame indel pathogenicity, ROC-AUC (Brandes et al. 2023 Fig. 6b, Supplementary Table 2)
- Dataset subset
- ClinVar in-frame deletion-insertions (Brandes et al. 2023)
- Procedure
- ClinVar in-frame indel pathogenicity, ROC-AUC (Brandes et al. 2023 Fig. 6b, Supplementary Table 2)
- Evaluation
- ESM1b-aPLLR on deletion-insertions (Brandes et al. 2023)
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Not reported by the source
- Evidence
- Author-reported evaluation · source checkedBrandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Supplementary Table 2 sheet 'Table_of_results_indels' cell E2, row 'ESM1b-aPLLR', column 'AUC_delins'
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Whole labelled set
- Adaptation
- zero-shot
- Scoring implementation
- ROC-AUC
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 0.887359 Individual claims | Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods Supplementary Table 2 sheet 'Table_of_results_indels' cell E2, row 'ESM1b-aPLLR', column 'AUC_delins' Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-14 author reported Audit detailsExtraction: Read the pinned workbook with the stdlib OOXML reader in extract/build_supp2.py; header row and the 7 method labels in column A asserted; 'NaN' cells skipped. Independent review 2026-10-14: matched. The cells use the General format, so printed_value keeps the stored decimal. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-supplementary-table-2.xlsx.gz Extraction artifact SHA-256: |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
2 source records and release history
- Brandes et al. 2023 Supplementary Table 2: full benchmark evaluations of ESM1b and all other methods · Original source · Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10)
- Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) · Original source · Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10)
Technical metadata and extraction receipts
Stable ID: gfm-guides-20261014-esm1b-result-clinvar-inframe-indels-delins-esm1b-apllr-auroc
- printed value
- 0.887359
- numeric value
- 0.887359
- metric
- auroc
- metric direction
- higher
- unit
- unitless
- metric qualifier
- ROC-AUC, pathogenic versus benign ClinVar deletion-insertions; zero-shot
- source locator
- Supplementary Table 2 sheet 'Table_of_results_indels' cell E2, row 'ESM1b-aPLLR', column 'AUC_delins'
- raw xml value
- 0.887359
- printed value basis
- Shortest round-trip decimal of the number stored in the workbook cell
- missing metadata
- uncertainty: reason: unreported
- review
- method: source-hash-verification; deterministic-table-parse; independent-cell-check; method note: Workbook re-downloaded from the publisher (SHA-256 matches). An independent openpyxl reader matched all 386 results with the cell each locator names (stored value, header, row label, metric) and checked, for head-to-head cells, the system (ESM1b or the comparator whose reported name is the row label), origin, the included flag and note against the comparator configuration, and head2head_n_variants against the subset's scored_count; every numeric cell in the head-to-head columns and the indel and stop-gain sheets was found exactly once. Six cells also checked by hand.; reviewer: claude; reviewer note: Claude (Opus 5.5) research agent, independent of the extractor; no human review claimed; date: 2026-10-14; artifact sha256: 093c32e4156a801c0dd809a437c87818a4f1d04df92e037cf40222471a3e72e2; retrieval url: https://static-content.springer.com/esm/art%3A10.1038%2Fs41588-023-01465-0/MediaObjects/41588_2023_1465_MOESM3_ESM.xlsx; note: Extraction: Read the pinned workbook with the stdlib OOXML reader in extract/build_supp2.py; header row and the 7 method labels in column A asserted; 'NaN' cells skipped. Independent review 2026-10-14: matched. The cells use the General format, so printed_value keeps the stored decimal.