AUROC (zero-shot; SpliceVarDB exonic SNVs) of Evo 2 40B (Evo 2 paper variant effect evaluations) on SpliceVarDB experimentally validated splice variants (Brixi et al. 2026)
0.684 AUROC (zero-shot; SpliceVarDB exonic SNVs)
Methods
- Tested configuration
- Evo 2 40B (Evo 2 paper variant effect evaluations)
- Protocol
- Zero-shot SpliceVarDB exonic SNVs classification (Evo 2 paper, Supplementary Data 1)
- Dataset
- SpliceVarDB experimentally validated splice variants (Brixi et al. 2026)
- Procedure
- Zero-shot SpliceVarDB exonic SNVs classification (Evo 2 paper, Supplementary Data 1)
- Evaluation
- Evo 2 40B on SpliceVarDB exonic SNVs (Evo 2 paper)
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Not reported by the source
- Evidence
- Author-reported evaluation · source checkedGenome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026); Genome modelling and design across all domains of life with Evo 2: Supplementary Information (Nature 2026) · Supplementary Data 1 (sheet 'Sheet1') cell I490, row 490: Model 'Evo 2 40B', Data 'SpliceVarDB', Genomic_Feature 'Exonic', Variant_Type 'SNV', Sub_category '', column 'AUROC'
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- No split; all variants in the stratum scored
- Adaptation
- zero-shot
- Scoring implementation
- Not reported
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
2 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 0.684 Individual claims | Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) Supplementary Data 1 (sheet 'Sheet1') cell I490, row 490: Model 'Evo 2 40B', Data 'SpliceVarDB', Genomic_Feature 'Exonic', Variant_Type 'SNV', Sub_category '', column 'AUROC' Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature 652 (2026), published 2026-03-04; Supplementary Data 1 file 41586_2026_10176_MOESM3_ESM.xlsx | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-14 author reported Audit detailsExtraction: Read sheet 'Sheet1' of the pinned Supplementary Data 1 workbook with a stdlib OOXML reader (extract/ooxml.py); header A1:J1 and 705 data rows asserted; printed_value is the shortest round-trip decimal of the stored cell value; the cell number format is '0.000'. Independent review 2026-10-14: matched. printed_value changed from the full stored decimal (0.683858596480376) to the displayed value under the cell format 0.000, following the store precedent for fixed-format workbook cells; numeric_value keeps the stored decimal. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
| Reported result 0.684 Individual claims | Genome modelling and design across all domains of life with Evo 2: Supplementary Information (Nature 2026) Supplementary Data 1 (sheet 'Sheet1') cell I490, row 490: Model 'Evo 2 40B', Data 'SpliceVarDB', Genomic_Feature 'Exonic', Variant_Type 'SNV', Sub_category '', column 'AUROC' Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature 652 (2026), published 2026-03-04; Supplementary Information file 41586_2026_10176_MOESM1_ESM.pdf (73 pages) | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-14 author reported Audit detailsExtraction: Read sheet 'Sheet1' of the pinned Supplementary Data 1 workbook with a stdlib OOXML reader (extract/ooxml.py); header A1:J1 and 705 data rows asserted; printed_value is the shortest round-trip decimal of the stored cell value; the cell number format is '0.000'. Independent review 2026-10-14: matched. printed_value changed from the full stored decimal (0.683858596480376) to the displayed value under the cell format 0.000, following the store precedent for fixed-format workbook cells; numeric_value keeps the stored decimal. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
3 source records and release history
- Genome modelling and design across all domains of life with Evo 2: Supplementary Data 1, human variant effect prediction results (Nature 2026) · Original source · Nature 652 (2026), published 2026-03-04; Supplementary Data 1 file 41586_2026_10176_MOESM3_ESM.xlsx
- Genome modelling and design across all domains of life with Evo 2: Supplementary Information (Nature 2026) · Original source · Nature 652 (2026), published 2026-03-04; Supplementary Information file 41586_2026_10176_MOESM1_ESM.pdf (73 pages)
- evo2: Journal full-text XML · Original source · Retrieved page snapshot; no immutable publisher revision supplied
Technical metadata and extraction receipts
Stable ID: gfm-guides-20261014-evo2-brixi2026-result-evo-2-40b-splicevardb-exonic-snv-auroc
- metric
- auroc
- metric direction
- higher
- unit
- unitless
- printed value
- 0.684
- numeric value
- 0.683858596480376
- metric qualifier
- zero-shot; SpliceVarDB exonic SNVs
- source locator
- Supplementary Data 1 (sheet 'Sheet1') cell I490, row 490: Model 'Evo 2 40B', Data 'SpliceVarDB', Genomic_Feature 'Exonic', Variant_Type 'SNV', Sub_category '', column 'AUROC'
- raw xml value
- 0.68385859648037595
- workbook number format
- 0.000
- review
- method: source-hash-verification; deterministic-table-parse; independent-cell-check; method note: Workbook re-downloaded from the publisher (SHA-256 matches). An independent openpyxl reader compared all 1,410 results with the cell each locator names (stored value, metric column, model label and configuration, stratum and protocol, class counts in the evaluation population, count arithmetic); every one of the 705 rows and both metric cells was found exactly once. Six cells also checked by hand.; reviewer: claude; reviewer note: Claude (Opus 5.5) research agent, independent of the extractor; no human review claimed; date: 2026-10-14; artifact sha256: 159f321775d634d9ed53b7a81c5cc1b5ef3d407880997e0f8c69bb8315a20a51; retrieval url: https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-026-10176-5/MediaObjects/41586_2026_10176_MOESM3_ESM.xlsx; note: Extraction: Read sheet 'Sheet1' of the pinned Supplementary Data 1 workbook with a stdlib OOXML reader (extract/ooxml.py); header A1:J1 and 705 data rows asserted; printed_value is the shortest round-trip decimal of the stored cell value; the cell number format is '0.000'. Independent review 2026-10-14: matched. printed_value changed from the full stored decimal (0.683858596480376) to the displayed value under the cell format 0.000, following the store precedent for fixed-format workbook cells; numeric_value keeps the stored decimal.
- missing metadata
- uncertainty: reason: unreported
- printed value basis
- Workbook number format 0.000