Source

EVEE manuscript repository: scripts/figure1/_heatmap_common.py

goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f · Project source

What this source contributed

Records in this release that cite this source. A record can cite several sources.

1 configuration
1 protocol
8 evaluations
8 results

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Value	Metric	Model or configuration	Dataset	Evaluation
0.574 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	AlphaGenome all-folds composite score (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	AlphaGenome on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.972 unitless	AUROC (Deconfounded ClinVar SNVs, missense only (value plotted in the Overall row))	AlphaMissense hg38, pre-computed (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	AlphaMissense on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.963 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	CADD v1.7 PHRED, pre-computed (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	CADD v1.7 on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.977 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	EVEE covariance probe on Evo 2 7B layer 27 (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	Evo2 Covariance on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.929 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	Evo 2 7B loss-based zero-shot score (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	Evo2 Loss on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.967 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	Evo 2 7B mean-pooled embedding probe (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	Evo2 Mean on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.952 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	GPN-MSA hg38 scores, pre-computed (Pearce et al. 2026)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	GPN-MSA on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)
0.710 unitless	AUROC (Deconfounded ClinVar SNVs, all consequence types (Overall row))	NTv3 cosine distance (Pearce et al. 2026 Figure 1b and 1c)	EVEE deconfounded ClinVar SNV benchmark (158,616 variants)	NTv3 on the EVEE deconfounded ClinVar SNV benchmark (Pearce et al. 2026)

Models, benchmarks and other records

Configurations (1)

NTv3 cosine distance (Pearce et al. 2026 Figure 1b and 1c)

Protocols (1)

EVEE deconfounded ClinVar SNV pathogenicity AUROC (Pearce et al. 2026 Figure 1c)

Evaluations (8)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

17 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
attributes.artifact_sha256 1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_sha256` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Artifact url https://raw.githubusercontent.com/goodfire-ai/evee-manuscript/e5e4a43d60bb217819144c09ab509ed44f49cd5f/scripts/figure1/_heatmap_common.py Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_url` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Media type text/x-python Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.media_type` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Missing metadata: licence: note No licence file checked in this pass Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.missing_metadata.licence.note` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Missing metadata: licence: reason unreported Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.missing_metadata.licence.reason` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Publication status official_project_source Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.publication_status` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieved at 2026-10-14T12:48:26Z Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.retrieved_at` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Url https://github.com/goodfire-ai/evee-manuscript/tree/e5e4a43d60bb217819144c09ab509ed44f49cd5f Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.url` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Version goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `attributes.version` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Description Plotting code for Figure 1b and 1c: column order, method renaming, dropped methods and the AlphaMissense Overall cell. Source metadata	EVEE manuscript repository: scripts/figure1/_heatmap_common.py Original source ↗ No field-specific location recorded Version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f Retrieved: 2026-10-14T12:48:26Z	catalogued No individual claim review recorded Audit details Field: `description` Source artifact SHA-256: `1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Plotting code for Figure 1b and 1c: column order, method renaming, dropped methods and the AlphaMissense Overall cell.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: gfm-guide-20261014-source-evee-repo-heatmap-common

areas: dna-genomes
contexts: research
artifact url: https://raw.githubusercontent.com/goodfire-ai/evee-manuscript/e5e4a43d60bb217819144c09ab509ed44f49cd5f/scripts/figure1/_heatmap_common.py
retrieved at: 2026-10-14T12:48:26Z
artifact sha256: 1d477d2e1eb4213591015b26baca654e5ca1ca19ee9a1fd39e5f106fc8e56e2e
publication status: official_project_source
url: https://github.com/goodfire-ai/evee-manuscript/tree/e5e4a43d60bb217819144c09ab509ed44f49cd5f
version: goodfire-ai/evee-manuscript commit e5e4a43d60bb217819144c09ab509ed44f49cd5f
media type: text/x-python
missing metadata: licence: reason: unreported; note: No licence file checked in this pass

Related records

Suggest a correction