Source

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026)

ca73fa47136d182f2d4ddb083d084712198fc0e2

What this source contributed

Records in this release that cite this source. A record can cite several sources.

1 configuration
1 task
2 protocols
1 dataset subset
1 evaluation
5 results
4 evidence claims
1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Model or configuration	Metric	Value	Dataset	Evaluation
ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	AUROC	0.394 unitless	ProteinGym v1.3 AMFR substitution assay full assay	ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay
ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	MCC	-0.139 unitless	ProteinGym v1.3 AMFR substitution assay full assay	ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay
ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	NDCG at 10%	0.440 unitless	ProteinGym v1.3 AMFR substitution assay full assay	ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay
ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Spearman	-0.209 unitless	ProteinGym v1.3 AMFR substitution assay full assay	ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay
ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Top 10% recall	0.057 unitless	ProteinGym v1.3 AMFR substitution assay full assay	ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

Claims

Statements this source supports, each with where in the source it was checked.

ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay): family ESM-2: Family ESM-2 (report.json: model.checkpoint and artifacts checkpoint_url/checkpoint_sha256; esm2_t6_8M_UR50D)
Relevance of ProteinGym v1.3 AMFR substitution assay to "Assess methods for protein stability experiments": Assessed by ProteinGym v1.3 AMFR substitution assay (rewire-local-20260920-source-proteingym-esm2: /metrics; /coverage; /execution/adapter_provenance; /execution; /input_information; /provenance; /protocol_results; rewire-local-20260920-instructions-proteingym: Data and method; Reproduce; profile-protocol-proteingym-reference-files-dms-substitutions-csv-a8f49801: AMFR_HUMAN_Tsuboyama_2023_4G3O row: seq_len, selection_assay, raw_DMS_phenotype_name and mutant-count columns; use-case-source-amfr-pilot-plan-194a78b: Question; Task and data; Methods; Gates and execution order; Runtime and cost)
Selected concrete evaluation protocol belongs to this benchmark; no suite-wide score is implied.: Part of ProteinGym (protocol_id; protocol_configuration; provenance)
The executed esm2_t6_8M_UR50D checkpoint is an ESM-2 family member; only this exact scoring configuration is evaluated.: Family ESM-2 (model; model_configuration; execution)

Models, benchmarks and other records

Configurations (1)

ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)

Tasks (1)

ProteinGym mutation effects

Protocols (2)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

15 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-02-a7f03c9e55a5
Property and statement	Original source and location	Review and provenance
Artifact SHA-256 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26 Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_sha256` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Artifact url https://raw.githubusercontent.com/rewire-bio/rewire-benchmarks/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/proteingym-esm2/report.json Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_url` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Evidence origin rewire_run Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.evidence_origin` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieval note Generated local execution report, verified against saved predictions; immutable public Git source. Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.retrieval_note` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieved at 2026-09-20T20:54:03Z Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.retrieved_at` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Source type rewire_execution_report Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.source_type` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Url https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/proteingym-esm2/report.json Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.url` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Version ca73fa47136d182f2d4ddb083d084712198fc0e2 Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `attributes.version` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Description Public sanitized execution evidence. Raw sequences and predictions remain local. Automated checking does not establish reproduction of a published model score. Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `description` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Facets: areas 1 values proteins-complexes Source metadata	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	catalogued No individual claim review recorded Audit details Field: `facets.areas` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-02-a7f03c9e55a5 · Record review: source checked

Public sanitized execution evidence. Raw sequences and predictions remain local. Automated checking does not establish reproduction of a published model score.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: rewire-local-20260920-source-proteingym-esm2

areas: proteins-complexes
url: https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/proteingym-esm2/report.json
artifact url: https://raw.githubusercontent.com/rewire-bio/rewire-benchmarks/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/proteingym-esm2/report.json
artifact sha256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26
version: ca73fa47136d182f2d4ddb083d084712198fc0e2
retrieved at: 2026-09-20T20:54:03Z
source type: rewire_execution_report
evidence origin: rewire_run
retrieval note: Generated local execution report, verified against saved predictions; immutable public Git source.

Related records

Suggest a correction