Evaluation

ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

Score all 2,972 variants in AMFR_HUMAN_Tsuboyama_2023_4G3O with ESM-2 8M masked marginals. This single-assay result is not a full ProteinGym track or suite score.

Research readiness

These checks assess whether the evidence supports a reproducible investigation. A source-checked score alone does not meet these requirements.

Release 2026-09-29-06401fd5b220 · Evidence verified: 2026-09-23

Evidence complete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.
proteingym-amfr-discrepancies: metric_replay: Saved predictions reproduce recorded metrics within 1e-9 absolute tolerance.

Verified: 2026-09-23

Evidence complete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.
proteingym-amfr-discrepancies: metric_replay: Saved predictions reproduce recorded metrics within 1e-9 absolute tolerance.
proteingym-amfr-discrepancies: annotations: Registered subgroup fields derive from the preserved source snapshot.
proteingym-amfr-discrepancies: dependence: Independent sampling unit unknown; descriptive analysis only.

Verified: 2026-09-23

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

recipe pinned: verification is missing
resource estimate: verification is missing
No pinned local execution recipe is declared.
Local execution requires exactly one pinned recipe code artifact.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.

Verified: 2026-09-23

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

independent validation: verification is missing
overlap checked: verification is missing
Data has already been exposed during hypothesis selection; it is not untouched validation.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.

Verified: 2026-09-23

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Limitations

Existing test outcomes are exposed; no independent validation is claimed.
Independent experimental grouping is unavailable; subgroup summaries are descriptive.
Prepared opaque IDs must not be joined to a newly prepared snapshot by ID.
One protein construct, complete selected assay but partial benchmark suite.
Official three-decimal metrics and score sign are preserved; no post-hoc sign reversal.
Upstream archive checksums were observed locally, not independently source-verified.
The sample independence unit is unresolved; descriptive analysis does not establish confirmatory inference.
Evidence is exploratory because this data has already been examined.

Artifacts and reproduction

One or more artifacts need a local resolver. Their checksums are recorded, but this release does not provide every download.

ProteinGym AMFR: negative correlation

Does the negative association differ between single and multiple substitutions?

Manifest: proteingym-amfr-discrepancies
Prepared outcome: DMS stability
Outcome type: continuous
Units: assay DMS score
Score direction: higher
Join identifier: id
Unit of independence: Not reported
Split: test
Prior data exposure: Already used for exploration; not untouched validation

Evaluation protocol and run instructions

Resolve each required artifact from its recorded source, verify its SHA-256, and retain the prepared snapshot and identifiers. A missing public URL means this release does not redistribute that file.

proteingym-amfr-discrepancies-prepared

prepared · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: d9e220a0e2b0675e73f4b79b4ae45b8f807dedbb886bbd4b2f8e886822ddf57f

Semantic SHA-256: 9f0e0cce34942797364990ef4dc9fb8b382f7b2fd59e6c72de23efc2992fb148 (separate from the file checksum)

proteingym-amfr-discrepancies-proteingym-esm2-predictions

proteingym-esm2-predictions · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 7ab0c8901406d5c1ab759038c8beef235721938394cfdf192c46671e98e0bbca

proteingym-amfr-discrepancies-proteingym-esm2-report

proteingym-esm2-report · json

Artifact source

File SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

proteingym-amfr-discrepancies-proteingym-esm2-source

proteingym-esm2-source · json

Artifact source

File SHA-256: 90888f1f18625cdc2e9fb0bf7ecc1b1c4c2d6b700e8b4971e4349297926f996f

proteingym-amfr-discrepancies-table

table · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 4fac11aacc0f00a70e385ed7b8d73427aaa0716906a74e93bc3d48df5ff43f9f

proteingym-amfr-discrepancies-receipt

receipt · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 9f117c65a3c860b83735d4874d06a807ee1a1523464e9afca493e7dcd9e59b56

proteingym-amfr-discrepancies-preparation_code

preparation code · py

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 8a1d691de8fdd9fcf8242fe6fa2cfe2703a1c3281b880fe2a226cb8bd946fd1b

proteingym-amfr-discrepancies-environment

environment · lock

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 748257abb3664595db1ebf892799e91c6f1a4ca79b0e65eccb3958cf5644a671

Registered recipes and verification receipt

Local recipes: None recorded
Verification: verified at: 2026-09-23T23:00:14.349323+00:00; checks: check: artifact_hashes; status: passed; detail: Exact local bytes recorded; source pins checked where available.; check: join_integrity; status: passed; detail: 2972 unique test IDs; scores and outcomes reconcile.; check: score_semantics; status: passed; detail: Higher scores predict higher DMS stability; metrics retain their own direction.; check: metric_replay; status: passed; detail: Saved predictions reproduce recorded metrics within 1e-9 absolute tolerance.; check: annotations; status: passed; detail: Registered subgroup fields derive from the preserved source snapshot.; check: dependence; status: passed; detail: Independent sampling unit unknown; descriptive analysis only.; limitations: Existing test outcomes are exposed; no independent validation is claimed.; Independent experimental grouping is unavailable; subgroup summaries are descriptive.; Prepared opaque IDs must not be joined to a newly prepared snapshot by ID.; One protein construct, complete selected assay but partial benchmark suite.; Official three-decimal metrics and score sign are preserved; no post-hoc sign reversal.; Upstream archive checksums were observed locally, not independently source-verified.

Read reviewed discrepancy investigations

Evaluation results

1 evaluation · 5 results. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	0.394 AUC dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.AUC
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	-0.139 MCC dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.MCC
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	0.44 NDCG dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.NDCG
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	-0.209 Spearman dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.Spearman
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	0.057 Top_recall dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.Top_recall

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Evaluation procedure

ProteinGym v1.3 AMFR substitution assay

Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)
Protocol: ProteinGym v1.3 AMFR substitution assay
Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay

origin: Rewire evaluation
configuration: 1.3
protocol id: rewire-protocol-proteingym-amfr-v13
dataset version: rewire-dataset-proteingym-amfr-v13
split: Full assay; no train/test split
subset: proteingym-dms-substitutions-v1.3
population: 2972/2972
inputs: Wild-type protein sequence and amino-acid substitutions; no MSA, structure or assay labels
adaptation: zero-shot masked marginals
metric implementation: fcc5ab65e6753cf29aabbdae231cbb5cdf72a3dd40ba8f109802c33223b33536
aggregation: single assay
budget: one local CPU evaluation; no hyperparameter search outside training

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split: Full assay; no train/test split
Adaptation: zero-shot masked marginals
Scoring implementation: fcc5ab65e6753cf29aabbdae231cbb5cdf72a3dd40ba8f109802c33223b33536

Generate and evaluate predictions

This committed script regenerates the selected AMFR assay using the recorded checkpoint and masked-marginal scorer. It is not the complete ProteinGym track or a reproduction of a paper score.

ProteinGym v1.3 AMFR substitution assay: reproduction instructions · Reproduce section

Access requirements and official instructions: ProteinGym v1.3 AMFR substitution assay

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

152 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
attributes.comparison.adaptation zero-shot masked marginals Context-only references	ProteinGym v1.3 AMFR substitution assay: reproduction instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `5acb7b2769aae657d48d23c282319e7c69b8ed0424b3c02c875637df7f8dd9fb` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.adaptation zero-shot masked marginals Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation single assay Context-only references	ProteinGym v1.3 AMFR substitution assay: reproduction instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `5acb7b2769aae657d48d23c282319e7c69b8ed0424b3c02c875637df7f8dd9fb` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation single assay Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no hyperparameter search outside training Context-only references	ProteinGym v1.3 AMFR substitution assay: reproduction instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `5acb7b2769aae657d48d23c282319e7c69b8ed0424b3c02c875637df7f8dd9fb` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no hyperparameter search outside training Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.dataset_version rewire-dataset-proteingym-amfr-v13 Context-only references	ProteinGym v1.3 AMFR substitution assay: reproduction instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.dataset_version` Source artifact SHA-256: `5acb7b2769aae657d48d23c282319e7c69b8ed0424b3c02c875637df7f8dd9fb` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.dataset_version rewire-dataset-proteingym-amfr-v13 Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.dataset_version` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.inputs Wild-type protein sequence and amino-acid substitutions; no MSA, structure or assay labels Context-only references	ProteinGym v1.3 AMFR substitution assay: reproduction instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.inputs` Source artifact SHA-256: `5acb7b2769aae657d48d23c282319e7c69b8ed0424b3c02c875637df7f8dd9fb` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.inputs Wild-type protein sequence and amino-acid substitutions; no MSA, structure or assay labels Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.inputs` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: source checked

2 source records and release history

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) · Original source · ca73fa47136d182f2d4ddb083d084712198fc0e2
ProteinGym v1.3 AMFR substitution assay: reproduction instructions · Original source · ca73fa47136d182f2d4ddb083d084712198fc0e2

Download this release

Technical metadata and extraction receipts

Stable ID: rewire-local-20260920-evaluation-proteingym-esm2

areas: proteins-complexes
origin: rewire_run
execution scope: complete_selected_evaluation
published score reproduction: false
protocol: ProteinGym v1.3 AMFR substitution assay
version: 1.3
source locator: metrics; coverage; protocol_results
original sdk scope: subset
original sdk completion: partial
eligible count: 2972
scored count: 2972
missing count: 0
coverage: 2972/2972
suite complete: false
comparison: protocol id: rewire-protocol-proteingym-amfr-v13; dataset version: rewire-dataset-proteingym-amfr-v13; split: Full assay; no train/test split; subset: proteingym-dms-substitutions-v1.3; population: 2972/2972; inputs: Wild-type protein sequence and amino-acid substitutions; no MSA, structure or assay labels; adaptation: zero-shot masked marginals; metric implementation: fcc5ab65e6753cf29aabbdae231cbb5cdf72a3dd40ba8f109802c33223b33536; aggregation: single assay; budget: one local CPU evaluation; no hyperparameter search outside training
input information: Wild-type protein sequence and amino-acid substitutions; no MSA, structure or assay labels
model configuration: checkpoint: esm2_t6_8M_UR50D; device: cpu; seed: 0; strategy: masked_marginals; threads: 1
provenance: assay sha256: AMFR HUMAN Tsuboyama 2023 4G3O: dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a; data verification: local_bytes_hashed_not_independently_source_verified; expected hashes checked: AMFR_HUMAN_Tsuboyama_2023_4G3O; reference sha256: a8f498011532a74aa9fe556a50555a75e928c5837d19c06a87592ae04049b308; source url: https://github.com/OATML-Markslab/ProteinGym/tree/144fe22b07dfaeec2b366f2346203a9838a55b4c; upstream revision: 144fe22b07dfaeec2b366f2346203a9838a55b4c
predictions sha256: 7ab0c8901406d5c1ab759038c8beef235721938394cfdf192c46671e98e0bbca
prepared sha256: 9f0e0cce34942797364990ef4dc9fb8b382f7b2fd59e6c72de23efc2992fb148
execution: adapter: ESM2Adapter; adapter provenance: checkpoint sha256: 46f002a9870c9bdecd0ea887acb1f9a38a6b561e8f8bf8a6990b679b9d31b928; checkpoint url: https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t6_8M_UR50D.pt; device: cpu; fair esm version: 2.0.0; implementation sha256: 4ff38cfbb91d718b52a44da53aa5594fe73d9d1d4d0ce43724807597c3929889; maximum sequence length: 1022; model: esm2_t6_8M_UR50D; published result reproduction: false; strategy: masked_marginals_wild_type_context_sum_log_odds; training overlap: unreported; UniRef50 pretraining may overlap benchmark proteins; batch size: 32; fitting: none; inference and fit seconds: 0.18779424999956973; mode: local_adapter; prediction type: scalar
timing seconds: evaluation: 0.03425099999731174
environment: architecture: arm64; container digest: unreported; container runtime: unreported; platform: macOS-26.6.2-arm64-arm-64bit; python: 3.11.13; rewirebench: 0.4.0; sdk code sha256: fcc5ab65e6753cf29aabbdae231cbb5cdf72a3dd40ba8f109802c33223b33536; sif sha256: unreported
run url: https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/proteingym-esm2/report.json
reproduction url: https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/README.md
limitations: One selected split or assay; no whole-suite score.; No published model score reproduction claim.; Raw predictions retained locally; hashes and inspected contribution bundle are public.; Single execution; no seed variability or uncertainty interval estimated.
reproduction: recipe owner id: rewire-protocol-proteingym-amfr-v13; recipe id: local-20260920-proteingym-repeat; applicability: generate_and_evaluate; explanation: This committed script regenerates the selected AMFR assay using the recorded checkpoint and masked-marginal scorer. It is not the complete ProteinGym track or a reproduction of a paper score.; source ids: rewire-local-20260920-instructions-proteingym; source locator: Reproduce section

Related records

configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)
protocol: ProteinGym v1.3 AMFR substitution assay
dataset subset: ProteinGym v1.3 AMFR substitution assay full assay
evaluation: ESM-2 8M masked-marginal scoring: AUC
evaluation: ESM-2 8M masked-marginal scoring: MCC
evaluation: ESM-2 8M masked-marginal scoring: NDCG
evaluation: ESM-2 8M masked-marginal scoring: Spearman
evaluation: ESM-2 8M masked-marginal scoring: Top_recall

Suggest a correction