Dataset subset

ProteinGym v1.3 AMFR substitution assay full assay

Score all 2,972 variants in AMFR_HUMAN_Tsuboyama_2023_4G3O with ESM-2 8M masked marginals. This single-assay result is not a full ProteinGym track or suite score.

Research readiness

These checks assess whether the evidence supports a reproducible investigation. A source-checked score alone does not meet these requirements.

Release 2026-09-29-06401fd5b220 · Evidence verified: 2026-09-23

Evidence complete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.
proteingym-amfr-discrepancies: metric_replay: Saved predictions reproduce recorded metrics within 1e-9 absolute tolerance.

Verified: 2026-09-23

Evidence complete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.
proteingym-amfr-discrepancies: metric_replay: Saved predictions reproduce recorded metrics within 1e-9 absolute tolerance.
proteingym-amfr-discrepancies: annotations: Registered subgroup fields derive from the preserved source snapshot.
proteingym-amfr-discrepancies: dependence: Independent sampling unit unknown; descriptive analysis only.

Verified: 2026-09-23

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

recipe pinned: verification is missing
resource estimate: verification is missing
No pinned local execution recipe is declared.
Local execution requires exactly one pinned recipe code artifact.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.

Verified: 2026-09-23

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

independent validation: verification is missing
overlap checked: verification is missing
Data has already been exposed during hypothesis selection; it is not untouched validation.

Supporting evidence

proteingym-amfr-discrepancies: artifact_hashes: Exact local bytes recorded; source pins checked where available.
proteingym-amfr-discrepancies: join_integrity: 2972 unique test IDs; scores and outcomes reconcile.
proteingym-amfr-discrepancies: score_semantics: Higher scores predict higher DMS stability; metrics retain their own direction.

Verified: 2026-09-23

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Limitations

Existing test outcomes are exposed; no independent validation is claimed.
Independent experimental grouping is unavailable; subgroup summaries are descriptive.
Prepared opaque IDs must not be joined to a newly prepared snapshot by ID.
One protein construct, complete selected assay but partial benchmark suite.
Official three-decimal metrics and score sign are preserved; no post-hoc sign reversal.
Upstream archive checksums were observed locally, not independently source-verified.
The sample independence unit is unresolved; descriptive analysis does not establish confirmatory inference.
Evidence is exploratory because this data has already been examined.

Artifacts and reproduction

One or more artifacts need a local resolver. Their checksums are recorded, but this release does not provide every download.

ProteinGym AMFR: negative correlation

Does the negative association differ between single and multiple substitutions?

Manifest: proteingym-amfr-discrepancies
Prepared outcome: DMS stability
Outcome type: continuous
Units: assay DMS score
Score direction: higher
Join identifier: id
Unit of independence: Not reported
Split: test
Prior data exposure: Already used for exploration; not untouched validation

Evaluation protocol and run instructions

Resolve each required artifact from its recorded source, verify its SHA-256, and retain the prepared snapshot and identifiers. A missing public URL means this release does not redistribute that file.

proteingym-amfr-discrepancies-prepared

prepared · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: d9e220a0e2b0675e73f4b79b4ae45b8f807dedbb886bbd4b2f8e886822ddf57f

Semantic SHA-256: 9f0e0cce34942797364990ef4dc9fb8b382f7b2fd59e6c72de23efc2992fb148 (separate from the file checksum)

proteingym-amfr-discrepancies-proteingym-esm2-predictions

proteingym-esm2-predictions · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 7ab0c8901406d5c1ab759038c8beef235721938394cfdf192c46671e98e0bbca

proteingym-amfr-discrepancies-proteingym-esm2-report

proteingym-esm2-report · json

Artifact source

File SHA-256: 6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26

proteingym-amfr-discrepancies-proteingym-esm2-source

proteingym-esm2-source · json

Artifact source

File SHA-256: 90888f1f18625cdc2e9fb0bf7ecc1b1c4c2d6b700e8b4971e4349297926f996f

proteingym-amfr-discrepancies-table

table · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 4fac11aacc0f00a70e385ed7b8d73427aaa0716906a74e93bc3d48df5ff43f9f

proteingym-amfr-discrepancies-receipt

receipt · json

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 9f117c65a3c860b83735d4874d06a807ee1a1523464e9afca493e7dcd9e59b56

proteingym-amfr-discrepancies-preparation_code

preparation code · py

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 8a1d691de8fdd9fcf8242fe6fa2cfe2703a1c3281b880fe2a226cb8bd946fd1b

proteingym-amfr-discrepancies-environment

environment · lock

No public download in this release. Obtain access using the documented source and terms.

File SHA-256: 748257abb3664595db1ebf892799e91c6f1a4ca79b0e65eccb3958cf5644a671

Registered recipes and verification receipt

Local recipes: None recorded
Verification: verified at: 2026-09-23T23:00:14.349323+00:00; checks: check: artifact_hashes; status: passed; detail: Exact local bytes recorded; source pins checked where available.; check: join_integrity; status: passed; detail: 2972 unique test IDs; scores and outcomes reconcile.; check: score_semantics; status: passed; detail: Higher scores predict higher DMS stability; metrics retain their own direction.; check: metric_replay; status: passed; detail: Saved predictions reproduce recorded metrics within 1e-9 absolute tolerance.; check: annotations; status: passed; detail: Registered subgroup fields derive from the preserved source snapshot.; check: dependence; status: passed; detail: Independent sampling unit unknown; descriptive analysis only.; limitations: Existing test outcomes are exposed; no independent validation is claimed.; Independent experimental grouping is unavailable; subgroup summaries are descriptive.; Prepared opaque IDs must not be joined to a newly prepared snapshot by ID.; One protein construct, complete selected assay but partial benchmark suite.; Official three-decimal metrics and score sign are preserved; no post-hoc sign reversal.; Upstream archive checksums were observed locally, not independently source-verified.

Read reviewed discrepancy investigations

Evaluation results

1 evaluation · 5 results. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	0.394 AUC dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.AUC
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	-0.139 MCC dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.MCC
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	0.44 NDCG dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.NDCG
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	-0.209 Spearman dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.Spearman
Configuration: ESM-2 8M masked-marginal scoring (ProteinGym v1.3 AMFR substitution assay)	Protocol: ProteinGym v1.3 AMFR substitution assay Dataset subset: ProteinGym v1.3 AMFR substitution assay full assay	0.057 Top_recall dimensionless · higher Uncertainty: Not reported Coverage: 2972/2972	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay ProteinGym v1.3 AMFR substitution assay Aggregation: Not reported ESM-2 8M masked-marginal scoring: local execution report (20 September 2026); ESM-2 8M masked-marginal scoring: automated execution audit · protocol_results.per_assay.AMFR_HUMAN_Tsuboyama_2023_4G3O.metrics.Top_recall

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Subset and evaluation context

This record describes a particular subset or cohort used in an evaluation. Its results do not describe the full dataset.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

22 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
attributes.dataset_id proteingym-dms-substitutions-v1.3 Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.dataset_id` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.provenance.assay_sha256.AMFR_HUMAN_Tsuboyama_2023_4G3O dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.provenance.assay_sha256.AMFR_HUMAN_Tsuboyama_2023_4G3O` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.provenance.data_verification local_bytes_hashed_not_independently_source_verified Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.provenance.data_verification` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.provenance.expected_hashes_checked 1 values AMFR_HUMAN_Tsuboyama_2023_4G3O Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.provenance.expected_hashes_checked` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.provenance.reference_sha256 a8f498011532a74aa9fe556a50555a75e928c5837d19c06a87592ae04049b308 Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.provenance.reference_sha256` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.provenance.source_url https://github.com/OATML-Markslab/ProteinGym/tree/144fe22b07dfaeec2b366f2346203a9838a55b4c Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.provenance.source_url` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.provenance.upstream_revision 144fe22b07dfaeec2b366f2346203a9838a55b4c Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.provenance.upstream_revision` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.selected_assay AMFR_HUMAN_Tsuboyama_2023_4G3O Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.selected_assay` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.source_evidence.archive_sha256 3a83766254ac9ac9984ec25cb73c6e010ea4418f5e35f143933e6b6e6473b921 Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.source_evidence.archive_sha256` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.source_evidence.archive_size_bytes 43021128 Context-only references	ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) Original source ↗ coverage; protocol_configuration; provenance Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 Retrieved: 2026-09-20T20:54:03Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.source_evidence.archive_size_bytes` Source artifact SHA-256: `6a14b3866141d5779c76da8071d85c73b7cce0dc4e101c93122706873e847d26` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: source checked

1 source records and release history

ESM-2 8M masked-marginal scoring: local execution report (20 September 2026) · Original source · ca73fa47136d182f2d4ddb083d084712198fc0e2

Download this release

Technical metadata and extraction receipts

Stable ID: rewire-dataset-proteingym-amfr-v13

areas: proteins-complexes
version: ProteinGym-v1.3
dataset id: proteingym-dms-substitutions-v1.3
selected assay: AMFR_HUMAN_Tsuboyama_2023_4G3O
source locator: coverage; protocol_configuration; provenance
test count: 2972
split counts: Not reported
provenance: assay sha256: AMFR HUMAN Tsuboyama 2023 4G3O: dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a; data verification: local_bytes_hashed_not_independently_source_verified; expected hashes checked: AMFR_HUMAN_Tsuboyama_2023_4G3O; reference sha256: a8f498011532a74aa9fe556a50555a75e928c5837d19c06a87592ae04049b308; source url: https://github.com/OATML-Markslab/ProteinGym/tree/144fe22b07dfaeec2b366f2346203a9838a55b4c; upstream revision: 144fe22b07dfaeec2b366f2346203a9838a55b4c
source evidence: retrieved at: 2026-09-20T20:45:43.949575+00:00; url: https://marks.hms.harvard.edu/proteingym/ProteinGym_v1.3/DMS_ProteinGym_substitutions.zip; archive sha256: 3a83766254ac9ac9984ec25cb73c6e010ea4418f5e35f143933e6b6e6473b921; archive size bytes: 43021128; member: DMS_ProteinGym_substitutions/AMFR_HUMAN_Tsuboyama_2023_4G3O.csv; member sha256: dd911d925eca79329a496eb6cb7b181ab448e15db244daf1c1f9c01e8a704c2a; hash status: observed_download_bytes_not_independently_published_checksums
subset scope: All selected test rows; this is not the whole benchmark suite.

Related records

same data as: ProteinGym v1.3 AMFR stability assay, 2,972 variants
dataset subset: ESM-2 8M masked-marginal scoring on ProteinGym v1.3 AMFR substitution assay

Suggest a correction