Evaluation

ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) on FLIP2 Rhomax by_wild_type

Frozen features with a fixed alpha-10 ridge head fitted only on training rows; all 184 Rhomax by_wild_type held-out proteins scored. This selected split is not a whole FLIP2 score.

Research readiness

These checks assess whether the evidence supports a reproducible investigation. A source-checked score alone does not meet these requirements.

Release 2026-09-29-06401fd5b220 · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
artifact hashes: verification is missing
join integrity: verification is missing
score semantics: verification is missing
metric replay: verification is missing

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
artifact hashes: verification is missing
join integrity: verification is missing
score semantics: verification is missing
metric replay: verification is missing
annotations: verification is missing
dependence: verification is missing

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
artifact hashes: verification is missing
join integrity: verification is missing
score semantics: verification is missing
recipe pinned: verification is missing
resource estimate: verification is missing

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
artifact hashes: verification is missing
join integrity: verification is missing
score semantics: verification is missing
independent validation: verification is missing
overlap checked: verification is missing

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Read reviewed discrepancy investigations

Evaluation results

1 evaluation · 2 results. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) (FLIP2 Rhomax by_wild_type)	Protocol: FLIP2 Rhomax by_wild_type Dataset subset: FLIP2 Rhomax by_wild_type test subset	0.896 ndcg dimensionless · higher Uncertainty: Not reported Coverage: 184/184	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) on FLIP2 Rhomax by_wild_type FLIP2 Rhomax by_wild_type Aggregation: Not reported ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report; ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit · metrics.ndcg
Configuration: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) (FLIP2 Rhomax by_wild_type)	Protocol: FLIP2 Rhomax by_wild_type Dataset subset: FLIP2 Rhomax by_wild_type test subset	-0.146 spearman dimensionless · higher Uncertainty: Not reported Coverage: 184/184	Rewire evaluation · Source checked Methods, coverage and source ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) on FLIP2 Rhomax by_wild_type FLIP2 Rhomax by_wild_type Aggregation: Not reported ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report; ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit · metrics.spearman

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Evaluation procedure

FLIP2 Rhomax by_wild_type

Configuration: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) (FLIP2 Rhomax by_wild_type)
Protocol: FLIP2 Rhomax by_wild_type
Dataset subset: FLIP2 Rhomax by_wild_type test subset

origin: Rewire evaluation
configuration: 1
protocol id: rewire-protocol-flip2-rhomax-by-wild-type-v1
dataset version: rewire-dataset-flip2-rhomax-by-wild-type-v3
split: canonical test
subset: flip2-rhomax-by-wild-type
population: 184/184
inputs: Complete amino-acid sequences; no labels enter encoder; no MSA/templates
adaptation: fixed train-only ridge on frozen representations
metric implementation: 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34
aggregation: all held-out test rows of the selected target
budget: one local CPU evaluation; no held-out hyperparameter selection

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split: canonical test
Adaptation: fixed train-only ridge on frozen representations
Scoring implementation: 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Access requirements and official instructions: FLIP2 Rhomax by_wild_type

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

363 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
attributes.comparison.adaptation fixed train-only ridge on frozen representations Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `4126d97686aaa026bab6989f85c405e097b383831e4b8012232394c2ec9178de` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.adaptation fixed train-only ridge on frozen representations Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `075be1919b8364a3ef1028f9204431d3caa7c3b4da586212d0ed6dd4b024c8da` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.adaptation fixed train-only ridge on frozen representations Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `d6df9e916e77872112571b404265e918276974c79dcbe25721b3eca0eae4e51a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation all held-out test rows of the selected target Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `4126d97686aaa026bab6989f85c405e097b383831e4b8012232394c2ec9178de` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation all held-out test rows of the selected target Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `075be1919b8364a3ef1028f9204431d3caa7c3b4da586212d0ed6dd4b024c8da` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.aggregation all held-out test rows of the selected target Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `d6df9e916e77872112571b404265e918276974c79dcbe25721b3eca0eae4e51a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `4126d97686aaa026bab6989f85c405e097b383831e4b8012232394c2ec9178de` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `075be1919b8364a3ef1028f9204431d3caa7c3b4da586212d0ed6dd4b024c8da` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.budget one local CPU evaluation; no held-out hyperparameter selection Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `d6df9e916e77872112571b404265e918276974c79dcbe25721b3eca0eae4e51a` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.comparison.dataset_version rewire-dataset-flip2-rhomax-by-wild-type-v3 Context-only references	ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit Original source ↗ metrics; coverage; protocol_results Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 1663d1f04b2bbd6dfcff77fea78129d30b0de191 Retrieved: 2026-09-22T21:18:22Z	not individually reviewed No individual claim review recorded rewire run Audit details Field: `attributes.comparison.dataset_version` Source artifact SHA-256: `4126d97686aaa026bab6989f85c405e097b383831e4b8012232394c2ec9178de` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: source checked

3 source records and release history

ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): report · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): audit · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191
ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): execution and verification instructions · Original source · 1663d1f04b2bbd6dfcff77fea78129d30b0de191

Download this release

Technical metadata and extraction receipts

Stable ID: rewire-local-20260921-evaluation-esm2-8m

areas: proteins-complexes
origin: rewire_run
execution scope: complete_selected_evaluation
published score reproduction: false
protocol: FLIP2 Rhomax by_wild_type
version: 1
source locator: metrics; coverage; protocol_results
original sdk scope: full
original sdk completion: complete
suite complete: false
eligible count: 184
scored count: 184
missing count: 0
coverage: 184/184
comparison: protocol id: rewire-protocol-flip2-rhomax-by-wild-type-v1; dataset version: rewire-dataset-flip2-rhomax-by-wild-type-v3; split: canonical test; subset: flip2-rhomax-by-wild-type; population: 184/184; inputs: Complete amino-acid sequences; no labels enter encoder; no MSA/templates; adaptation: fixed train-only ridge on frozen representations; metric implementation: 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34; aggregation: all held-out test rows of the selected target; budget: one local CPU evaluation; no held-out hyperparameter selection
input information: Complete amino-acid sequences; no labels enter encoder; no MSA/templates
model configuration: checkpoint: esm2_t6_8M_UR50D; device: cpu; dimension: 320; encoder frozen: true; head: alpha: 10; feature scaling: none; fit intercept: true; max iter: 1000000; name: rewire-frozen-embedding-ridge-v1; refit: train only; relationship to published baseline: Frozen-embedding extension of upstream alpha-10 ridge. The published baseline uses one-hot features and scales targets using train plus validation rows. This extension is not that published baseline.; solver: auto; target scaling: StandardScaler fitted only on train labels; tol: 0.00001; validation use: none; fixed hyperparameters; pooling: mean over residues excluding BOS/EOS and padding; representation layer: 6; scope: one complete Rhomax split, not FLIP2 suite; torch seed: 0; torch threads: 4; validation used: false
protocol configuration: adapter input contract: opaque-sequence-inputs-v1; canonical split counts: test: 184; train: 584; validation: 116; canonical test count: 184; data verification: pinned_source_bytes; dataset: rhomax; embedding probe: alpha: 10; feature scaling: none; fit intercept: true; max iter: 1000000; name: rewire-frozen-embedding-ridge-v1; refit: train only; relationship to published baseline: Frozen-embedding extension of upstream alpha-10 ridge. The published baseline uses one-hot features and scales targets using train plus validation rows. This extension is not that published baseline.; solver: auto; target scaling: StandardScaler fitted only on train labels; tol: 0.00001; validation use: none; fixed hyperparameters; input representation: amino-acid sequence; metric: spearman; metric direction: higher; metrics: spearman; ndcg; prepared rows sha256: d86fe3926529df72649f4638ffa0d89ec62460d831ceb62ad47ff86249469a12; smoke limit per split: Not reported; source note: Not reported; source reuse terms: CC-BY-4.0 archive; source attribution under resources/flip2; split: by_wild_type; suite complete: false; target units: nm; task: regression
provenance: data verification: pinned_source_bytes; dataset version: zenodo-18433203-v3; evaluator url: https://github.com/J-SNACKKB/FLIP/blob/62cace8735f5610e2743cf06ce0f944b37fffaa6/baselines/aggregate.py#L131-L135; source csv sha256: 8e78f6a16cd5298131dca83130d4b94cc0ab4ae9c699460880441c19a65f656f; source sha256: 7c6d2f02cb89310378ac9897c321fbbd909fb0757ac88803dcce84f5f6b05ce3; source url: https://zenodo.org/api/records/18433203/files/rhomax/by_wild_type.csv.gz/content; split origin: Archived assignments: validation=True held out of training; upstream revision: 62cace8735f5610e2743cf06ce0f944b37fffaa6
predictions sha256: b975eff8d023c310860b6d8215fbe365a694985950afc95966db632eaa5eeb6c
prepared sha256: f417d43497e0c8c453bd7c66cfea05959b4d6d47462283096e42a2dae8b72419
execution: adapter: ESM2Embeddings; adapter provenance: checkpoint sha256: 46f002a9870c9bdecd0ea887acb1f9a38a6b561e8f8bf8a6990b679b9d31b928; checkpoint url: https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t6_8M_UR50D.pt; device: cpu; fair esm version: 2.0.0; implementation sha256: 1f475a084964698145fb07f7d0cd851a486d06a508e266904160d6d6ec9ef9bb; maximum sequence length: 1022; model: esm2_t6_8M_UR50D; published result reproduction: false; reference: https://github.com/facebookresearch/esm#usage; representation layer: 6; strategy: frozen_final_layer_residue_mean_excluding_special_tokens; training overlap: unreported; UniRef50 pretraining may overlap benchmark proteins; batch size: 4; fitting: protocol_owned_probe; inference and fit seconds: 17.59812141599832; mode: local_adapter; prediction type: embedding
timing seconds: evaluation: 0.048134666998521425
environment: architecture: arm64; container digest: unreported; container runtime: unreported; platform: macOS-26.6.2-arm64-arm-64bit; python: 3.11.13; rewirebench: 0.4.0; sdk code sha256: 9350e367b20ec48d9ce1195cb5b73c4bd98ce8530a2d1f7ba2157fc8be75fa34; sif sha256: unreported
run url: https://github.com/rewire-bio/rewire-benchmarks/blob/1663d1f04b2bbd6dfcff77fea78129d30b0de191/research/baseline-programme-2026-09-21/rhomax/esm2.report.json
reproduction url: https://github.com/rewire-bio/rewire-benchmarks/blob/1663d1f04b2bbd6dfcff77fea78129d30b0de191/research/baseline-programme-2026-09-21/README.md
reproduction note: The linked pinned scripts describe this exact execution. No generic suite recipe is claimed to reproduce it; this review only checked existing artifacts.
limitations: No confidence intervals or seed variability estimate.; Source checking and local execution do not reproduce a published paper score.; Pretraining overlap is unreported; conclusions apply to the exact evaluated configuration.

Related records

configuration: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire) (FLIP2 Rhomax by_wild_type)
protocol: FLIP2 Rhomax by_wild_type
dataset: FLIP2 Rhomax by_wild_type test subset
evaluation: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): ndcg
evaluation: ESM-2 8M frozen residue-mean embeddings + fixed ridge (Rewire): spearman

Suggest a correction