Training-mean control: local execution report (20 September 2026)
ca73fa47136d182f2d4ddb083d084712198fc0e2
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
| Model or configuration | Metric | Value | Dataset | Evaluation |
|---|---|---|---|---|
| Training-mean control (FLIP2 Rhomax by_wild_type) | NDCG (cutoff not stated) | 0.9206667522227658 unitless | FLIP2 Rhomax by_wild_type test subset | Training-mean control on FLIP2 Rhomax by_wild_type |
| Training-mean control (FLIP2 Rhomax by_wild_type) | Spearman | undefined unitless | FLIP2 Rhomax by_wild_type test subset | Training-mean control on FLIP2 Rhomax by_wild_type |
Claims
Statements this source supports, each with where in the source it was checked.
- Evidence summary: Assess rhodopsin wavelength prediction across sequence backgrounds: Summary RhoMax, which adds structural binding-pocket inputs, had the lowest wavelength error on each of its own wild-type holdout splits. In separate Rewire runs on the FLIP2 split, two amino-acid composition probes outranked two frozen ESM-2 probes, and both ESM-2 probes fell below a constant control. Averaged over four RhoMax splits, mean absolute error was 10.45 nm for RhoMax, 13.46 nm for RhoMax plus retinal and 14.90 nm for the BLASSO comparator, with standard deviations of 10.9 to 11.7 nm; per-split RhoMax error ran from 7.28 to 16.68 nm. In electronvolts the three methods are closer, and on split 3 all three print the same mean absolute error of 0.045 eV. On the FLIP2 Rhomax by_wild_type split of 184 held-out sequences, Rewire's runs gave a Spearman correlation of 0.418 for a 40-feature amino-acid composition probe and 0.418 for a 22-feature one, against -0.146 for a frozen ESM-2 8M probe and -0.222 for a 35M probe; NDCG was 0.955, 0.955, 0.896 and 0.907 against 0.921 for a training-mean control, whose Spearman is undefined. The two split definitions are different and their scores must not be pooled. The RhoMax values are reported by its authors with no independent reproduction, and the Rewire runs are single runs with no seed variability, with the 35M checkpoint chosen after the 8M outcome was known. (uc20260930-source-rhomax-2024: per-split and mean rows of the wild-type holdout table, nm and eV columns; rewire-local-20260920 and rewire-local-20260921 run outputs: metrics.spearman and metrics.ndcg)
- Relevance of FLIP2 Rhomax by_wild_type to "Assess rhodopsin wavelength prediction across sequence backgrounds": Assessed by FLIP2 Rhomax by_wild_type (rewire-local-20260921-instructions-esm2-35m: Matched Rhomax evaluations; Procedure and evidence; Re-run or inspect; rewire-local-20260921-source-esm2-35m: /metrics; /coverage; /model_configuration; /protocol_configuration; /execution; /provenance; rewire-local-20260921-source-esm2-8m: /metrics; /coverage; /model_configuration; /protocol_configuration; /execution; /provenance; rewire-local-20260921-source-composition22: /metrics; /model_configuration; /protocol_configuration; rewire-local-20260920-source-flip2-train-mean: /metrics; /coverage; /model_configuration; rewire-local-20260920-source-flip2-composition: /metrics; /coverage; /model_configuration; /provenance)
- Training-mean control (FLIP2 Rhomax by_wild_type): facets.method_types: Facets: method types ["conventional_pipeline"] (Fixed training-mean and composition-ridge probes (record descriptions))
Models, benchmarks and other records
Configurations (1)
Protocols (1)
Baselines (1)
Evaluations (1)
Use cases (1)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
15 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 eb89260f6e138bb9ae2fdf3b882a87fdf8aeabd54a0ac6f17a73f5671c96b4d8 Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://raw.githubusercontent.com/rewire-bio/rewire-benchmarks/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/report.json Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Evidence origin rewire_run Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieval note Generated local execution report, verified against saved predictions; immutable public Git source. Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-09-20T20:54:03Z Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Source type rewire_execution_report Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Url https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/report.json Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Version ca73fa47136d182f2d4ddb083d084712198fc0e2 Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description Public sanitized execution evidence. Raw sequences and predictions remain local. Automated checking does not establish reproduction of a published model score. Source metadata | Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Facets: areas1 values
| Training-mean control: local execution report (20 September 2026) No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: source checked
Public sanitized execution evidence. Raw sequences and predictions remain local. Automated checking does not establish reproduction of a published model score.
Technical metadata and extraction receipts
Stable ID: rewire-local-20260920-source-flip2-train-mean
- areas
- proteins-complexes
- url
- https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/report.json
- artifact url
- https://raw.githubusercontent.com/rewire-bio/rewire-benchmarks/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/report.json
- artifact sha256
- eb89260f6e138bb9ae2fdf3b882a87fdf8aeabd54a0ac6f17a73f5671c96b4d8
- version
- ca73fa47136d182f2d4ddb083d084712198fc0e2
- retrieved at
- 2026-09-20T20:54:03Z
- source type
- rewire_execution_report
- evidence origin
- rewire_run
- retrieval note
- Generated local execution report, verified against saved predictions; immutable public Git source.