Training-mean control: automated execution audit
ca73fa47136d182f2d4ddb083d084712198fc0e2
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
| Model or configuration | Metric | Value | Dataset | Evaluation |
|---|---|---|---|---|
| Training-mean control (FLIP2 Rhomax by_wild_type) | NDCG (cutoff not stated) | 0.9206667522227658 unitless | FLIP2 Rhomax by_wild_type test subset | Training-mean control on FLIP2 Rhomax by_wild_type |
| Training-mean control (FLIP2 Rhomax by_wild_type) | Spearman | undefined unitless | FLIP2 Rhomax by_wild_type test subset | Training-mean control on FLIP2 Rhomax by_wild_type |
Claims
Statements this source supports, each with where in the source it was checked.
- Evidence summary: Assess rhodopsin wavelength prediction across sequence backgrounds: Summary RhoMax, which adds structural binding-pocket inputs, had the lowest wavelength error on each of its own wild-type holdout splits. In separate Rewire runs on the FLIP2 split, two amino-acid composition probes outranked two frozen ESM-2 probes, and both ESM-2 probes fell below a constant control. Averaged over four RhoMax splits, mean absolute error was 10.45 nm for RhoMax, 13.46 nm for RhoMax plus retinal and 14.90 nm for the BLASSO comparator, with standard deviations of 10.9 to 11.7 nm; per-split RhoMax error ran from 7.28 to 16.68 nm. In electronvolts the three methods are closer, and on split 3 all three print the same mean absolute error of 0.045 eV. On the FLIP2 Rhomax by_wild_type split of 184 held-out sequences, Rewire's runs gave a Spearman correlation of 0.418 for a 40-feature amino-acid composition probe and 0.418 for a 22-feature one, against -0.146 for a frozen ESM-2 8M probe and -0.222 for a 35M probe; NDCG was 0.955, 0.955, 0.896 and 0.907 against 0.921 for a training-mean control, whose Spearman is undefined. The two split definitions are different and their scores must not be pooled. The RhoMax values are reported by its authors with no independent reproduction, and the Rewire runs are single runs with no seed variability, with the 35M checkpoint chosen after the 8M outcome was known. (uc20260930-source-rhomax-2024: per-split and mean rows of the wild-type holdout table, nm and eV columns; rewire-local-20260920 and rewire-local-20260921 run outputs: metrics.spearman and metrics.ndcg)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
13 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 9b3ed3267a1b8557bb67214594db1fa0044bd6d6ff41b57324b8d603a88ad974 Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://raw.githubusercontent.com/rewire-bio/rewire-benchmarks/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/audit.json Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-09-20T20:54:03Z Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Source type automated_execution_audit Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Url https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/audit.json Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Version ca73fa47136d182f2d4ddb083d084712198fc0e2 Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description Verification of coverage, saved predictions and reference metric calculations; no human review or published-score reproduction implied. Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Facets: areas1 values
| Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Record ID rewire-local-20260920-audit-source-flip2-train-mean Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Record type source Source metadata | Training-mean control: automated execution audit No field-specific location recorded Version: ca73fa47136d182f2d4ddb083d084712198fc0e2 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-21-c8bea25e5abb · Record review: source checked
Verification of coverage, saved predictions and reference metric calculations; no human review or published-score reproduction implied.
Technical metadata and extraction receipts
Stable ID: rewire-local-20260920-audit-source-flip2-train-mean
- areas
- proteins-complexes
- url
- https://github.com/rewire-bio/rewire-benchmarks/blob/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/audit.json
- artifact url
- https://raw.githubusercontent.com/rewire-bio/rewire-benchmarks/ca73fa47136d182f2d4ddb083d084712198fc0e2/research/local-runs-2026-09-20/flip2-train-mean/audit.json
- artifact sha256
- 9b3ed3267a1b8557bb67214594db1fa0044bd6d6ff41b57324b8d603a88ad974
- version
- ca73fa47136d182f2d4ddb083d084712198fc0e2
- retrieved at
- 2026-09-20T20:54:03Z
- source type
- automated_execution_audit