What the evidence shows
Only two sequence-only controls have been measured on the target split, both in Rewire's own runs, so there is no model comparison on it yet.
On the 15,003 held-out records of the mRNABench designed-reporter test split, the composition ridge control reached a mean squared error of 1.914 against 2.366 for the training-mean control, and a Pearson correlation of 0.437; the training-mean control has no defined correlation, because its prediction is constant. The published FramePool panels are the only model comparison recorded here: neural models (FramePool and Optimus) reached Pearson 0.743 to 0.966 on the two random-sequence panels and 0.700 to 0.894 on the two human panels, while k-mer random forests, which were evaluated on the random panels only, reached 0.616 to 0.877. Those values are reported in the FramePool study on its own library, one split per panel with no uncertainty interval, and they are not measured on the designed-reporter split, so they are not baselines for it.
Both Rewire controls are single runs on one split with no seed variability, and their automated audit checks the run itself, not a published score.
- You bring
- Complete processed sequences and measured mean ribosome load from a defined reporter library
- A fixed training, validation and test assignment, with test labels withheld from fitting and model selection
- You want
- Two exact baseline configurations, their matched held-out evaluations and a repeat recipe; no foundation-model ranking or prediction of therapeutic performance.
- Evidence in this release
- 5 comparisons shown · 6 tools · 12 configurations
- Kept up to date
- Evidence check not yet recorded · re-checked monthly
Baseline
k-mer random forests. Random forests on k-mer counts are the non-deep-learning models the FramePool study trained on the same data to test whether deep learning helps. They are trained machine-learning models; the training-mean and composition controls this use case also asks about have no method records.
Machine-learning methods are compared with the baseline on the same data in 2 tables: FramePool study MRL: Random 25-100nt, FramePool study MRL: Random 50nt. Baseline rows are marked in each table.
Comparisons
Each table is one study or protocol with its own truth set and scoring. Compare tools within a table, not across tables.
FramePool study MRL: Random 25-100nt
Proxy evidenceAuthor-reportedMean ribosome load prediction, Pearson correlation, random-25-100nt
Sorted by Pearson (higher is better). The best value in each column is highlighted.
Caveats and method
- Not the locally evaluated mRNABench designed-MRL split.
- Pearson correlation is not absolute MRL calibration or prospective design hit rate.
- Human 50nt/25–100nt reporter sequences are truncated; not full-length endogenous UTR validation.
- No per-configuration uncertainty interval is supplied in these tables.
Protocol: FramePool study MRL: Random 25-100nt
FramePool study MRL: Random 50nt
Proxy evidenceAuthor-reportedMean ribosome load prediction, Pearson correlation, random-50nt
Sorted by Pearson (higher is better). The best value in each column is highlighted.
Caveats and method
- Not the locally evaluated mRNABench designed-MRL split.
- Pearson correlation is not absolute MRL calibration or prospective design hit rate.
- Human 50nt/25–100nt reporter sequences are truncated; not full-length endogenous UTR validation.
- No per-configuration uncertainty interval is supplied in these tables.
Protocol: FramePool study MRL: Random 50nt
FramePool study MRL: Human 50nt
Proxy evidenceAuthor-reportedMean ribosome load prediction, Pearson correlation, human-50nt
Sorted by Pearson (higher is better). The best value in each column is highlighted.
Caveats and method
- Not the locally evaluated mRNABench designed-MRL split.
- Pearson correlation is not absolute MRL calibration or prospective design hit rate.
- Human 50nt/25–100nt reporter sequences are truncated; not full-length endogenous UTR validation.
- No per-configuration uncertainty interval is supplied in these tables.
Protocol: FramePool study MRL: Human 50nt
FramePool study MRL: Human 25-100nt
Proxy evidenceAuthor-reportedMean ribosome load prediction, Pearson correlation, human-25-100nt
Sorted by Pearson (higher is better). The best value in each column is highlighted.
Caveats and method
- Not the locally evaluated mRNABench designed-MRL split.
- Pearson correlation is not absolute MRL calibration or prospective design hit rate.
- Human 50nt/25–100nt reporter sequences are truncated; not full-length endogenous UTR validation.
- No per-configuration uncertainty interval is supplied in these tables.
Protocol: FramePool study MRL: Human 25-100nt
mRNABench Sample designed MRL
Direct evidenceRewire runPredicting target_mrl_designed on the complete 15,003-record canonical test split of the Sample designed reporter dataset.
Sorted by MSE (lower is better). The best value in each column is highlighted.
Scroll sideways to see all 3 columns.
Caveats and method
- One target, one split and one execution per method; no uncertainty or seed variability. No homology-separated or compositional-generalisation claim.
- This training-only protocol differs from upstream probing and is not an aggregate mRNABench score or a reproduction of a published model result.
- The original run’s raw inputs and saved predictions are not publicly archived. The recipe obtains source data and runs the controls again, including separate protein controls; it does not rescore prior predictions. Source-data reuse terms and cross-platform reproducibility remain unresolved.
- Research baseline evidence only, with no new execution, human domain review or clinical validation.
Protocol: mRNABench Sample designed MRL
Not covered yet
- Only two procedural controls have been evaluated here; no pretrained model or more complex sequence model has a matched result in this protocol.
- There is one split and one execution per configuration, with no uncertainty interval or seed-variability estimate. The split does not establish homology separation or compositional generalisation.
- RidgeCV uses training labels only and leaves the validation split unused. This differs from upstream probing; the selected test MSE must not be compared as if it were the paper’s aggregated Pearson score or default validation result.
- The training-mean control has undefined Pearson and Spearman correlations because its predictions are constant. Unavailable correlations are not zero performance scores.
- The original run’s raw inputs and saved predictions are not publicly archived. The public reports record hashes and a recipe for obtaining source data and running the controls again; prior predictions cannot be rescored from those reports. Dataset reuse terms are unreported.
- The recorded repeat recipe executes all four local sequence controls, including a separate protein dataset. Portable command examples have not been rerun verbatim during this review. Timings cover the reported calculation sections, not complete setup or cross-machine performance.
- 30 September 2026 audit: FramePool/Sample random and truncated-human libraries differ from the locally evaluated mRNABench designed-MRL split. Learned results exist for the use case, but no new result fills that exact local protocol gap.
- 30 September 2026 audit: Sequence-length transfer is measured; Pearson correlation does not establish absolute calibration, design success, therapeutic translation or full-length endogenous prediction.
- 30 September 2026 audit: Local protocol raw predictions/input reuse terms remain unresolved; supplementary tables do not establish permissions for those separate inputs.
- 30 September 2026 audit: Per-configuration uncertainty and exact human cohort denominators remain unextracted.
Contribute evidence or propose a correction
Question, sources and review
Who this is for, setting and exclusions
Establish training-mean and sequence-composition controls on a fixed held-out reporter dataset before deciding whether a more complex model adds useful prediction. The current evidence contains these two procedural controls and no pretrained model.
- Who this is for
- Computational researchers building models of reporter translation
- Experimental researchers evaluating a designed 5′UTR library before selecting a modelling strategy
- Biological setting
- The current comparison covers only the Sample designed subset in mRNABench and its measured mean ribosome load. Both controls use the full processed source sequence and the same seed-2541 split: 70,011 training, 15,003 validation and 15,003 test records. Both score all 15,003 held-out test records.
- Research setting
- This is research evidence for designing a baseline comparison. Reporter mean ribosome load does not establish therapeutic efficacy or clinical suitability.
Outside this use case
- Benchmark-wide mRNABench performance or comparisons with the paper’s aggregated model scores
- Claims about novel combinations of sequence motifs, homology-separated generalisation, other reporter systems or RNA chemistries
- Therapeutic potency, in-vivo protein output and clinical decisions
Related articles
Read the background, methods and practical considerations behind this question.
Separates UTR translation and RNA stability tasks, with guidance on input scope, simple baselines and held-out evaluation.
Sources, review and downloads
Automated source review · 2026-09-30 · Codex source curation, independent worker cross-review and root integration review
Source-backed literature curation with independent automated transcription and scope checks. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.
Release 2026-11-02-a7f03c9e55a5 · question use-case-utr-translation-baselines
Use-case input digest 43a80aef5473f1a49c013b6d46b720b850ff5a0d18dcbb741dcdeed1ed082db1
Download questions and review metadata (JSON) (gzip) · Verify release checksums (gzip)