rewire.itbenchmarks
Protocol

MFASS v1 (superseded)

MFASS v1 is a superseded historical protocol whose baseline used an incorrectly centred sequence window.

SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

No reviewed evaluations are linked here in this release. See the sources and separately identified configurations below.

0 evaluations · 0 results

Overview

Datasets

Historical MFASS exon-recognition evaluation; the correction notice documents a sequence-orientation mismatch in the v1 baseline.

Metrics

Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.

Allowed inputs

Historical baseline uses the incorrectly centered sequence window; corrected assay-oriented pairs belong to v2.

SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
Evaluation procedure diagram
How it worksComputational evaluation flow
Computational evaluation flow1. Input: Historical baseline uses the incorrectly centered sequence window; corrected assay-oriented pairs belong to v2.. Then: 2. Evaluation: Historical supervised baseline; its comparisons and split-cost claims are withdrawn pending corrected reruns.. Then: 3. Readout: Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.Computational evaluation flow1. Input: Historical baseline uses the incorrectly centered sequence window; corrected assay-oriented pairs belong to v2.. Then: 2. Evaluation: Historical supervised baseline; its comparisons and split-cost claims are withdrawn pending corrected reruns.. Then: 3. Readout: Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.Computational evaluation flow1. Input: Historical baseline uses the incorrectly centered sequence window; corrected assay-oriented pairs belong to v2.. Then: 2. Evaluation: Historical supervised baseline; its comparisons and split-cost claims are withdrawn pending corrected reruns.. Then: 3. Readout: Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.

Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.

SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

limited source coverage · Automated source review, 2026-09-23. All specifications and missing details

Results

All evaluations

0 evaluations · 0 results. Different protocols are not a single leaderboard.

Applied filters: All linked evaluations

No evaluations linked in this release.

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

Historical MFASS exon-recognition evaluation; the correction notice documents a sequence-orientation mismatch in the v1 baseline. The source distinguishes protocol versions from split versions. The earlier exon-only split-cost experiment requires a corrected rerun; its conclusions are withdrawn. Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants. Historical baseline and specialist results are preserved for audit. The corrected v2 baseline and DNABERT-2 protocol must not be relabelled as v1. The v1 windowing defect invalidates its baseline comparisons as current evidence; this is separate from the grouped-split controls.

SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

These source-backed links do not make different protocols or scores interchangeable.

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 0 active baseline roles have published Rewire measurements in this release. 1 historical protocols are retained separately. Measurements on a selected protocol do not establish coverage of an entire suite.

No execution recipe linked to this protocol. Recipe availability does not establish a completed evaluation.

No reviewed evaluations with results linked in this release.

Literature evidence is not a Rewire measurement. Executed but unpublished runs and private review status are not included.

Null control

Historical protocol

Training-set class prior where supervised fitting is permitted

Superseded protocol: retain history; do not start new runs.

Conventional reference

Historical protocol

Regularised classifier on simple permitted features, or protocol's conventional reference

Superseded protocol: retain history; do not start new runs.

Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums

Coverage is derived from release 2026-09-29-06401fd5b220. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths and considerations

No source-reviewed explanatory claims are recorded here yet.

Limitations and conditions

  • Superseded baseline comparisons and split-cost claims must not be used as current evidence. This profile documents the correction and does not reproduce or rehabilitate the historical results.
    SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
Profile review details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Stable record: rewire-mfass-v1

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-23. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
PropertyDescription and evidence
DatasetsHistorical MFASS exon-recognition evaluation; the correction notice documents a sequence-orientation mismatch in the v1 baseline.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
SplitsThe source distinguishes protocol versions from split versions. The earlier exon-only split-cost experiment requires a corrected rerun; its conclusions are withdrawn.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
MetricsPrecision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
BaselinesHistorical baseline and specialist results are preserved for audit. The corrected v2 baseline and DNABERT-2 protocol must not be relabelled as v1.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
Leakage controlsThe v1 windowing defect invalidates its baseline comparisons as current evidence; this is separate from the grouped-split controls.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
UncertaintyThe archived SpliceAI comparison records 2,000 paired bootstrap draws, 454 independent groups and 95% interval bounds. These quantify sampling uncertainty in the historical run; they do not correct its sequence-window error and must not be reused as current baseline evidence.
Sources (2)Archived MFASS v1 SpliceAI comparison; MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Archived JSON: independent_groups and paired.*.{draws,ci95_low,ci95_high}; README correction notice
Entity typeSuperseded historical computational protocol.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
OrganismsHuman (Homo sapiens): naturally occurring ExAC variants in or beside human exons, measured in the MFASS minigene assay.
SourcesMFASS original assay accession GSE120695 · GSE120695 SOFT: Series_summary and Sample_organism_ch1 fields
AssaysMFASS functional exon-recognition labels.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
Allowed inputsHistorical baseline uses the incorrectly centered sequence window; corrected assay-oriented pairs belong to v2.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
AdaptationHistorical supervised baseline; its comparisons and split-cost claims are withdrawn pending corrected reruns.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot
Supersession causeThe archived v1 baseline combined raw sequence with assay-coordinate rel_position. For 7,770 of 27,733 eligible variants the raw field was the reverse complement of original_seq, placing the 21-base 3-mer window at the wrong base. The source withdraws v1 baseline comparisons and split-cost claims; preserved JSON remains historical evidence only.
SourcesMFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e · README.md correction notice lines 3–10 and cohort validation lines 59–64

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.

Paper or primary resourceVersionReference
MFASS benchmark: corrected v2 protocol and reproducibility instructionsPrimary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256Read source
Search and extraction details

historical results preserved

Searches

  • Pangolin SpliceAI MFASS splice variant benchmark rewire

Evidence locations

  • Pinned README correction and archived v1 results links

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

20 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statementOriginal source and locationReview and provenance
Diagram caption
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram steps
  • Input: Historical baseline uses the incorrectly centered sequence window; corrected assay-oriented pairs belong to v2.
  • Evaluation: Historical supervised baseline; its comparisons and split-cost claims are withdrawn pending corrected reruns.
  • Readout: Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Diagram title
Computational evaluation flow
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.diagram.title

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Datasets
Historical MFASS exon-recognition evaluation; the correction notice documents a sequence-orientation mismatch in the v1 baseline.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Splits
The source distinguishes protocol versions from split versions. The earlier exon-only split-cost experiment requires a corrected rerun; its conclusions are withdrawn.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Adaptation
Historical supervised baseline; its comparisons and split-cost claims are withdrawn pending corrected reruns.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.10.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Supersession cause
The archived v1 baseline combined raw sequence with assay-coordinate rel_position. For 7,770 of 27,733 eligible variants the raw field was the reverse complement of original_seq, placing the 21-base 3-mer window at the wrong base. The source withdraws v1 baseline comparisons and split-cost claims; preserved JSON remains historical evidence only.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

README.md correction notice lines 3–10 and cohort validation lines 59–64

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.11.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Metrics
Precision at a fixed review capacity, average precision and AUROC. Each method’s point estimates use its scored subset; paired comparisons use common scored variants.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.2.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Baselines
Historical baseline and specialist results are preserved for audit. The corrected v2 baseline and DNABERT-2 protocol must not be relabelled as v1.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.3.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Leakage controls
The v1 windowing defect invalidates its baseline comparisons as current evidence; this is separate from the grouped-split controls.
Individual claims
MFASS computational protocol README at bee9133b83f3aedaf2bbb9013f1875515845607e

Original source ↗

Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot

Version: bee9133b83f3aedaf2bbb9013f1875515845607e
Retrieved: 2026-09-16T20:55:04.446365+00:00

source checked

automated source review · 2026-09-23

Audit details

Follow-up review of Supersession cause. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.

Field: attributes.profile.facts.4.value

Source artifact SHA-256: 62a9381484fd2360767e78b114d9aa6cd8a4fc6487a981881a4c4999f0ab1b23

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: superseded

5 source records and release history

Superseded by MFASS v2

Download this release
Technical metadata and extraction receipts

Stable ID: rewire-mfass-v1

areas
dna-genomes
entity level
protocol
version
v1
task
Splice-variant prioritisation
scope note
The mfass-v1 baseline used a mis-centred k-mer window for 7,770 assay variants whose raw sequence was reverse-complemented. mfass-v2 validates assay-oriented reference and mutant pairs and rebuilds the baseline; mfass-v1 remains a historical record.
benchmark research
review date: 2026-09-17; status: historical_results_preserved; primary sources: evidence-expansion-mfass-readme-62a93814; inspected locators: Pinned README correction and archived v1 results links; searched queries: Pangolin SpliceAI MFASS splice variant benchmark rewire; gaps: None recorded; claim scope: Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
historical missing metadata
None recorded
metadata review scope
historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
legacy kinds
benchmark
entity classification
review date: 2026-09-17; rationale: The source-backed record identifies a specified evaluated procedure and its dataset/split/scoring context. Classify it as a protocol while preserving version and comparison restrictions.; source ids: evidence-benchmark-mfass-pinned-readme; source locator: Pinned README: correction notice; Dataset; Cohort reconciliation; Split; MFASS-v2 results; Paired comparisons; Limits; Archived mfass-v1 snapshot; ambiguities: None recorded
Related records

Suggest a correction