rewirebio.iobenchmarks

Use caseDNA and genomesResearch and clinical research

Select a patient-RNA splicing-variant validation workflow

Which DNA/RNA evidence workflow identifies splice-altering variants for inherited-disorder follow-up when patient RNA and tissue-specific context are available?

What the evidence shows

The one recovery result already on this page is FRASER on the 119-sample Kremer fibroblast cohort, reported by its developers: a mean 85% (11 of 13) of known pathogenic splicing events recovered at a 30-sample subsample.

Six published comparisons add evidence. Segarra-Casas et al. (2025), independent of the tool developers, recorded which callers reported 16 known pathogenic splicing events in clinical muscle RNA-seq from 98 samples: FRASER with |dPSI| > 0.1 reported 13, FRASER with |dPSI| > 0.3 reported 9, FRASER2 10 (and one more only with the full cohort), rMATS-turbo 7 and LeafCutterMD 4 (Table 2); the events were established partly with these callers. Segers et al. (2026), the saseR developers, ranked 12 reported disease genes in the Kremer cohort: the gene ranked within the top 10 in 9 of 12 samples for saseR-junctions, in 6 of 12 for FRASER 2.0 with the autoencoder and in 7 of 12 for FRASER 2.0 with PCA (Table 2); the 12 genes were reported in earlier analyses of the same cohort, partly with OUTRIDER and FRASER, and several are expression rather than splicing defects.

Silverstein et al. (2026, preprint), a clinical research group that did not develop the splicing callers, scored them on 97 previously diagnosed neuromuscular RNA-seq samples and also counted outlier calls in true-negative samples (Table S9). On known pathogenic splicing events in muscle and fibroblast RNA-seq, rMATS-turbo detected 15 of 23 and MINTIE 8 of 23; on muscle, fibroblast and cycloheximide-treated fibroblast RNA-seq, FRASER detected 21 of 68 and LeafcutterMD none of 67; FRASER2.0, run on muscle only, detected 2 of 16. rMATS-turbo and MINTIE reported outliers in all 9 true-negative samples they were run on, FRASER and LeafcutterMD in all 21, and FRASER2.0 in 2 of 6. Denominators differ by tool, a negative sample with any outlier counts as one false positive, and the known events were chosen because they were clinically interpretable and visible in the RNA-seq, which the authors say favours clearly defined events.

Drost et al. (2025), a diagnostic laboratory independent of the tool developers, scored 243 clinically ascertained variants, each tested for a splicing effect in patient cells or by exon trapping, with four DNA-based predictors. On all 243, AUROC was 0.895 for SpliceAI, 0.885 for Pangolin, 0.830 for SPiP and 0.806 for SQUIRLS (Data S1 Table S3); at literature thresholds sensitivity was 0.871, 0.858, 0.832 and 0.800 and specificity 0.750, 0.773, 0.761 and 0.591 (Table S4), and the authors' rule of requiring at least two of the four tools gave sensitivity 0.916 and specificity 0.727. Most in-house variants were sent for testing because another predictor (Alamut) flagged them, so negatives are mostly predicted-but-unconfirmed variants.

Two further DNA-based comparisons add proxy evidence. Fortier et al. (2026), independent of the SpliceAI, CI-SpliceAI and OpenSpliceAI developers, report balanced accuracy on functional splice-assay variants in ABCA4 and MYBPC3 of 0.836 for SpliceAI, 0.794 for CI-SpliceAI and 0.746 for OpenSpliceAI, and on 242 deep intronic pathogenic variants with 242 common controls of 0.940, 0.890 and 0.841 (Tables 2 and 4); both sets of thresholds were tuned on the same data, so these are upper-bound estimates, and CI-SpliceAI was trained on all chromosomes. Alavi (2026, preprint) applied thresholds tuned on OPA1 variants to 76 functionally validated spliceogenic variants (26 at canonical ±1,2, 35 at near-splice intronic positions and 15 exonic, by the paper's Supplemental Table VI) and 107 ClinVar benign intronic variants across 64 genes: sensitivity and specificity were 88.2% and 99.1% for SpliceAI, 85.5% and 86.9% for pretrained NTv3 650M, 71.1% and 100% for Evo2 40B with a human identifier and 27.6% and 85.0% for Genos 10B (Table IV).

Version 1 of the paper scored every AlphaGenome variant one base from its true position (upstream issue #2), and those rows are withheld. In version 2, rerun with the corrected position, sensitivity and specificity were 92.1% and 99.1% for AlphaGenome's splice-junction score, 78.9% and 100% for its splice-site score, 57.9% and 100% for the composite score of AlphaGenome's splicing notebook and 23.7% and 100% for splice-site usage (Table IV); the four scores used different thresholds, all tuned on OPA1 variants. Jaganathan et al. (2026, preprint), the SpliceAI2 developers at Illumina, scored four DNA-based predictors on rare variants that create a GT or AG dinucleotide in 5,435 Genomics England rare-disease participants with blood RNA-seq, against new junctions seen in the carriers' RNA (3,065 cryptic gains and 1,775,800 nulls): auPRC was 0.59 for SpliceAI2, 0.48 for SpliceAI, 0.40 for Pangolin and 0.36 for AlphaGenome, or 0.45 with AlphaGenome's raw splice site score (Fig. 4g; Methods).

Only splicing visible in blood can be confirmed, so some nulls may alter splicing in other tissues, and SpliceAI2 was trained on all human genes except those on chromosomes 21 and 22. Of 23 rare variants in phenotype-matched panel genes that SpliceAI2 prioritised, 17 showed the predicted splicing change in RNA-seq; no other tool was put through this step. The paper compares no rule-based splice-site scorer.

All values are transcribed from the publications, not reproduced; none is a diagnostic-yield estimate, and results from different sources measure different endpoints and must not be pooled. This summary describes evidence and does not recommend a clinical workflow.

You bring
  • Patient skin-fibroblast (or matched tissue) RNA-seq aligned reads
  • The number of samples available for a FRASER-style aberrant-splicing cohort
You want
A sourced recovery curve (fraction of known pathogenic splicing events recovered as cohort size grows) and a separate sequence-classification proxy; no diagnostic-yield claim for unknown cases.
Evidence in this release
8 comparisons shown · 1 held for review · 24 tools · 51 configurations
Kept up to date
Evidence last checked 11 October 2026 · re-checked monthly, next due 11 November 2026
Executive Takeaway

For "Select a patient-RNA splicing-variant validation workflow", on Splice predictors and genomic foundation models on OPA1 and multi-gene splice variants (Alavi 2026), OPA1 evaluation sets (thresholds tuned here) (Accuracy, a proxy protocol), ALPHAGENOME SPLICE-SITE USAGE and ALPHAGENOME SPLICE-SITES share the best value, 96.1%, reported by Probing Genomic Foundation Models with Splice-Variant Perturbations (Alavi, bioRxiv 2026, version 2).

Best reported valueALPHAGENOME SPLICE-SITE USAGEProbing Genomic Foundation Models with Splice-Variant…
Accuracy96.1%Splice predictors and genomic foundation models on OPA1 and multi-gene splice variants (Alavi 2026), OPA1 evaluation sets (thresholds tuned here)
Baseline on this protocolNone scoredNo margin is computed
Ranked tools1313 from independent studies

13 tools have a value on this protocol; 13 of them come from studies whose authors did not develop the tool. No baseline method is scored on this protocol. Reviewers judged this protocol a proxy for the use case.

Ranking

Ranked on Splice predictors and genomic foundation models on OPA1 and multi-gene splice variants (Alavi 2026), OPA1 evaluation sets (thresholds tuned here) (Accuracy, higher is better). Proxy protocol Reviewers judged this protocol a proxy for the use case, not a direct test of it. Only results on this protocol are ranked; other tools are listed below the table.

6 tools in 13 rows, from 2 sources. 13 of 13 results (100%) come from an independent study.

Rank▲ModelAccuracy (higher is better)EvidenceCoverageReported byLicense
#196.1%Source-checked Independent study2/8Probing Genomic Foundation Models with…Non-commercial AlphaGenome…
#196.1%Source-checked Independent study2/8Probing Genomic Foundation Models with…Non-commercial AlphaGenome…
#395.3%Source-checked Independent study2/8Probing Genomic Foundation Models with…Non-commercial AlphaGenome…
#395.3%Source-checked Independent study2/8Probing Genomic Foundation Models with…Non-commercial AlphaGenome…
#395.3%Source-checked Independent study4/8Probing Genomic Foundation Models with…CC-BY-NC-4.0 for trained…
#693.0%Source-checked Independent study1/8Probing Genomic Foundation Models with…Apache-2.0 is declared in…
#792.2%Source-checked Independent study1/8Probing Genomic Foundation Models with…Apache-2.0 is declared in…
#891.4%Source-checked Independent study1/8Probing Genomic Foundation Models with…Not recorded
#989.1%Source-checked Independent study1/8Probing Genomic Foundation Models with…Not recorded
#989.1%Source-checked Independent study1/8Probing Genomic Foundation Models with…Not recorded
#1186.7%Source-checked Independent study1/8Probing Genomic Foundation Models with…Not recorded
#1277.3%Source-checked Independent study1/8Probing Genomic Foundation Models with…Not recorded
#1366.4%Source-checked Independent study1/8Probing Genomic Foundation Models with…Not recorded
18 tools are evaluated elsewhere in this use case but not on this protocol, so not ranked

Baseline

FRASER and OUTRIDER, with a legacy splice-site vote. On patient RNA, FRASER (splicing) and OUTRIDER (expression) are the standard outlier callers, run in the DROP workflow; they fit count models to the cohort and are not trained on labelled variants. On DNA, the reference is the Legacy Ensemble, a vote across MaxEntScan, NNSplice, GeneSplicer and a SpliceSiteFinder-style matrix; NNSplice is an early neural network, so this vote is a legacy comparator rather than a strictly pre-machine-learning one.

Machine-learning methods are compared with the baseline on the same data in one table: SpliceAI and its open-source reimplementations on six splice benchmarks (Fortier et al. 2026). Baseline rows are marked in each table.

Tools compared

Comparisons

Unified model comparison for this use case. Compare tools against protocol baselines in a single view.

Splice predictors and genomic foundation models on OPA1 and multi-gene splice variants (Alavi 2026)

Proxy evidenceIndependent study: Reported in a study whose authors did not develop the evaluated tool.

Accuracy (higher is better), with the other metrics below each value. The best value in each column is highlighted.

ToolOPA1 evaluation sets (thresholds tuned here)Independent 65-gene test set
ALPHAGENOME SPLICE-SITE USAGE4 sweepsmachine learning
96.1% (best in column)68.3%
SPLICEAImachine learning
95.3%94.5%
EVO2 40B (W/O HU. IDENT.)2 sweepsfoundation model
93.0%85.2%
BORZOI (32-BP)2 sweepsmachine learning
91.4%86.9%
NTV3 650M POST2 sweepsfoundation model
89.1%71.6%
GENOS 10B2 sweepsfoundation model
77.3%61.7%

R recall, P precision, Sp specificity.

Caveats and method

What each column measures:

  • OPA1 evaluation sets (thresholds tuned here): Accuracy, PPV, specificity, sensitivity and error rates of SpliceAI, AlphaGenome (version 2 rerun), Borzoi and the genomic foundation models NTv3 650M, Evo2 40B and Genos 10B on OPA1 LOVD evaluation sets (49 splicing, 79 benign)
  • Independent 65-gene test set: Accuracy, PPV, specificity, sensitivity and error rates of SpliceAI, AlphaGenome (version 2 rerun), Borzoi and the genomic foundation models NTv3 650M, Evo2 40B and Genos 10B on Multi-gene spliceogenic test set (76 spliceogenic, 107 benign, 65 genes)
  • In-sample: thresholds were optimised on these 128 variants, so the values are optimistic.
  • Positives are LOVD entries annotated as splicing defects; the assay behind each annotation is not stated.
  • Thresholds were set by one-dimensional grid search on the OPA1 evaluation sets, optimising accuracy and specificity (Methods B).
  • Single-author preprint; no uncertainty intervals are printed for the tabulated metrics.
  • Foundation-model scores are zero-shot log-likelihood ratios with author-chosen context windows, aggregation and strand rules (Methods F to H).
  • Preprint (bioRxiv versions 1 and 2), not peer reviewed; version 2 changed only the AlphaGenome rows.
  • AlphaGenome's three version 1 configurations stay excluded over the 1-base coordinate offset in their scripts (upstream issue #2). Its four configurations from version 2 of the preprint (posted 2026-10-31) were rerun with the corrected position and are included.
  • Spliceogenic positives span canonical ±1,2 (26 of 76), near-splice intronic positions from +3 to +7, -3 to -7 and -13 (35) and exonic substitutions (15), by the paper's Supplemental Table VI, although its text calls them canonical splice-site variants; benign controls are ClinVar intronic substitutions at ±3 to ±10 from 46 of the 64 gene symbols the table lists.
  • Positives were confirmed by RNA studies in the cited sources (references 13 and 14); this preprint does not restate the assay per variant.
  • Genos 10B ClinVar is fine-tuned on ClinVar labels and the benign controls are ClinVar variants; training overlap is not tested in the source.

Protocols: Splice-defect classification of OPA1 variants with thresholds tuned on the same sets (Alavi Table III, evaluation sets), Spliceogenic versus benign classification on an independent 65-gene set with OPA1-derived thresholds (Alavi Table IV)

Primary benchmark evaluation shown (Splice predictors and genomic foundation models on OPA1 and multi-gene splice variants (Alavi 2026)). 7 additional study protocols (DNA splice predictors against patient-RNA or exon-trapping results (Drost et al. 2025), RNA-seq callers on 16 known pathogenic splicing events in muscle (Segarra-Casas et al. 2025), Rank of known disease genes in 12 Kremer fibroblast samples (Segers et al. 2026) and others) are cited in individual tool drawer rows and under Question, sources and review.

Held for review

Recorded but not shown as evidence until the reason below is resolved.

RNA-seq tools on known diagnoses and true-negative samples, neuromuscular patient RNA-seq (Silverstein et al. 2026) (Expression-outlier caller)

Not shown: Table S7 true-negative rows conflict with Table S9 rows 7 and 8 for OUTRIDER and OUTRIDER + GTEx. Kept at needs_review by the independent review (docs/reviews/use-cases/splicing-cnv-use-cases-2026-10-12.md).

Not covered yet

  • No numeric confidence interval is reported for the 85% recovery figure in the inspected main text.
  • Subsampling repeat count and exact score-aggregation method are unextracted.
  • Feng et al. 2025 Table 1 splice-site classification is not linked to this use case: it measures AUC on benchmark splice-site sequences (DNABERT-2/NT datasets), not patient RNA, and is linked to 'Prioritise variants for splicing experiments'. The DNA foundation-model results linked here (Alavi et al. 2026: Evo 2 40B, Genos 10B and NTv3 650M) are threshold classifications of variants with established splicing defects, also proxies; none recovers events from patient RNA-seq.
  • AbSplice (Wagner et al. 2023, Nat Genet) reports its tissue-aware comparison with SpliceAI, MMSplice, SQUIRLS and CADD-Splice only in figures and prose in the copies screened, so no tissue-aware DNA predictor comparison is stored.
  • The FRASER 2.0 article (Scheller et al. 2023, AJHG) could not be retrieved as full text, and its comparison with FRASER, LeafCutterMD and SPOT on patient cohorts was not extracted.
  • No table comparing MMSplice, CADD-Splice or AbSplice against RNA-validated variants was found in this pass; Drost et al. cover SpliceAI, Pangolin, SPiP and SQUIRLS only.
  • Drost et al. variant-type strata (Data S1 Tables S5 and S6) are not extracted.
  • False positives are tabulated only as true-negative samples with any outlier (Silverstein et al. 2026, Table S9); outlier counts per sample remain figure-only.
  • Minigene or MFASS-style proxies beyond the existing MFASS mapping were not added in this pass.
  • MAJIQ-CLIN (Genet Med 2026) compares splicing outlier tools but could not be retrieved as full text.
  • Silverstein et al. 2026 expression-outlier true-negative counts conflict between Tables S7 and S9, so the OUTRIDER rows are not linked.
  • No machine-learning method has been compared with FRASER or OUTRIDER on the same patient RNA data in the linked evidence.

Contribute evidence or propose a correction

How to cite

Select a patient-RNA splicing-variant validation workflow. Rewire biological model benchmark database, release 2026-11-21-c8bea25e5abb. https://benchmarks.rewirebio.io/use-cases/patient-rna-splicing-validation/

Release 2026-11-21-c8bea25e5abb. Rankings and values change between releases, so keep the release id in the citation. Cite the original publication as well for any value you reuse: each result links its source.

The compilation and annotations made by Rewire in this release are offered under CC BY 4.0; values and text taken from cited sources stay under the terms of those sources and are not covered by this licence. Data licence

Question, sources and review

Who this is for, setting and exclusions

Inspect the FRASER known pathogenic-event recovery curve and the sequence-classification proxy evidence before selecting a workflow and designing patient-RNA follow-up for the intended cohort size.

Who this is for
  • Clinical researchers scoping the available diagnostic-genomics evidence
  • Computational researchers comparing exact evaluated configurations
Biological setting
FRASER, applied to a retrospective Kremer rare-mitochondrial-disorder skin-fibroblast RNA cohort (119 samples/105 individuals, 13 known pathogenic splicing events), recovers a mean 85% (11 of 13) of known events at a reduced 30-sample subsample. Separate, proxy sequence-classification results for DNA foundation models (Alavi et al. 2026: Evo 2 40B, Genos 10B and NTv3 650M classifying variants with established splicing defects against benign variants) are also linked, scoped as a different endpoint. Feng et al. 2025 Table 1 splice acceptor and donor classification is linked to 'Prioritise variants for splicing experiments', not here.
Clinical research scope
Clinical applicability is not established beyond known-event recovery. This does not demonstrate prospective diagnostic yield, blinded adjudication, or general tissue transfer.

Outside this use case

  • Diagnostic yield in patients without a known pathogenic event (this is recovery of already-known events, not discovery)
  • Full-cohort outlier-gene workload counts (12/7/10 genes/sample), a different population from the 30-sample recovery evaluation and not ingested here
  • Blinded pathogenicity adjudication or VUS reclassification
Evidence collection plan
Evidence collection planCollecting evidence

Already on record: 8 comparisons shown · 1 held for review · 24 tools, in the evaluated evidence above. The status above describes only the specific comparison in this plan, which remains open; it does not mean no evidence has been collected. The plan defines a comparison to investigate; it does not establish model performance or suitability.

Comparison question

Which DNA/RNA evidence workflow identifies splice-altering variants for inherited-disorder follow-up when patient RNA and tissue-specific context are available?

Baselines, outcomes and validation requirements

Baselines to include

  • The conventional/author-introduced workflow measured in the linked primary source(s).

Outcomes to measure

  • The declared endpoint in this use case's active mapping(s); see evidence_gaps for what remains open.

Validation requirements

  • Independent held-out population matched to the intended clinical setting.
  • Qualified human scientific review before any clinical-validation claim.

Next collection task

A dedicated rewire-benchmarks protocol/run task with a prospective, blinded patient-RNA cohort including cases without a pre-known pathogenic event.

Sources, review and downloads

Automated source review · 2026-10-07 · Claude Sonnet AMP-integration worker, bounded transcription of Codex-checked primary values; independently reviewed by Codex (workbench/amp-supervision/primary-review.md, integration-review-corrections.md)

Bounded primary-source transcription, independently Codex-checked. No new model execution, independent experimental reproduction, qualified human scientific review or clinical validation.

Release 2026-11-21-c8bea25e5abb · question use-case-patient-rna-splicing-validation

Use-case input digest bfbb7ab9303becad1e3cb395568c26f41e72ddb67cd1a89a06d9eaa0f76bd95f

Download questions and review metadata (JSON) (gzip) · Verify release checksums (gzip)