rewirebio.iobenchmarks
Evaluation

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

Published comparison; transcribed, not reproduced.

Research readiness

These checks assess whether the evidence supports a reproducible investigation. A source-checked score alone does not meet these requirements.

Release 2026-10-09-ba02f2f4a36e · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • metric replay: verification is missing

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • metric replay: verification is missing
  • annotations: verification is missing
  • dependence: verification is missing

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • recipe pinned: verification is missing
  • resource estimate: verification is missing

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • independent validation: verification is missing
  • overlap checked: verification is missing

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Read reviewed discrepancy investigations

Evaluation results

1 evaluation · 6 results. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: At least 2 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset: Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
0.885 f1-score
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

rna-splicing-20261009-protocol-drost2025-merged-243

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'Entire Dataset', cell F13; row AtLeast2, column F1
Configuration: At least 2 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset: Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
0.831 negative-predictive-value
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

rna-splicing-20261009-protocol-drost2025-merged-243

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'Entire Dataset', cell H13; row AtLeast2, column Neg Pred Value
Configuration: At least 2 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset: Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
0.855 precision
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

rna-splicing-20261009-protocol-drost2025-merged-243

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'Entire Dataset', cell G13; row AtLeast2, column Pos Pred Value
Configuration: At least 2 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset: Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
0.855 precision
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

rna-splicing-20261009-protocol-drost2025-merged-243

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'Entire Dataset', cell E13; row AtLeast2, column Precision
Configuration: At least 2 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset: Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
0.916 recall
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

rna-splicing-20261009-protocol-drost2025-merged-243

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'Entire Dataset', cell C13; row AtLeast2, column Sensitivity (Recall)
Configuration: At least 2 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset: Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
0.727 specificity
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 2 of 4 splice predictors on 243 scorable variants (in-house plus CAGI6)

rna-splicing-20261009-protocol-drost2025-merged-243

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'Entire Dataset', cell D13; row AtLeast2, column Specificity

Source checking is not independent reproduction. Release 2026-10-09-ba02f2f4a36e.

Evaluation procedure

rna-splicing-20261009-protocol-drost2025-merged-243

Configuration
At least 2 of 4 splice predictors (Drost et al.)
Protocol
Splice-effect prediction against patient-RNA or exon-trapping results, all 243 scorable variants (Drost et al. Data S1)
Dataset
Drost et al. RNA-tested clinical variants plus CAGI6 Splicing VUS, 243 scorable
origin
Author-reported evaluation
configuration
Primary source as retrieved 2026-10-09
dataset version
Data S1 Table S2 variant set (as published)
split
No training split
inputs
Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors
adaptation
None; pretrained predictors with literature thresholds
metric implementation
R: plotROC 2.3.1 (ROC), yardstick 1.2.0 (PR), caret 6.0.94 (binary statistics)
aggregation
Pooled over variants
budget
Not reported
population
243 scorable variants (in-house plus CAGI6)
protocol id
rna-splicing-20261009-protocol-drost2025-merged-243

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split
No training split
Adaptation
None; pretrained predictors with literature thresholds
Scoring implementation
R: plotROC 2.3.1 (ROC), yardstick 1.2.0 (PR), caret 6.0.94 (binary statistics)

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

38 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-09-ba02f2f4a36e
Property and statementOriginal source and locationReview and provenance
attributes.comparison.adaptation
None; pretrained predictors with literature thresholds
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.adaptation

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.adaptation
None; pretrained predictors with literature thresholds
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.adaptation

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.aggregation
Pooled over variants
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.aggregation

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.aggregation
Pooled over variants
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.aggregation

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.budget
Not reported
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

missing or unspecified

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.budget

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.budget
Not reported
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

missing or unspecified

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.budget

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.dataset_version
Data S1 Table S2 variant set (as published)
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.dataset_version

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.dataset_version
Data S1 Table S2 variant set (as published)
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.dataset_version

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.inputs
Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.inputs

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.inputs
Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

Entire Dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.inputs

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

Sources and history

Release 2026-10-09-ba02f2f4a36e · Record review: source checked

2 source records and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: rna-splicing-20261009-eval-drost2025-merged-al2

areas
dna-genomes
contexts
clinical_research
origin
author_reported
protocol
rna-splicing-20261009-protocol-drost2025-merged-243
version
Primary source as retrieved 2026-10-09
comparison
dataset version: Data S1 Table S2 variant set (as published); split: No training split; inputs: Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors; adaptation: None; pretrained predictors with literature thresholds; metric implementation: R: plotROC 2.3.1 (ROC), yardstick 1.2.0 (PR), caret 6.0.94 (binary statistics); aggregation: Pooled over variants; budget: Not reported; population: 243 scorable variants (in-house plus CAGI6); protocol id: rna-splicing-20261009-protocol-drost2025-merged-243
source locator
Entire Dataset block of Data S1 Table S4
limitations
Consensus rule devised and evaluated by the same authors on the same variants.
Related records

Suggest a correction