rewirebio.iobenchmarks
Evaluation

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

Published comparison; transcribed, not reproduced.

Research readiness

These checks assess whether the evidence supports a reproducible investigation. A source-checked score alone does not meet these requirements.

Release 2026-10-09-ba02f2f4a36e · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • metric replay: verification is missing

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • metric replay: verification is missing
  • annotations: verification is missing
  • dependence: verification is missing

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • recipe pinned: verification is missing
  • resource estimate: verification is missing

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • artifact hashes: verification is missing
  • join integrity: verification is missing
  • score semantics: verification is missing
  • independent validation: verification is missing
  • overlap checked: verification is missing

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Read reviewed discrepancy investigations

Evaluation results

1 evaluation · 6 results. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: At least 1 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset: CAGI6 Splicing VUS variants as scored by Drost et al.
0.854 f1-score
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

rna-splicing-20261009-protocol-drost2025-cagi6

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'CAGI6 dataset', cell F25; row AtLeast1, column F1
Configuration: At least 1 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset: CAGI6 Splicing VUS variants as scored by Drost et al.
0.818 negative-predictive-value
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

rna-splicing-20261009-protocol-drost2025-cagi6

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'CAGI6 dataset', cell H25; row AtLeast1, column Neg Pred Value
Configuration: At least 1 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset: CAGI6 Splicing VUS variants as scored by Drost et al.
0.778 precision
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

rna-splicing-20261009-protocol-drost2025-cagi6

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'CAGI6 dataset', cell G25; row AtLeast1, column Pos Pred Value
Configuration: At least 1 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset: CAGI6 Splicing VUS variants as scored by Drost et al.
0.778 precision
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

rna-splicing-20261009-protocol-drost2025-cagi6

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'CAGI6 dataset', cell E25; row AtLeast1, column Precision
Configuration: At least 1 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset: CAGI6 Splicing VUS variants as scored by Drost et al.
0.946 recall
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

rna-splicing-20261009-protocol-drost2025-cagi6

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'CAGI6 dataset', cell C25; row AtLeast1, column Sensitivity (Recall)
Configuration: At least 1 of 4 splice predictors (Drost et al.)Protocol: Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset: CAGI6 Splicing VUS variants as scored by Drost et al.
0.474 specificity
fraction · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

At least 1 of 4 splice predictors on 56 CAGI6 Splicing VUS variants

rna-splicing-20261009-protocol-drost2025-cagi6

Aggregation: Not reported

Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools; Drost et al. 2025, Data S1 (Tables S1-S6) · Data S1 Table S4 'CAGI6 dataset', cell D25; row AtLeast1, column Specificity

Source checking is not independent reproduction. Release 2026-10-09-ba02f2f4a36e.

Evaluation procedure

rna-splicing-20261009-protocol-drost2025-cagi6

Configuration
At least 1 of 4 splice predictors (Drost et al.)
Protocol
Splice-effect prediction against patient-RNA or exon-trapping results, CAGI6 Splicing VUS subset (Drost et al. Data S1)
Dataset
CAGI6 Splicing VUS variants as scored by Drost et al.
origin
Author-reported evaluation
configuration
Primary source as retrieved 2026-10-09
dataset version
Data S1 Table S2 variant set (as published)
split
No training split
inputs
Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors
adaptation
None; pretrained predictors with literature thresholds
metric implementation
R: plotROC 2.3.1 (ROC), yardstick 1.2.0 (PR), caret 6.0.94 (binary statistics)
aggregation
Pooled over variants
budget
Not reported
population
56 CAGI6 Splicing VUS variants
protocol id
rna-splicing-20261009-protocol-drost2025-cagi6

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split
No training split
Adaptation
None; pretrained predictors with literature thresholds
Scoring implementation
R: plotROC 2.3.1 (ROC), yardstick 1.2.0 (PR), caret 6.0.94 (binary statistics)

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

38 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-09-ba02f2f4a36e
Property and statementOriginal source and locationReview and provenance
attributes.comparison.adaptation
None; pretrained predictors with literature thresholds
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.adaptation

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.adaptation
None; pretrained predictors with literature thresholds
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.adaptation

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.aggregation
Pooled over variants
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.aggregation

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.aggregation
Pooled over variants
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.aggregation

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.budget
Not reported
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

missing or unspecified

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.budget

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.budget
Not reported
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

missing or unspecified

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.budget

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.dataset_version
Data S1 Table S2 variant set (as published)
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.dataset_version

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.dataset_version
Data S1 Table S2 variant set (as published)
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.dataset_version

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

attributes.comparison.inputs
Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors
Context-only references
Routine RNA-based analysis of potential splicing variants facilitates genomic diagnostics and reveals limitations of in silico prediction tools

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: HGG Advances 7(1):100521, published online 2025-09-22; PMC12547740 full-text XML
Retrieved: 2026-10-09T20:30:41Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.inputs

Source artifact SHA-256: 2a2e970526e4348d505d26356b830d96da9e32688c4de84841ac8137b62d5d32

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

attributes.comparison.inputs
Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors
Context-only references
Drost et al. 2025, Data S1 (Tables S1-S6)

Original source ↗

CAGI6 dataset block of Data S1 Table S4

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: mmc2.xlsx (Data S1. Tables S1-S6) inside the Europe PMC supplementaryFiles zip for PMC12547740
Retrieved: 2026-10-09T20:31:37Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.inputs

Source artifact SHA-256: a3a69202b8f0d9ecb7fa22a16991d5e4d583b5ae72fd598206ea5c2b4c5c14ca

Hash scope: SHA-256 of mmc2.xlsx as extracted from the supplementaryFiles zip (zip SHA-256 1c89a7ebc0bf686c6087d0fee86ba6f364a6d754d5bc7eb9c446b7db017f1ebc; the zip is re-built by Europe PMC on each request, so only the member hash is stable).

Inspected artifact

Sources and history

Release 2026-10-09-ba02f2f4a36e · Record review: source checked

2 source records and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: rna-splicing-20261009-eval-drost2025-cagi6-al1

areas
dna-genomes
contexts
clinical_research
origin
author_reported
protocol
rna-splicing-20261009-protocol-drost2025-cagi6
version
Primary source as retrieved 2026-10-09
comparison
dataset version: Data S1 Table S2 variant set (as published); split: No training split; inputs: Variant (GRCh37/38 coordinates as annotated by each tool); no patient RNA input to the predictors; adaptation: None; pretrained predictors with literature thresholds; metric implementation: R: plotROC 2.3.1 (ROC), yardstick 1.2.0 (PR), caret 6.0.94 (binary statistics); aggregation: Pooled over variants; budget: Not reported; population: 56 CAGI6 Splicing VUS variants; protocol id: rna-splicing-20261009-protocol-drost2025-cagi6
source locator
CAGI6 dataset block of Data S1 Table S4
limitations
Consensus rule devised and evaluated by the same authors on the same variants.
Related records

Suggest a correction