Relevance of Splice-altering variant classification, SPiP benchmark variant set (99,601 variants) (Fortier et al. Table 3) to "Select a patient-RNA splicing-variant validation workflow"
- Use case
- Select a patient-RNA splicing-variant validation workflow
- Assessed by
- Splice-altering variant classification, SPiP benchmark variant set (99,601 variants) (Fortier et al. Table 3)
- Relevance
- Proxy evidence
- Endpoint
- Sensitivity, specificity, F1 and balanced accuracy of SpliceAI, CI-SpliceAI, OpenSpliceAI and a legacy N-of-4 vote on the SPiP benchmark variant set (99,601 variants)
Proxy: DNA-based splice predictors scored against experimentally established splicing outcomes, the sequence-classification kind of evidence the use case keeps separate from patient-RNA recovery; inputs are variants, not patient RNA-seq.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
19 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group fortier2026-spliceai-reimplementations Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title SpliceAI and its open-source reimplementations on six splice benchmarks (Fortier et al. 2026) Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints2 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Sensitivity, specificity, F1 and balanced accuracy of SpliceAI, CI-SpliceAI, OpenSpliceAI and a legacy N-of-4 vote on the SPiP benchmark variant set (99,601 variants) Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:rna-splicing-20261011-fortier2026-protocol-spip Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric balanced-accuracy Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations4 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records23 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale Proxy: DNA-based splice predictors scored against experimentally established splicing outcomes, the sequence-classification kind of evidence the use case keeps separate from patient-RNA recovery; inputs are variants, not patient RNA-seq. Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
1 source record and release history
- Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) · Original source · PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML
Technical metadata and extraction receipts
Stable ID: use-case-mapping-rna-splicing-20261011-fortier2026-spip
- field
- links:assessed_by:rna-splicing-20261011-fortier2026-protocol-spip
- value
- rna-splicing-20261011-fortier2026-protocol-spip
- relevance
- proxy
- endpoint
- Sensitivity, specificity, F1 and balanced accuracy of SpliceAI, CI-SpliceAI, OpenSpliceAI and a legacy N-of-4 vote on the SPiP benchmark variant set (99,601 variants)
- rationale
- Proxy: DNA-based splice predictors scored against experimentally established splicing outcomes, the sequence-classification kind of evidence the use case keeps separate from patient-RNA recovery; inputs are variants, not patient RNA-seq.
- constraints
- Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; variant sets, labels and thresholds differ.
- limitations
- Labels and variant sets are reused from earlier publications. CI-SpliceAI was trained on all human chromosomes, so no benchmark variant is held out from its training; ABCA4 lies on a test chromosome held out by both SpliceAI and OpenSpliceAI (Methods 'Technical overview of deep learning methods'; Discussion).; No uncertainty intervals are printed; significance is from two-proportion Z-tests on accuracy in the Results.; Independent of the SpliceAI, CI-SpliceAI and OpenSpliceAI developers; the authors' employer supplies the Legacy Ensemble software, so those rows are author_reported.; Variant-level splicing effect, not pathogenicity or diagnostic yield; tissue context is not modelled.
- citation locators
- source id: rna-splicing-20261011-fortier2026-source-article; locator: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3
- source locator
- rna-splicing-20261011-fortier2026-source-article: Methods 'SPiP benchmark dataset'; Results 'SPiP benchmark results'; Table 3
- headline metric
- balanced-accuracy
- comparison group
- fortier2026-spliceai-reimplementations
- comparison title
- SpliceAI and its open-source reimplementations on six splice benchmarks (Fortier et al. 2026)
- stratum label
- SPiP benchmark variant set (99,601 variants)
- stratum order
- 3
- reason
- Recorded from the 2026-10-11 use-case literature pass (data/omics/use-case-coverage-rna-splicing-20261011-fortier2026). Independently reviewed in docs/reviews/use-cases/splicing-cnv-use-cases-2026-10-12.md.
- review
- method: source-hash-verification; independent-cell-check; automated-source-review; scope-review; reviewer: claude; reviewer note: Independent Claude review agent reviewer-scnv; did not extract this batch; no human review claimed; date: 2026-10-12; artifact sha256: dffe74cee7782cbdfdbc9423cf00f0a846b3d720b90b10e9732d6f768c485176; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13170886/fullTextXML; method note: Drafted by the collector agent uc-splicing-cnv from the pinned source.; note: Checked by reviewer-scnv against the pinned source; verdict in docs/reviews/use-cases/splicing-cnv-use-cases-2026-10-12.md and in this batch's review.md.
- reviewed evaluations
- rna-splicing-20261011-fortier2026-eval-spip-ci-spliceai; rna-splicing-20261011-fortier2026-eval-spip-legacy-ensemble; rna-splicing-20261011-fortier2026-eval-spip-openspliceai; rna-splicing-20261011-fortier2026-eval-spip-spliceai
- pins
- record id: rna-splicing-20261011-fortier2026-eval-spip-ci-spliceai; fields: status; links; attributes.origin; attributes.comparison; sha256: de341050f809e2b242d4c0f43b75ac82d312ae173f522d877f2b12616ebc9bca; record id: rna-splicing-20261011-fortier2026-eval-spip-legacy-ensemble; fields: status; links; attributes.origin; attributes.comparison; sha256: e445080cf7fa17ab291bf4fb157fa792456a32d0159ecc5a1a8c98b4b1baa0bb; record id: rna-splicing-20261011-fortier2026-eval-spip-openspliceai; fields: status; links; attributes.origin; attributes.comparison; sha256: 50c16188f689eee0dbcfc7b6b55a9786be600e7e7836369017f0a6e354013568; record id: rna-splicing-20261011-fortier2026-eval-spip-spliceai; fields: status; links; attributes.origin; attributes.comparison; sha256: 0555f4d323df304bd87fc78e0f02aa3a7af26d801d8d7d050bd9de65fb49793d; record id: rna-splicing-20261011-fortier2026-protocol-spip; fields: status; name; description; facets; source_ids; links; attributes; sha256: 7f9b7f319a705291eadf779c78c490b47a26e5f22d3628fba96a79202f1216ba; record id: rna-splicing-20261011-fortier2026-result-spip-ci-spliceai-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6fdabc6259abfd92ba4f3688dc9ef320fd7b845309e98e513004f4a06669693c; record id: rna-splicing-20261011-fortier2026-result-spip-ci-spliceai-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0c051a3d4aedf444d55174e390b6b3ea5548c447d5f99e9aa59138b4e3d53116; record id: rna-splicing-20261011-fortier2026-result-spip-ci-spliceai-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e9033e17716462dd93ca7fcdb08946240b96575e4e5b88ff48fb2b8230c669d9; record id: rna-splicing-20261011-fortier2026-result-spip-ci-spliceai-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b44f016ba759ae4473fae46741ba0b751dd9810f8fc52d73dcc5f2cf851aa38; record id: rna-splicing-20261011-fortier2026-result-spip-legacy-ensemble-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d60f17d8ac94b1e7af8e89ee1240fe7c53213e543803d2ff580f037cc0fda4a9; record id: rna-splicing-20261011-fortier2026-result-spip-legacy-ensemble-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3a50ea4e7f1994b3146b5ad88cfc0815f50d9019874117d861e5745228ce0ce3; record id: rna-splicing-20261011-fortier2026-result-spip-legacy-ensemble-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: fad024226b84287b97ff3ef84e8dd965fec53073fafe2c5f65e91d88ee846746; record id: rna-splicing-20261011-fortier2026-result-spip-legacy-ensemble-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 383b08f1fd79e02c6ca0d9a219d08461b5d6924d845ac9b1c2c67db4eaa9bda0; record id: rna-splicing-20261011-fortier2026-result-spip-openspliceai-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 823e13b2faeb7b0456337e2b92f7c50792c4e27a03e7a44e07c59e33f75a3e66; record id: rna-splicing-20261011-fortier2026-result-spip-openspliceai-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e1c807046618f241a129eeaa70c38f0f5c8dec4e122dc54d3ea97ebcbaea9406; record id: rna-splicing-20261011-fortier2026-result-spip-openspliceai-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 737e0719088588abf3bfde5b72dc684d8f83309bac2666b8d39d7800524ff2c2; record id: rna-splicing-20261011-fortier2026-result-spip-openspliceai-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b44f016ba759ae4473fae46741ba0b751dd9810f8fc52d73dcc5f2cf851aa38; record id: rna-splicing-20261011-fortier2026-result-spip-spliceai-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 52f252327a6fdad5a3d4aeff29a3fb5f04bcb34854261e12742dad7efb935302; record id: rna-splicing-20261011-fortier2026-result-spip-spliceai-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: acf956306adc1a503e9b241d7256a6f3b621c7ec5069dea3c495311a9f07b572; record id: rna-splicing-20261011-fortier2026-result-spip-spliceai-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 096224d9845f1806f2b08377c0ea74c6d172899c56beede444dc38c1f7625c07; record id: rna-splicing-20261011-fortier2026-result-spip-spliceai-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b44f016ba759ae4473fae46741ba0b751dd9810f8fc52d73dcc5f2cf851aa38; record id: rna-splicing-20261011-fortier2026-source-article; fields: status; attributes.artifact_sha256; sha256: f632aac0a971c54f908ad18bc9965d498c05b69bd07552d9915807db55a22233; record id: use-case-patient-rna-splicing-validation; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: b85cad56e15fbfad66a2e70c3b193ed56c464e5e44b62a19b12f391e409ec897