Relevance of Splice-altering variant classification, CI-SpliceAI benchmark variant set (1,316 variants) (Fortier et al. Table 1) to "Select a patient-RNA splicing-variant validation workflow"
- Use case
- Select a patient-RNA splicing-variant validation workflow
- Assessed by
- Splice-altering variant classification, CI-SpliceAI benchmark variant set (1,316 variants) (Fortier et al. Table 1)
- Relevance
- Proxy evidence
- Endpoint
- Sensitivity, specificity, F1 and balanced accuracy of SpliceAI, CI-SpliceAI, OpenSpliceAI and a legacy N-of-4 vote on the CI-SpliceAI benchmark variant set (1,316 variants)
Proxy: DNA-based splice predictors scored against experimentally established splicing outcomes, the sequence-classification kind of evidence the use case keeps separate from patient-RNA recovery; inputs are variants, not patient RNA-seq.
4 reviewed evaluations
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
19 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group fortier2026-spliceai-reimplementations Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title SpliceAI and its open-source reimplementations on six splice benchmarks (Fortier et al. 2026) Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints2 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Sensitivity, specificity, F1 and balanced accuracy of SpliceAI, CI-SpliceAI, OpenSpliceAI and a legacy N-of-4 vote on the CI-SpliceAI benchmark variant set (1,316 variants) Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:rna-splicing-20261011-fortier2026-protocol-ci-spliceai-benchmark Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric balanced-accuracy Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations5 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records23 values
| Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale Proxy: DNA-based splice predictors scored against experimentally established splicing outcomes, the sequence-classification kind of evidence the use case keeps separate from patient-RNA recovery; inputs are variants, not patient RNA-seq. Context-only references | Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1 Version: PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
1 source record and release history
- Analyzing the performance of deep learning splice prediction algorithms (Fortier, Rudy and Scherer, PLOS One 2026) · Original source · PLOS One 21(5):e0348885, published 2026-05-13; PMC13170886 full-text XML
Technical metadata and extraction receipts
Stable ID: use-case-mapping-rna-splicing-20261011-fortier2026-ci-spliceai-benchmark
- field
- links:assessed_by:rna-splicing-20261011-fortier2026-protocol-ci-spliceai-benchmark
- value
- rna-splicing-20261011-fortier2026-protocol-ci-spliceai-benchmark
- relevance
- proxy
- endpoint
- Sensitivity, specificity, F1 and balanced accuracy of SpliceAI, CI-SpliceAI, OpenSpliceAI and a legacy N-of-4 vote on the CI-SpliceAI benchmark variant set (1,316 variants)
- rationale
- Proxy: DNA-based splice predictors scored against experimentally established splicing outcomes, the sequence-classification kind of evidence the use case keeps separate from patient-RNA recovery; inputs are variants, not patient RNA-seq.
- constraints
- Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; variant sets, labels and thresholds differ.
- limitations
- Labels and variant sets are reused from earlier publications. CI-SpliceAI was trained on all human chromosomes, so no benchmark variant is held out from its training; ABCA4 lies on a test chromosome held out by both SpliceAI and OpenSpliceAI (Methods 'Technical overview of deep learning methods'; Discussion).; No uncertainty intervals are printed; significance is from two-proportion Z-tests on accuracy in the Results.; Independent of the SpliceAI, CI-SpliceAI and OpenSpliceAI developers; the authors' employer supplies the Legacy Ensemble software, so those rows are author_reported.; Variant-level splicing effect, not pathogenicity or diagnostic yield; tissue context is not modelled.; The variant set was assembled and published by the CI-SpliceAI developers (reference 9).
- citation locators
- source id: rna-splicing-20261011-fortier2026-source-article; locator: Methods 'CI-SpliceAI benchmark dataset'; Table 1
- source locator
- rna-splicing-20261011-fortier2026-source-article: Methods 'CI-SpliceAI benchmark dataset'; Table 1
- headline metric
- balanced-accuracy
- comparison group
- fortier2026-spliceai-reimplementations
- comparison title
- SpliceAI and its open-source reimplementations on six splice benchmarks (Fortier et al. 2026)
- stratum label
- CI-SpliceAI benchmark variant set (1,316 variants)
- stratum order
- 1
- reason
- Recorded from the 2026-10-11 use-case literature pass (data/omics/use-case-coverage-rna-splicing-20261011-fortier2026). Independently reviewed in docs/reviews/use-cases/splicing-cnv-use-cases-2026-10-12.md.
- review
- method: source-hash-verification; independent-cell-check; automated-source-review; scope-review; reviewer: claude; reviewer note: Independent Claude review agent reviewer-scnv; did not extract this batch; no human review claimed; date: 2026-10-12; artifact sha256: dffe74cee7782cbdfdbc9423cf00f0a846b3d720b90b10e9732d6f768c485176; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13170886/fullTextXML; method note: Drafted by the collector agent uc-splicing-cnv from the pinned source.; note: Checked by reviewer-scnv against the pinned source; verdict in docs/reviews/use-cases/splicing-cnv-use-cases-2026-10-12.md and in this batch's review.md.
- reviewed evaluations
- rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-ci-spliceai; rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-legacy-ensemble; rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-openspliceai; rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-spliceai
- pins
- record id: rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-ci-spliceai; fields: status; links; attributes.origin; attributes.comparison; sha256: 083e5f7cbda0da3776540666a18fd3bab881aecfb0d541937b327550d973dbdc; record id: rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-legacy-ensemble; fields: status; links; attributes.origin; attributes.comparison; sha256: c485a1bd5f158f273c49672bf4c17d292d05ea39c7a68cd1cf9319d066f7cf88; record id: rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-openspliceai; fields: status; links; attributes.origin; attributes.comparison; sha256: 62e1a41cb77390ff337965bdf65819ec4c396ffd7bd2ff074e956e9cc7eada41; record id: rna-splicing-20261011-fortier2026-eval-ci-spliceai-benchmark-spliceai; fields: status; links; attributes.origin; attributes.comparison; sha256: d122ffcf7d818eee118a391af5288dcaa326e9bdd85a0654b0d4aaadfb4ee8cf; record id: rna-splicing-20261011-fortier2026-protocol-ci-spliceai-benchmark; fields: status; name; description; facets; source_ids; links; attributes; sha256: 8334450a9ff18062dbb562871b38b45a8b454636a2ef121112a7dd64b8a16bd5; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-ci-spliceai-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b1a6eb8b25c4d560a8c75283333c40d3a86459928acbd67a2fe7743a79fae564; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-ci-spliceai-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9cae415b4a32cd2d03c85eafc3f070ff7d5e500dc55d7f4872240dbb17bd3362; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-ci-spliceai-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4cc37f4e6bc77564766c43189933c450eb8841cb9b55e20134b9b103050392c9; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-ci-spliceai-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 10888f445cffb8f8a8300557db3ae7db5e07ee75f6bad115d556e934b87b347c; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-legacy-ensemble-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d556c7f308b12abfb6c36aa42e9f6fbcaa128e7f0636d3b0cd698246a27ebae7; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-legacy-ensemble-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 402c93f40773a56d237df350d7358dc7e68e66fde7bf305d6e1fabf4204bf2f2; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-legacy-ensemble-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 35fa1d9688155ee48dcd14da71479543b71d5d28542aba9b84f6c0b9bedb6408; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-legacy-ensemble-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 1ed4b7beaea43870a220d6ad46c2a6c253d05b1d4272a753acae0deadcc5ec2b; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-openspliceai-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 85ce5a561faf88d08af48c00af1551efb6e66f1bc56188b99fd2b383e00cc7aa; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-openspliceai-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 85a74b833ba61005fe88920d3cbb9995d2390042a72f0ff843e985a2946b33bd; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-openspliceai-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 28f434ff9519eb2b43e68fa54264b61ec25cc3814b497cc2ccf416b61351bc15; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-openspliceai-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 27876fce6b6ba540255af818c645b9795f8c0641f8e879e1ced4749ad4240e49; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-spliceai-balanced-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d07724b6d86314c3db61a437c29a2489f5be673489cbb5d54063de3c6f8edf13; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-spliceai-f1-score; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0f055e6fc736d3fc86640720cef7da12a4af7d9e9e8dcd354e84802167f744c0; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-spliceai-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ebcbe0e4651d31b497d9fd632e41b577fc12a029642bc47a84c2055ab548b8ea; record id: rna-splicing-20261011-fortier2026-result-ci-spliceai-benchmark-spliceai-specificity; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 383b08f1fd79e02c6ca0d9a219d08461b5d6924d845ac9b1c2c67db4eaa9bda0; record id: rna-splicing-20261011-fortier2026-source-article; fields: status; attributes.artifact_sha256; sha256: f632aac0a971c54f908ad18bc9965d498c05b69bd07552d9915807db55a22233; record id: use-case-patient-rna-splicing-validation; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: b85cad56e15fbfad66a2e70c3b193ed56c464e5e44b62a19b12f391e409ec897