36,034 pathogenic and 345 benign stop-gain variants with at least one ClinVar gold star and RefSeq exon annotations, split by whether the 50 bp rule predicts nonsense-mediated decay.
4 evaluations · 4 results. Different protocols are not a single leaderboard.
Applied filters: All linked evaluations
Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.
0 of 4 readiness checks met. These checks assess whether the evidence supports a reproducible investigation; a source-checked score alone does not meet them.
Release 2026-10-22-1b7cf027e4f6 · Evidence verified: Not verified
Exact outcomes, predictions, identifiers and evaluator are connected.
Missing or unresolved evidence
Verified: Not verified
Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.
Missing or unresolved evidence
Verified: Not verified
A pinned recipe describes the inputs, environment and resource requirements.
Missing or unresolved evidence
Verified: Not verified
Separate data and exposure records support an independent test.
Missing or unresolved evidence
Verified: Not verified
Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.
No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.
A dataset supplies biological observations. The evaluation protocol defines how those observations are split, used and scored.
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
16 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Denominator warning The Methods ('AUC metrics for pathogenicity classification', stop-gain bootstrap paragraph) give 32,441 pathogenic NMD variants and 36,113 pathogenic overall; the Fig. 6d legend and the Supplementary Methods give 32,362 and 36,034. Review 2026-10-14: each workbook stop-gain value times twice the product of the class counts printed in the Methods (3,672 x 147 without NMD; 32,441 x 198 with NMD; 36,113 x 345 overall) is an integer, as a ROC-AUC over the whole set is; with the legend counts (32,362 and 36,034 pathogenic) it is not, and a mean of 20 bootstrap ROC-AUCs on 140 + 140 variants would not be. The workbook values therefore appear to be whole-set ROC-AUCs on the Methods counts, not the plotted bootstrap means; the source does not say so. Context-only references | Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-pmc10484790.xml.gz |
| Denominator warning The Methods ('AUC metrics for pathogenicity classification', stop-gain bootstrap paragraph) give 32,441 pathogenic NMD variants and 36,113 pathogenic overall; the Fig. 6d legend and the Supplementary Methods give 32,362 and 36,034. Review 2026-10-14: each workbook stop-gain value times twice the product of the class counts printed in the Methods (3,672 x 147 without NMD; 32,441 x 198 with NMD; 36,113 x 345 overall) is an integer, as a ROC-AUC over the whole set is; with the legend counts (32,362 and 36,034 pathogenic) it is not, and a mean of 20 bootstrap ROC-AUCs on 140 + 140 variants would not be. The workbook values therefore appear to be whole-set ROC-AUCs on the Methods counts, not the plotted bootstrap means; the source does not say so. Context-only references | Brandes et al. 2023 Supplementary Information (Supplementary Table 1 and Methods) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-supplementary-information.pdf.gz |
| Denominator 36379 Context-only references | Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-pmc10484790.xml.gz |
| Denominator 36379 Context-only references | Brandes et al. 2023 Supplementary Information (Supplementary Table 1 and Methods) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-supplementary-information.pdf.gz |
| Population ClinVar stop-gain variants with at least one gold star; 110 without exon annotations removed Context-only references | Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-pmc10484790.xml.gz |
| Population ClinVar stop-gain variants with at least one gold star; 110 without exon annotations removed Context-only references | Brandes et al. 2023 Supplementary Information (Supplementary Table 1 and Methods) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-supplementary-information.pdf.gz |
| Source location Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Context-only references | Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-pmc10484790.xml.gz |
| Source location Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Context-only references | Brandes et al. 2023 Supplementary Information (Supplementary Table 1 and Methods) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-supplementary-information.pdf.gz |
| Split Whole labelled set; no training on labels Context-only references | Genome-wide prediction of disease variant effects with a deep protein language model (Brandes et al., Nature Genetics 2023) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-pmc10484790.xml.gz |
| Split Whole labelled set; no training on labels Context-only references | Brandes et al. 2023 Supplementary Information (Supplementary Table 1 and Methods) Supplementary Methods 'Indels and stop gain ClinVar benchmarks' paragraph 2; Fig. 6d legend; Methods 'AUC metrics for pathogenicity classification', paragraph on bootstrapping the stop-gain ROC-AUC Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Version of record, Nature Genetics 55, 1512-1522 (published 2023-08-10) | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/brandes2023-supplementary-information.pdf.gz |
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
Stable ID: gfm-guides-20261014-esm1b-data-clinvar-stop-gains