1.23 benignity-likelihood-ratio
primateAI_b BRCA2 likelihood ratio for benignity
- Tested configuration
- primateAI_b (Cubuk et al. 2021)
- Protocol
- BRCA2 functional truth set, binary in silico calls (Cubuk et al. 2021)
- Dataset
- BRCA2 functional truth set: 188 missense variants classed deleterious or tolerated by Homology-directed repair
- Procedure
- brca-20261009-protocol-cubuk2021-brca2
- Evaluation
- primateAI_b on the BRCA2 functional truth set (Cubuk et al. 2021)
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Sources conflict: Printed interval (.425-3.60) has the same log-scale width as this row's positive likelihood ratio interval and is not consistent with the Supplementary Table 6 counts; kept in printed_source_cell only.
- Evidence
- Independent external evaluation · source checkedClinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes; Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx) · Supplementary Table 9 (sheet 'SupTable9_Likelihood_Ratios'), I42; tool 'primateAI_b'; column 'BRCA2_negative_LR'
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Whole truth set
- Adaptation
- Not reported
- Scoring implementation
- Binary call at the Supplementary Table 5 threshold; PLR = TPR/FPR and likelihood ratio for benignity = TNR/FNR (Supplementary Table 7)
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
2 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 1.23 Individual claims | Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes Supplementary Table 9 (sheet 'SupTable9_Likelihood_Ratios'), I42; tool 'primateAI_b'; column 'BRCA2_negative_LR' Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Genetics in Medicine 23(11):2096, published 2021-07-06; PMC8553612 full-text XML | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-09 independent paper Audit detailsExtracted by deterministic parse of the pinned XLSX cell XML (extract/extract_brca.py with extract/rawxlsx.py), with sheet names, column headers, row labels and the likelihood-ratio definitions in Supplementary Table 7 asserted. Cells print 'value (lower-upper)'; printed_value is the value as printed and the whole cell is kept in printed_source_cell. Pending independent review. Independent review 2026-10-09: value and identity match the source and the counts. The printed interval is not used: in 166 of 168 BRCA1/BRCA2 cells it disagrees with the 95% interval from the counts, and in 131 it has the log-scale width of the same row's positive likelihood ratio interval. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
| Reported result 1.23 Individual claims | Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx) Supplementary Table 9 (sheet 'SupTable9_Likelihood_Ratios'), I42; tool 'primateAI_b'; column 'BRCA2_negative_LR' Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 41436_2021_1265_MOESM3_ESM.xlsx | source checked ["source-hash-verification","deterministic-table-parse","independent-cell-check"] · 2026-10-09 independent paper Audit detailsExtracted by deterministic parse of the pinned XLSX cell XML (extract/extract_brca.py with extract/rawxlsx.py), with sheet names, column headers, row labels and the likelihood-ratio definitions in Supplementary Table 7 asserted. Cells print 'value (lower-upper)'; printed_value is the value as printed and the whole cell is kept in printed_source_cell. Pending independent review. Independent review 2026-10-09: value and identity match the source and the counts. The printed interval is not used: in 166 of 168 BRCA1/BRCA2 cells it disagrees with the 95% interval from the counts, and in 131 it has the log-scale width of the same row's positive likelihood ratio interval. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
Sources and history
Release 2026-10-10-84341e0b121f · Record review: source checked
2 source records and release history
- Clinical likelihood ratios and balanced accuracy for 44 in silico tools against multiple large-scale functional assays of cancer susceptibility genes · Original source · Genetics in Medicine 23(11):2096, published 2021-07-06; PMC8553612 full-text XML
- Cubuk et al. 2021, Supplementary tables 1-13 (41436_2021_1265_MOESM3_ESM.xlsx) · Original source · 41436_2021_1265_MOESM3_ESM.xlsx
Technical metadata and extraction receipts
Stable ID: brca-20261009-result-cubuk2021-primateai-b-brca2-blr
- metric
- benignity-likelihood-ratio
- metric direction
- higher
- unit
- unitless
- metric qualifier
- Source 'negative likelihood ratio' (TNR/FNR): likelihood ratio for benignity of a tolerated call, BRCA2 functional truth set
- printed value
- 1.23
- numeric value
- 1.23
- source locator
- Supplementary Table 9 (sheet 'SupTable9_Likelihood_Ratios'), I42; tool 'primateAI_b'; column 'BRCA2_negative_LR'
- review
- method: source-hash-verification; deterministic-table-parse; independent-cell-check; method note: Re-downloaded the workbook and matched its SHA-256. Read the cells with a separate stdlib OOXML reader written for this review; the extractor's scripts were not run. Checked the printed and numeric value, sheet, column, tool label, metric, denominator and the linked configuration and protocol. Recomputed the positive likelihood ratio (TPR/FPR), the benignity likelihood ratio (TNR/FNR) and the counts from Supplementary Table 6, and the 95% log-scale intervals from the same counts.; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; date: 2026-10-09; artifact sha256: 02df1b0dbf916d598dd8278ba45091022bf78c5722ac13a07d5b1b70d64dc179; retrieval url: https://static-content.springer.com/esm/art%3A10.1038%2Fs41436-021-01265-z/MediaObjects/41436_2021_1265_MOESM3_ESM.xlsx; note: Extracted by deterministic parse of the pinned XLSX cell XML (extract/extract_brca.py with extract/rawxlsx.py), with sheet names, column headers, row labels and the likelihood-ratio definitions in Supplementary Table 7 asserted. Cells print 'value (lower-upper)'; printed_value is the value as printed and the whole cell is kept in printed_source_cell. Pending independent review. Independent review 2026-10-09: value and identity match the source and the counts. The printed interval is not used: in 166 of 168 BRCA1/BRCA2 cells it disagrees with the 95% interval from the counts, and in 131 it has the log-scale width of the same row's positive likelihood ratio interval.
- printed source cell
- 1.23 (.425-3.60)
- denominator
- 188
- denominator note
- Variants with a deleterious or tolerated call (Supplementary Table 6 BRCA2_total_all); the truth set has 188
- missing metadata
- uncertainty: reason: conflicting; note: Printed interval (.425-3.60) has the same log-scale width as this row's positive likelihood ratio interval and is not consistent with the Supplementary Table 6 counts; kept in printed_source_cell only.