AUROC (per-perturbation AUROC for DE targets, summarised over perturbations; RPE1 CRISPRi) of Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026) on Replogle genome-scale CRISPRi, RPE1, 1,167 perturbations, confound battery (Kendiukhov 2026)
0.747 AUROC (per-perturbation AUROC for DE targets, summarised over perturbations; RPE1 CRISPRi)
Methods
- Tested configuration
- Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)
- Protocol
- Perturbation-first AUROC, RPE1 CRISPRi (Kendiukhov 2026)
- Dataset subset
- Replogle genome-scale CRISPRi, RPE1, 1,167 perturbations, confound battery (Kendiukhov 2026)
- Procedure
- Perturbation-first AUROC, RPE1 CRISPRi (Kendiukhov 2026)
- Evaluation
- Geneformer V2-316M attention edges, layer not stated on Replogle genome-scale CRISPRi, RPE1, 1,167 perturbations, confound battery
- Coverage
- Not reported scored / Not reported eligible
- Uncertainty
- Not reported by the source
- Evidence
- Independent external evaluation · source checkedSystematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Page 10, 'Cross-cell-type generalisation of confound pattern', item (i): 'variance AUROC = 0.866, mean expression = 0.851, dropout = 0.797 vs. attention = 0.747 and correlation = 0.658'
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- No train/test split; scores computed from control cells and compared with DE labels
- Adaptation
- Foundation-model attention read out zero-shot; correlation and gene-level statistics computed from the same control cells; logistic regression fitted under GroupKFold
- Scoring implementation
- Per-perturbation AUROC for classifying differentially expressed targets (Mann-Whitney U test with Benjamini-Hochberg correction and an LFC threshold define DE hits); summary across perturbations not stated
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 0.747 Individual claims | Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) Page 10, 'Cross-cell-type generalisation of confound pattern', item (i): 'variance AUROC = 0.866, mean expression = 0.851, dropout = 0.797 vs. attention = 0.747 and correlation = 0.658' Version: Version of record, BMC Genomics 27:634, published 2026-07-22 (received 2026-03-10, accepted 2026-05-14) | source checked ["source-hash-verification","pdf-text-parse","independent-cell-check"] · 2026-10-14 independent paper Audit detailsExtraction: Hand transcription from the article text; extract/extract_kendiukhov2026.py asserts that the quoted fragment, which contains the printed value, occurs on the stated page of the pdftotext output of the pinned PDF. Independent review 2026-10-14: matched. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: source checked
1 source record and release history
- Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Original source · Version of record, BMC Genomics 27:634, published 2026-07-22 (received 2026-03-10, accepted 2026-05-14)
Technical metadata and extraction receipts
Stable ID: gfm-guides-20261014-kendiukhov2026-result-pf-rpe1-crispri-rpe1-crispri-1167-geneformer-v2-316m-attention-layer-unstated-auroc
- metric
- auroc
- metric direction
- higher
- unit
- unitless
- printed value
- 0.747
- numeric value
- 0.747
- metric qualifier
- per-perturbation AUROC for DE targets, summarised over perturbations; RPE1 CRISPRi
- source locator
- Page 10, 'Cross-cell-type generalisation of confound pattern', item (i): 'variance AUROC = 0.866, mean expression = 0.851, dropout = 0.797 vs. attention = 0.747 and correlation = 0.658'
- review
- method: source-hash-verification; pdf-text-parse; independent-cell-check; method note: Archived PDF hashes to the pinned SHA-256 (a fresh download returned a bot-check page). Each value, its sentence, page, perturbation count, configuration and protocol re-read by hand in the pdftotext layer; the K562 n = 151 assignment confirmed by the Fig. 3 caption.; reviewer: claude; reviewer note: Claude (Opus 5.5) research agent, independent of the extractor; no human review claimed; date: 2026-10-14; artifact sha256: 4c1699cb964ae588117a6737f5e9e189eb9a5ef5ff78568d2a99e7a13d6608fe; retrieval url: https://bmcgenomics.biomedcentral.com/counter/pdf/10.1186/s12864-026-12965-8.pdf; note: Extraction: Hand transcription from the article text; extract/extract_kendiukhov2026.py asserts that the quoted fragment, which contains the printed value, occurs on the stated page of the pdftotext output of the pinned PDF. Independent review 2026-10-14: matched.
- missing metadata
- uncertainty: reason: unreported