rewirebio.iobenchmarks
Configuration

Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)

Attention edges from Geneformer V2-316M where the article does not name the layer (K562 CRISPRa, K562 propensity-matched, RPE1 confound battery).

5 evaluations · 5 results3 open disagreements between studies involve its model

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Evaluations and results

5 evaluations · 5 results. Different protocols are not a single leaderboard.

Sorted by AUROC (AUROC over propensity-matched perturbation-gene pairs (k = 5 negatives per positive)) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configurationProtocol and datasetAUROC (AUROC over propensity-matched perturbation-gene pairs (k = 5 negatives per positive))AUROC (per-perturbation AUROC for DE targets, summarised over perturbations; RPE1 CRISPRi)AUROC difference (Geneformer V2-316M attention edge minus absolute Spearman correlation edge (per-perturbation attention advantage); per-perturbation AUROC for DE targets, summary over perturbations not stated; RPE1 CRISPRi restricted to the top 2,000 HVGs)AUROC difference (Geneformer V2-316M attention edge minus absolute Spearman correlation edge (per-perturbation attention advantage); per-perturbation AUROC for DE targets, summary over perturbations not stated; RPE1 CRISPRi)
Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)EvaluationPropensity-matched edge AUROC (Kendiukhov 2026), Replogle CRISPRi K562, propensity-matched pairs (Kendiukhov 2026)0.609 (best in column)Not reportedNot reportedNot reported
Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)EvaluationPropensity-matched edge AUROC (Kendiukhov 2026), Replogle CRISPRi RPE1, propensity-matched pairs (Kendiukhov 2026)0.568Not reportedNot reportedNot reported
Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)EvaluationPerturbation-first AUROC, RPE1 CRISPRi restricted to the top 2,000 HVGs (Kendiukhov 2026), Replogle genome-scale CRISPRi, RPE1, 418 perturbations with the top 2,000 HVGs (Kendiukhov 2026)Not reportedNot reported0.168Not reported
Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)EvaluationPerturbation-first AUROC, RPE1 CRISPRi (Kendiukhov 2026), Replogle genome-scale CRISPRi, RPE1, 1,167 perturbations, confound battery (Kendiukhov 2026)Not reported0.747Not reportedNot reported
Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)EvaluationPerturbation-first AUROC, RPE1 CRISPRi (Kendiukhov 2026), Replogle genome-scale CRISPRi, RPE1, 1,251 evaluable perturbations (Kendiukhov 2026)Not reportedNot reportedNot reported0.09
All 5 result rows with coverage, uncertainty and sources
Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)Protocol: Perturbation-first AUROC, RPE1 CRISPRi restricted to the top 2,000 HVGs (Kendiukhov 2026)
Dataset subset: Replogle genome-scale CRISPRi, RPE1, 418 perturbations with the top 2,000 HVGs (Kendiukhov 2026)
0.168 AUROC difference (Geneformer V2-316M attention edge minus absolute Spearman correlation edge (per-perturbation attention advantage); per-perturbation AUROC for DE targets, summary over perturbations not stated; RPE1 CRISPRi restricted to the top 2,000 HVGs)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

Geneformer V2-316M attention edges, layer not stated on Replogle genome-scale CRISPRi, RPE1, 418 perturbations with the top 2,000 HVGs

gfm-guides-20261014-kendiukhov2026-protocol-pf-rpe1-crispri-hvg2000

Aggregation: Not reported

Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Page 10, 'Context-dependent attention-correlation relationship', sentence beginning 'A methodological asymmetry': 'increases the attention advantage (∆ = +0.168, n = 418'
Configuration: Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)Protocol: Perturbation-first AUROC, RPE1 CRISPRi (Kendiukhov 2026)
Dataset subset: Replogle genome-scale CRISPRi, RPE1, 1,167 perturbations, confound battery (Kendiukhov 2026)
0.747 AUROC (per-perturbation AUROC for DE targets, summarised over perturbations; RPE1 CRISPRi)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

Geneformer V2-316M attention edges, layer not stated on Replogle genome-scale CRISPRi, RPE1, 1,167 perturbations, confound battery

gfm-guides-20261014-kendiukhov2026-protocol-pf-rpe1-crispri

Aggregation: Not reported

Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Page 10, 'Cross-cell-type generalisation of confound pattern', item (i): 'variance AUROC = 0.866, mean expression = 0.851, dropout = 0.797 vs. attention = 0.747 and correlation = 0.658'
Configuration: Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)Protocol: Perturbation-first AUROC, RPE1 CRISPRi (Kendiukhov 2026)
Dataset subset: Replogle genome-scale CRISPRi, RPE1, 1,251 evaluable perturbations (Kendiukhov 2026)
0.09 AUROC difference (Geneformer V2-316M attention edge minus absolute Spearman correlation edge (per-perturbation attention advantage); per-perturbation AUROC for DE targets, summary over perturbations not stated; RPE1 CRISPRi)
unitless · higher

Uncertainty: 95% CI 0.079 to 0.101. Methods 'Statistical framework': bootstrap CIs use 200 iterations with perturbation-level resampling unless otherwise noted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

Geneformer V2-316M attention edges, layer not stated on Replogle genome-scale CRISPRi, RPE1, 1,251 evaluable perturbations

gfm-guides-20261014-kendiukhov2026-protocol-pf-rpe1-crispri

Aggregation: Not reported

Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Page 10, 'Context-dependent attention-correlation relationship', sentence beginning 'Bootstrap 95% CIs on the per-perturbation attention advantage': 'RPE1 (∆ = +0.090 [+0.079, +0.101]'
Configuration: Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)Protocol: Propensity-matched edge AUROC (Kendiukhov 2026)
Dataset subset: Replogle CRISPRi K562, propensity-matched pairs (Kendiukhov 2026)
0.609 AUROC (AUROC over propensity-matched perturbation-gene pairs (k = 5 negatives per positive))
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

Geneformer V2-316M attention edges, layer not stated on Replogle CRISPRi K562, propensity-matched pairs

gfm-guides-20261014-kendiukhov2026-protocol-propensity-matched

Aggregation: Not reported

Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Page 9, paragraph 1 (continuing 'No incremental pairwise value'), sentence 3: 'retain modest raw discriminability (AUROC = 0.609)'
Configuration: Geneformer V2-316M attention edges, layer not stated (Kendiukhov 2026)Protocol: Propensity-matched edge AUROC (Kendiukhov 2026)
Dataset subset: Replogle CRISPRi RPE1, propensity-matched pairs (Kendiukhov 2026)
0.568 AUROC (AUROC over propensity-matched perturbation-gene pairs (k = 5 negatives per positive))
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

Geneformer V2-316M attention edges, layer not stated on Replogle CRISPRi RPE1, propensity-matched pairs

gfm-guides-20261014-kendiukhov2026-protocol-propensity-matched

Aggregation: Not reported

Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction (Kendiukhov 2026, BMC Genomics 27:634) · Page 10, 'Cross-cell-type generalisation of confound pattern', item (iii): '(attention = 0.568, correlation = 0.561)'

Source checking is not independent reproduction. Release 2026-11-02-a7f03c9e55a5.

Use this configuration

How it works, versions and access

Additional methodological details, versions and access terms have not yet been extracted into a structured profile for this configuration. Consult the linked sources and evaluation records below.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-02-a7f03c9e55a5
Property and statementOriginal source and locationReview and provenance

No evidence rows match these filters. Choose another scope or clear the search.

Sources and history

Release 2026-11-02-a7f03c9e55a5 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: gfm-guides-20261014-kendiukhov2026-config-geneformer-v2-316m-attention-layer-unstated

areas
single-cell; biological-networks
contexts
research
method types
foundation_model
reported name
attention
foundation model eligible
true
representation
Attention edges from Geneformer V2-316M where the article does not name the layer (K562 CRISPRa, K562 propensity-matched, RPE1 confound battery).
source locator
Page 13 'Models and data', 'Attention extraction and edge score computation', 'Model coverage by analysis family'
training setting
Zero-shot; pretrained weights, attention read out without fine-tuning
checkpoint
Geneformer V2-316M, HuggingFace ctheodoris/Geneformer subfolder Geneformer-V2-316M (18 layers, 18 heads), loaded with attn_implementation='eager'; revision not stated
version
Geneformer V2-316M
access reuse
Analysis code github.com/Biodyn-AI/biomechinterp-framework and Zenodo 10.5281/zenodo.18701417; not pinned to a commit in the article
Related records

Suggest a correction