Result

L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits) of scBERT (Ahlmann-Eltze et al. 2025) on Replogle et al. 2022 Perturb-seq essential-gene screen, K562 CRISPRi (GEARS distribution, Ahlmann-Eltze et al. 2025)

4.19 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits) Computed by Rewire from the authors' source data

Evidence concern: excluded from comparisons

The Fig. 2 legend gives "134, 210 and 24 unseen single perturbations across two test-training splits". In Panel A these are the split 1 counts only (Replogle K562 134, Replogle RPE1 210, Adamson 24 rows per method); split 2 has 128, 203 and 24, so the two-split totals are 262, 413 and 48. The Fig. 2a means recorded here use all Panel A rows; whether the plotted red lines use both splits is not stated beyond the sheet itself. Reviewer check 2026-10-11: using split 1 rows only would change the per-model means by up to 0.61 (L2 distance) and 0.06 (Pearson delta), and would change the model with the lowest mean L2 distance on Adamson and Replogle RPE1, so the choice is material.Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Fig. 2 legend versus Source Data Fig. 2, sheet "Panel A"

Methods restricts the single-perturbation analysis to perturbations predicted by all models: 73 for Adamson, 398 for Replogle K562 and 629 for Replogle RPE1. No sheet of Source Data Fig. 2 has these counts (Panel A: 48, 262 and 413 per method over two splits; Panel C: 20, 214 and 224 test or val rows per method).Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Methods 'Single perturbation benchmark setup' paragraph 4 versus Source Data Fig. 2, sheets "Panel A" and "Panel C"

The test and val rows in Panel C (20, 214 and 224 per method for Adamson, Replogle K562 and Replogle RPE1) match neither the Fig. 2 legend counts nor the Methods counts. Fig. 2c was not extracted.Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Fig. 2, sheet "Panel C" versus the Fig. 2 legend and Methods

The main text says CPA was not included in the single-perturbation benchmark, but method "cpa" has rows in Panel C. Fig. 2c was not extracted.Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Main text paragraph 16 versus Source Data Fig. 2, sheet "Panel C"

How this value was computed

Computed by Rewire from the authors' source data. This value is not printed in the source; Rewire computed it with the script below from the data files listed.

Inputs

Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Data file, Source Data Fig. 2 (MOESM4), sheet "Panel A"; rows where dataset_name == "replogle_k562_essential" and method == "scbert"; train in {test, val}; seed in {1, 2}; column l2 (262 rows) · file checksum 2ccfa7239c19

Aggregation, as the source describes it

The horizontal red lines show the mean per model · Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines · Data file, Fig. 2 legend, panel a

Script

scripts/omics/derive/ahlmann_eltze_2025.py · checksum 452a7a7aaab7

Precision

Exact arithmetic mean of the stored cell values, rounded half to even to 3 decimal places

Note

The sheet labels each held-out perturbation test or val; both are pooled, as for Fig. 1. The Fig. 2 legend counts are the split 1 counts only; this mean uses both splits as the sheet does (see the source's evidence concerns).

Methods

Tested configuration: scBERT (Ahlmann-Eltze et al. 2025)
Protocol: Replogle K562: expression prediction for unseen single perturbations (Ahlmann-Eltze et al. 2025 Fig. 2a and Extended Data Fig. 8a)
Dataset: Replogle et al. 2022 Perturb-seq essential-gene screen, K562 CRISPRi (GEARS distribution, Ahlmann-Eltze et al. 2025)
Procedure: Replogle K562: expression prediction for unseen single perturbations (Ahlmann-Eltze et al. 2025 Fig. 2a and Extended Data Fig. 8a)
Evaluation: scBERT on Replogle K562 (Ahlmann-Eltze et al. 2025)
Coverage: 262/262 perturbation-split rows scored
Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one
Evidence: Independent external evaluation · source checkedDeep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Fig. 2 (MOESM4), sheet "Panel A", rows with dataset_name "replogle_k562_essential" and method "scbert", column "l2"; plotted as the red line in Fig. 2a

Review of a computed result checks its inputs and script against the source's data, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.

Reproduction

Split: GEARS 'simulation' test-training split, run twice (two splits)
Adaptation: Fine-tuned, fitted or applied per split by the paper's authors with default parameters where possible
Scoring implementation: L2 distance and Pearson delta over the 1,000 most highly expressed genes in the control condition (Methods 'Evaluation metrics')

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

2 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
Computed by Rewire from source data 4.192 Individual claims	Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines Original source ↗ Source Data Fig. 2 (MOESM4), sheet "Panel A", rows with dataset_name "replogle_k562_essential" and method "scbert", column "l2"; plotted as the red line in Fig. 2a Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML Retrieved: 2026-10-11T00:48:13Z	source checked ["source-hash-verification","source-data-recomputation","independent-cell-check"] · 2026-10-11T03:00:00Z independent paper Audit details Computed by scripts/omics/derive/ahlmann_eltze_2025.py (sha256 452a7a7aaab7e4965142de87ed9f00209a6d9664760919cb39b921bb45aec45c) from the re-downloaded workbook, whose SHA-256 the script checks along with every row count. Not printed in the source. Independent review 2026-10-11: value recomputed and matches; not printed in the source and not a reproduction of any model run. See docs/reviews/use-cases/genetic-perturbation-response-2026-10-11.md. Field: `attributes.printed_value` Source artifact SHA-256: `dc5ccf62cb51d301f327a3c3e629d60be719bb2aaf02b57fb4999be43a4d95c7` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact Extraction artifact SHA-256: `2ccfa7239c195f329e1632b51510adebfd493e0a5a7427a0f012e078e0e59c35` Extraction artifact
Computed by Rewire from source data 4.192 Individual claims	Source Data Fig. 2, Ahlmann-Eltze et al. 2025 Original source ↗ Source Data Fig. 2 (MOESM4), sheet "Panel A", rows with dataset_name "replogle_k562_essential" and method "scbert", column "l2"; plotted as the red line in Fig. 2a Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 2 (41592_2025_2772_MOESM4_ESM.xlsx) as published with the article Retrieved: 2026-10-11T00:48:13Z	source checked ["source-hash-verification","source-data-recomputation","independent-cell-check"] · 2026-10-11T03:00:00Z independent paper Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Computed by scripts/omics/derive/ahlmann_eltze_2025.py (sha256 452a7a7aaab7e4965142de87ed9f00209a6d9664760919cb39b921bb45aec45c) from the re-downloaded workbook, whose SHA-256 the script checks along with every row count. Not printed in the source. Independent review 2026-10-11: value recomputed and matches; not printed in the source and not a reproduction of any model run. See docs/reviews/use-cases/genetic-perturbation-response-2026-10-11.md. Field: `attributes.printed_value` Source artifact SHA-256: `2ccfa7239c195f329e1632b51510adebfd493e0a5a7427a0f012e078e0e59c35` Hash scope: Hash scope not separately documented; inspect source record Extraction artifact SHA-256: `2ccfa7239c195f329e1632b51510adebfd493e0a5a7427a0f012e078e0e59c35` Extraction artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

2 source records and release history

Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines · Original source · Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML
Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Original source · Source Data Fig. 2 (41592_2025_2772_MOESM4_ESM.xlsx) as published with the article

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-scbert

areas: cells-tissues; single-cell
contexts: research
printed value: 4.192
numeric value: 4.192
metric: l2-distance
metric qualifier: 1,000 most highly expressed control genes; mean over held-out perturbations and splits
metric direction: lower
unit: unit-unreported
source locator: Source Data Fig. 2 (MOESM4), sheet "Panel A", rows with dataset_name "replogle_k562_essential" and method "scbert", column "l2"; plotted as the red line in Fig. 2a
coverage: scored: 262; eligible: 262; unit: perturbation-split rows
derivation: method: computed-from-source-data; inputs: source id: perturbation-ahlmann-20261011-source-data-fig2; artifact sha256: 2ccfa7239c195f329e1632b51510adebfd493e0a5a7427a0f012e078e0e59c35; locator: Source Data Fig. 2 (MOESM4), sheet "Panel A"; row filter: dataset_name == "replogle_k562_essential" and method == "scbert"; train in {test, val}; seed in {1, 2}; column l2; row count: 262; aggregation: The horizontal red lines show the mean per model; aggregation source id: perturbation-ahlmann-20261011-source-article; aggregation locator: Fig. 2 legend, panel a; script: scripts/omics/derive/ahlmann_eltze_2025.py; script sha256: 452a7a7aaab7e4965142de87ed9f00209a6d9664760919cb39b921bb45aec45c; precision: Exact arithmetic mean of the stored cell values, rounded half to even to 3 decimal places; note: The sheet labels each held-out perturbation test or val; both are pooled, as for Fig. 1. The Fig. 2 legend counts are the split 1 counts only; this mean uses both splits as the sheet does (see the source's evidence concerns).
review: method: source-hash-verification; source-data-recomputation; independent-cell-check; method note: Re-downloaded all four workbooks and the article XML and matched each SHA-256. Checked the script's SHA-256, reran it on the fresh downloads (output identical to derived-values.jsonl), and recomputed every mean with a separate openpyxl reader written for this review. Checked the row filter and count, the quoted aggregation at its locator, rounding (no value within 0.001 of a rounding tie), metric, qualifier, unit, direction, and the evaluation's configuration, protocol, dataset and origin.; reviewer: claude; reviewer note: Separate Claude (Opus 5.5) review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-11T03:00:00Z; artifact sha256: 2ccfa7239c195f329e1632b51510adebfd493e0a5a7427a0f012e078e0e59c35; retrieval url: https://static-content.springer.com/esm/art%3A10.1038%2Fs41592-025-02772-6/MediaObjects/41592_2025_2772_MOESM4_ESM.xlsx; note: Computed by scripts/omics/derive/ahlmann_eltze_2025.py (sha256 452a7a7aaab7e4965142de87ed9f00209a6d9664760919cb39b921bb45aec45c) from the re-downloaded workbook, whose SHA-256 the script checks along with every row count. Not printed in the source. Independent review 2026-10-11: value recomputed and matches; not printed in the source and not a reproduction of any model run. See docs/reviews/use-cases/genetic-perturbation-response-2026-10-11.md.
missing metadata: uncertainty: reason: unreported; note: No spread of the mean is stated; Rewire did not compute one
unit detail: distance between expression profiles; the article does not state the expression unit or transform for the GEARS-distributed Adamson and Replogle data

Related records

evaluation: scBERT on Replogle K562 (Ahlmann-Eltze et al. 2025)

Suggest a correction