Relevance of Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) to "Assess models for genetic perturbation experiments"
- Use case
- Assess models for genetic perturbation experiments
- Assessed by
- Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a)
- Relevance
- Proxy evidence
- Endpoint
- Mean L2 distance and mean Pearson delta between predicted and observed expression over the 1,000 most highly expressed genes, for scGPT, scFoundation, GEARS, CPA and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the no change and additive baselines, Norman doubles (K562 CRISPRa)
scGPT, scFoundation, GEARS, CPA and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the no change and additive baselines are scored on the same held-out perturbations, which is the methods-and-controls comparison the use case asks for, from authors who did not develop the deep models. It is proxy evidence: expression-prediction error does not establish mechanism or experimental prioritisation (use-case exclusion), and the per-model means are computed by Rewire from the authors' Source Data rather than printed.
9 reviewed evaluations
- Additive on Norman doubles (Ahlmann-Eltze et al. 2025)
- CPA on Norman doubles (Ahlmann-Eltze et al. 2025)
- GEARS on Norman doubles (Ahlmann-Eltze et al. 2025)
- Geneformer on Norman doubles (Ahlmann-Eltze et al. 2025)
- No change on Norman doubles (Ahlmann-Eltze et al. 2025)
- scBERT on Norman doubles (Ahlmann-Eltze et al. 2025)
- scFoundation on Norman doubles (Ahlmann-Eltze et al. 2025)
- scGPT on Norman doubles (Ahlmann-Eltze et al. 2025)
- UCE on Norman doubles (Ahlmann-Eltze et al. 2025)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
60 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations3 values
| Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Citation locations3 values
| Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Extended Data Fig. 2 (41592_2025_2772_MOESM6_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Citation locations3 values
| Source Data Fig. 1, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 1 (41592_2025_2772_MOESM3_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group ahlmann2025-perturbation-baselines Context-only references | Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group ahlmann2025-perturbation-baselines Context-only references | Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Extended Data Fig. 2 (41592_2025_2772_MOESM6_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group ahlmann2025-perturbation-baselines Context-only references | Source Data Fig. 1, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 1 (41592_2025_2772_MOESM3_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025) Context-only references | Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025) Context-only references | Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Extended Data Fig. 2 (41592_2025_2772_MOESM6_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025) Context-only references | Source Data Fig. 1, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 1 (41592_2025_2772_MOESM3_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints3 values
| Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
3 source records and release history
- Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines · Original source · Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML
- Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Original source · Source Data Fig. 1 (41592_2025_2772_MOESM3_ESM.xlsx) as published with the article
- Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Original source · Source Data Extended Data Fig. 2 (41592_2025_2772_MOESM6_ESM.xlsx) as published with the article
Technical metadata and extraction receipts
Stable ID: use-case-mapping-perturbation-ahlmann-20261011-norman-double
- field
- links:assessed_by:perturbation-ahlmann-20261011-protocol-norman-double
- value
- perturbation-ahlmann-20261011-protocol-norman-double
- relevance
- proxy
- endpoint
- Mean L2 distance and mean Pearson delta between predicted and observed expression over the 1,000 most highly expressed genes, for scGPT, scFoundation, GEARS, CPA and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the no change and additive baselines, Norman doubles (K562 CRISPRa)
- rationale
- scGPT, scFoundation, GEARS, CPA and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the no change and additive baselines are scored on the same held-out perturbations, which is the methods-and-controls comparison the use case asks for, from authors who did not develop the deep models. It is proxy evidence: expression-prediction error does not establish mechanism or experimental prioritisation (use-case exclusion), and the per-model means are computed by Rewire from the authors' Source Data rather than printed.
- constraints
- Inspect every linked evaluation's source locator, derivation and preserved limitations before citing a result.; Do not combine these results with another protocol's results; splits, gene subsets and metric implementations differ between sources.; Cite these values as computed by Rewire from the authors' Source Data, not as printed values.
- limitations
- Values are computed by Rewire from the authors' per-perturbation Source Data (attributes.derivation on each result); the paper prints no per-model number.; The authors ran all models themselves; the deep models were not run by their developers, although the developers were asked to review the code.; No spread of the mean is stated, so small differences between models are not testable from these values.; Training includes every single perturbation and half of the doubles, the setting the use case's GEARS combination guidance requires.; UCE and scBERT post-perturbation embeddings use ground-truth expression of the perturbed genes (possible test leakage, Methods).; The Pearson delta for the no change baseline is undefined and not recorded.; The Norman gene count differs between the main text (19,264) and the Extended Data Fig. 10 legend (19,624).
- citation locators
- source id: perturbation-ahlmann-20261011-source-data-fig1; locator: Source Data Fig. 1, sheet "Panel A"; source id: perturbation-ahlmann-20261011-source-data-ed-fig2; locator: Source Data Extended Data Fig. 2, sheet "Panel A"; source id: perturbation-ahlmann-20261011-source-article; locator: Fig. 1 and Extended Data Fig. 2 legends; main text paragraphs 4 to 6; Methods
- source locator
- perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A"
- revision
- 1
- reason
- Recorded from the Ahlmann-Eltze et al. 2025 Source Data pass 2026-10-11 (data/omics/perturbation-ahlmann-derived-20261011); independently reviewed 2026-10-11.
- comparison group
- ahlmann2025-perturbation-baselines
- comparison title
- Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025)
- headline metric
- l2-distance
- stratum label
- Norman doubles (K562 CRISPRa)
- stratum order
- 1
- review
- method: source-hash-verification; ai-assisted-source-review; reviewer: claude; reviewer note: Separate Claude (Opus 5.5) review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-11T03:00:00Z; artifact sha256: dc5ccf62cb51d301f327a3c3e629d60be719bb2aaf02b57fb4999be43a4d95c7; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12328236/fullTextXML; method note: Checked the endpoint against the use case's question, inputs, output and exclusions; checked proxy against the relevance definitions; checked the limitations against the article and the Source Data evidence concerns; checked that every model on the protocol was scored on the same perturbations in each split. Checked 9 evaluations (listed in reviewed_evaluations); none excluded.; note: Proxy relevance confirmed in independent review. Pins are set by the integrator with use-cases:repin after the records are in the store. Not human scientific review. See docs/reviews/use-cases/genetic-perturbation-response-2026-10-11.md.
- reviewed evaluations
- perturbation-ahlmann-20261011-eval-norman-additive; perturbation-ahlmann-20261011-eval-norman-cpa; perturbation-ahlmann-20261011-eval-norman-gears; perturbation-ahlmann-20261011-eval-norman-geneformer; perturbation-ahlmann-20261011-eval-norman-no-change; perturbation-ahlmann-20261011-eval-norman-scbert; perturbation-ahlmann-20261011-eval-norman-scfoundation; perturbation-ahlmann-20261011-eval-norman-scgpt; perturbation-ahlmann-20261011-eval-norman-uce
- pins
- record id: perturbation-ahlmann-20261011-eval-norman-additive; fields: status; links; attributes.origin; attributes.comparison; sha256: 30acf65a14f9ed48623a44b60a8d4039a8415a5dd437bc45ecf469b695cc8e2e; record id: perturbation-ahlmann-20261011-eval-norman-cpa; fields: status; links; attributes.origin; attributes.comparison; sha256: 166f24cc4d347637aff309b3b71a9601f82b40cffce5c7d859ed2f74c485f31f; record id: perturbation-ahlmann-20261011-eval-norman-gears; fields: status; links; attributes.origin; attributes.comparison; sha256: 3e3c365231ffdc8300ea8e33ba6697a3f605e16b34488680af745c60619f58a0; record id: perturbation-ahlmann-20261011-eval-norman-geneformer; fields: status; links; attributes.origin; attributes.comparison; sha256: 6eedb988ecbb561b91fabd38662cf72c345cf040ab604833177e360dfa0a7bb6; record id: perturbation-ahlmann-20261011-eval-norman-no-change; fields: status; links; attributes.origin; attributes.comparison; sha256: b3fecd366e9dc3b4378720a4cd07cb3ba4a99dcf7437e03875ed713e1910bc47; record id: perturbation-ahlmann-20261011-eval-norman-scbert; fields: status; links; attributes.origin; attributes.comparison; sha256: 0782a7eb11a3e5b5797309c55410cc053ebb220079158b2e91aef2b072bf0fb6; record id: perturbation-ahlmann-20261011-eval-norman-scfoundation; fields: status; links; attributes.origin; attributes.comparison; sha256: 24ce6de842f2c9d54b2296c606bbebf53831cd26f530dec6216df759b1007700; record id: perturbation-ahlmann-20261011-eval-norman-scgpt; fields: status; links; attributes.origin; attributes.comparison; sha256: c6720192e91838b4a9425e00318df371744ae6da52cc14b555643784a585161e; record id: perturbation-ahlmann-20261011-eval-norman-uce; fields: status; links; attributes.origin; attributes.comparison; sha256: 4fd894775f443b293ec6164fcf6381cf600bcef8cc09a899ec494b055de69c83; record id: perturbation-ahlmann-20261011-protocol-norman-double; fields: status; name; description; facets; source_ids; links; attributes; sha256: 865a7b822f5cf2bc23b3a00b6172680ecc35d222722818fdfd99334906622298; record id: perturbation-ahlmann-20261011-result-ed2a-norman-additive; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0d44df86764e5e7418a3825d2f7df176e4a14e9ff798eb10fb3f3b31abe70444; record id: perturbation-ahlmann-20261011-result-ed2a-norman-cpa; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ce888366cd352d3e2e11e9c0074c824acd938d5e3ea4c0755bfcd2c06cd08203; record id: perturbation-ahlmann-20261011-result-ed2a-norman-gears; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d511c8c10bf88b254bc6436c6e85a251de962e35e2b7356261a4895dbc2b3df2; record id: perturbation-ahlmann-20261011-result-ed2a-norman-geneformer; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 067df70dd839e2fbc571651ac33337ef70f07cc22f94e0c6d98fd837230793dc; record id: perturbation-ahlmann-20261011-result-ed2a-norman-scbert; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b20e8facf80f4ea9009a1895a336a5cf1c65e8f73b74ec136bd75ca9ff840353; record id: perturbation-ahlmann-20261011-result-ed2a-norman-scfoundation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: df5a81e612a8cfaa0be00e15e749b6500570e1a0622df38660d775df286e1925; record id: perturbation-ahlmann-20261011-result-ed2a-norman-scgpt; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 412cf779749af37939c3a617ca427a14e25bd843e80768236ef701494cf7ce44; record id: perturbation-ahlmann-20261011-result-ed2a-norman-uce; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b20e8facf80f4ea9009a1895a336a5cf1c65e8f73b74ec136bd75ca9ff840353; record id: perturbation-ahlmann-20261011-result-fig1a-norman-additive; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 87831b2be7007b7e4e58e1e1341d0a59a5cfcfd4f25aeba901b531f594d9b12d; record id: perturbation-ahlmann-20261011-result-fig1a-norman-cpa; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c43333e2127cfb07045ec8386fcbaad2dd059220b659050cb47bf6cd92c8e58e; record id: perturbation-ahlmann-20261011-result-fig1a-norman-gears; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 02d1db5c2d8d4e72a5328a7706cf5b01a0908422bca3cc73f8c81cbe5ad5171a; record id: perturbation-ahlmann-20261011-result-fig1a-norman-geneformer; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ea8fc7ebb814a5f2c86ef8c28b6f92fd122e10d10d7503ab6aae935c1b6136dc; record id: perturbation-ahlmann-20261011-result-fig1a-norman-no-change; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f2bad774ec09f0a60a3f80cf478974001ba3c806260e95346ef542cef624696b; record id: perturbation-ahlmann-20261011-result-fig1a-norman-scbert; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d255fa300d42e7facbc806d43556793f1c1ae1bfacd423eec6ed88a9f4c0a119; record id: perturbation-ahlmann-20261011-result-fig1a-norman-scfoundation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 07f5e55588a08f5a32b299daf9f322201a971ea1c43fd57c6f5ab9a1178c918e; record id: perturbation-ahlmann-20261011-result-fig1a-norman-scgpt; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7fd4d9eb182532131b48a6bb47def8f4e5b4923ee36bd25b1b733c3fbe08fdf6; record id: perturbation-ahlmann-20261011-result-fig1a-norman-uce; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d255fa300d42e7facbc806d43556793f1c1ae1bfacd423eec6ed88a9f4c0a119; record id: perturbation-ahlmann-20261011-source-article; fields: status; attributes.artifact_sha256; sha256: 4cc432f631d9f7012a62ce39d0f9ffc6b889422623700e3bb0e10b33d63fef61; record id: perturbation-ahlmann-20261011-source-data-ed-fig2; fields: status; attributes.artifact_sha256; sha256: 57f6089caeebabb657da42a227451701d2937cd9163916700aa2e4804bfba2b4; record id: perturbation-ahlmann-20261011-source-data-fig1; fields: status; attributes.artifact_sha256; sha256: d505d7d587784cc2f54cf1543fcb5cf01bd43b5ceddd3028a15ad33d7fbe45f0; record id: use-case-genetic-perturbation-response; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 440f0f2e30cb7c828212bed728de2420ba8e13521280be58be7bdcdfe57ac973