Relevance of Replogle K562: expression prediction for unseen single perturbations (Ahlmann-Eltze et al. 2025 Fig. 2a and Extended Data Fig. 8a) to "Assess models for genetic perturbation experiments"
- Use case
- Assess models for genetic perturbation experiments
- Assessed by
- Replogle K562: expression prediction for unseen single perturbations (Ahlmann-Eltze et al. 2025 Fig. 2a and Extended Data Fig. 8a)
- Relevance
- Proxy evidence
- Endpoint
- Mean L2 distance and mean Pearson delta between predicted and observed expression over the 1,000 most highly expressed genes, for scGPT, GEARS and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the mean baseline and a linear model, Replogle K562 (CRISPRi)
scGPT, GEARS and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the mean baseline and a linear model are scored on the same held-out perturbations, which is the methods-and-controls comparison the use case asks for, from authors who did not develop the deep models. It is proxy evidence: expression-prediction error does not establish mechanism or experimental prioritisation (use-case exclusion), and the per-model means are computed by Rewire from the authors' Source Data rather than printed.
7 reviewed evaluations
- GEARS on Replogle K562 (Ahlmann-Eltze et al. 2025)
- Geneformer on Replogle K562 (Ahlmann-Eltze et al. 2025)
- Linear model (G and P from training data) on Replogle K562 (Ahlmann-Eltze et al. 2025)
- Mean on Replogle K562 (Ahlmann-Eltze et al. 2025)
- scBERT on Replogle K562 (Ahlmann-Eltze et al. 2025)
- scGPT on Replogle K562 (Ahlmann-Eltze et al. 2025)
- UCE on Replogle K562 (Ahlmann-Eltze et al. 2025)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
60 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations3 values
| Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Citation locations3 values
| Source Data Extended Data Fig. 8, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Extended Data Fig. 8 (41592_2025_2772_MOESM10_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Citation locations3 values
| Source Data Fig. 2, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 2 (41592_2025_2772_MOESM4_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group ahlmann2025-perturbation-baselines Context-only references | Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group ahlmann2025-perturbation-baselines Context-only references | Source Data Extended Data Fig. 8, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Extended Data Fig. 8 (41592_2025_2772_MOESM10_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group ahlmann2025-perturbation-baselines Context-only references | Source Data Fig. 2, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 2 (41592_2025_2772_MOESM4_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025) Context-only references | Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025) Context-only references | Source Data Extended Data Fig. 8, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Extended Data Fig. 8 (41592_2025_2772_MOESM10_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025) Context-only references | Source Data Fig. 2, Ahlmann-Eltze et al. 2025 perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Source Data Fig. 2 (41592_2025_2772_MOESM4_ESM.xlsx) as published with the article | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints3 values
| Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A" Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
3 source records and release history
- Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines · Original source · Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML
- Source Data Fig. 2, Ahlmann-Eltze et al. 2025 · Original source · Source Data Fig. 2 (41592_2025_2772_MOESM4_ESM.xlsx) as published with the article
- Source Data Extended Data Fig. 8, Ahlmann-Eltze et al. 2025 · Original source · Source Data Extended Data Fig. 8 (41592_2025_2772_MOESM10_ESM.xlsx) as published with the article
Technical metadata and extraction receipts
Stable ID: use-case-mapping-perturbation-ahlmann-20261011-replogle-k562-single
- field
- links:assessed_by:perturbation-ahlmann-20261011-protocol-replogle-k562-single
- value
- perturbation-ahlmann-20261011-protocol-replogle-k562-single
- relevance
- proxy
- endpoint
- Mean L2 distance and mean Pearson delta between predicted and observed expression over the 1,000 most highly expressed genes, for scGPT, GEARS and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the mean baseline and a linear model, Replogle K562 (CRISPRi)
- rationale
- scGPT, GEARS and three foundation models with a linear decoder (Geneformer, UCE, scBERT) against the mean baseline and a linear model are scored on the same held-out perturbations, which is the methods-and-controls comparison the use case asks for, from authors who did not develop the deep models. It is proxy evidence: expression-prediction error does not establish mechanism or experimental prioritisation (use-case exclusion), and the per-model means are computed by Rewire from the authors' Source Data rather than printed.
- constraints
- Inspect every linked evaluation's source locator, derivation and preserved limitations before citing a result.; Do not combine these results with another protocol's results; splits, gene subsets and metric implementations differ between sources.; Cite these values as computed by Rewire from the authors' Source Data, not as printed values.
- limitations
- Values are computed by Rewire from the authors' per-perturbation Source Data (attributes.derivation on each result); the paper prints no per-model number.; The authors ran all models themselves; the deep models were not run by their developers, although the developers were asked to review the code.; No spread of the mean is stated, so small differences between models are not testable from these values.; The Fig. 2 legend's counts are split 1 only, and the Methods counts (73, 398, 629) match no sheet; the Source Data sources carry evidence concerns, so these results do not support automatic comparison.; scFoundation and CPA are not evaluated here; Fig. 2c (embedding variants, bootstrapped ratios) was not extracted because its aggregation is ambiguous.; UCE and scBERT post-perturbation embeddings use ground-truth expression of the perturbed genes; the authors note for UCE that this could leak test data (Methods 'Software versions and parameters').
- citation locators
- source id: perturbation-ahlmann-20261011-source-data-fig2; locator: Source Data Fig. 2, sheet "Panel A"; source id: perturbation-ahlmann-20261011-source-data-ed-fig8; locator: Source Data Extended Data Fig. 8, sheet "Panel A"; source id: perturbation-ahlmann-20261011-source-article; locator: Fig. 2 and Extended Data Fig. 8 legends; main text paragraphs 13 to 16; Methods
- source locator
- perturbation-ahlmann-20261011-source-article: Fig. 2 and Extended Data Fig. 8 legends and Methods; perturbation-ahlmann-20261011-source-data-fig2: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig8: sheet "Panel A"
- revision
- 1
- reason
- Recorded from the Ahlmann-Eltze et al. 2025 Source Data pass 2026-10-11 (data/omics/perturbation-ahlmann-derived-20261011); independently reviewed 2026-10-11.
- comparison group
- ahlmann2025-perturbation-baselines
- comparison title
- Deep learning and foundation models against simple baselines (Ahlmann-Eltze et al. 2025)
- headline metric
- l2-distance
- stratum label
- Replogle K562 (CRISPRi)
- stratum order
- 3
- review
- method: source-hash-verification; ai-assisted-source-review; reviewer: claude; reviewer note: Separate Claude (Opus 5.5) review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-11T03:00:00Z; artifact sha256: dc5ccf62cb51d301f327a3c3e629d60be719bb2aaf02b57fb4999be43a4d95c7; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12328236/fullTextXML; method note: Checked the endpoint against the use case's question, inputs, output and exclusions; checked proxy against the relevance definitions; checked the limitations against the article and the Source Data evidence concerns; checked that every model on the protocol was scored on the same perturbations in each split. Checked 7 evaluations (listed in reviewed_evaluations); none excluded.; note: Proxy relevance confirmed in independent review. Pins are set by the integrator with use-cases:repin after the records are in the store. Not human scientific review. See docs/reviews/use-cases/genetic-perturbation-response-2026-10-11.md.
- reviewed evaluations
- perturbation-ahlmann-20261011-eval-replogle-k562-gears; perturbation-ahlmann-20261011-eval-replogle-k562-geneformer; perturbation-ahlmann-20261011-eval-replogle-k562-linear-model; perturbation-ahlmann-20261011-eval-replogle-k562-mean; perturbation-ahlmann-20261011-eval-replogle-k562-scbert; perturbation-ahlmann-20261011-eval-replogle-k562-scgpt; perturbation-ahlmann-20261011-eval-replogle-k562-uce
- pins
- record id: perturbation-ahlmann-20261011-eval-replogle-k562-gears; fields: status; links; attributes.origin; attributes.comparison; sha256: 240036faac7d09263a23774e7647d90daa67b2dbe989478a81bb7bf48462ed95; record id: perturbation-ahlmann-20261011-eval-replogle-k562-geneformer; fields: status; links; attributes.origin; attributes.comparison; sha256: 0712f4f1d46a69a076a31694eb3ac3504a7703e67e7ddbb5de7c539457e93164; record id: perturbation-ahlmann-20261011-eval-replogle-k562-linear-model; fields: status; links; attributes.origin; attributes.comparison; sha256: 101344056cf489953e2d5103990165e6e50360b9045a10039c643c00f3082658; record id: perturbation-ahlmann-20261011-eval-replogle-k562-mean; fields: status; links; attributes.origin; attributes.comparison; sha256: 7ae5fc02b41906e1275aeabf2168e20fca31a7989f01e09691e0df54903ca952; record id: perturbation-ahlmann-20261011-eval-replogle-k562-scbert; fields: status; links; attributes.origin; attributes.comparison; sha256: f558cf00a9083daf55dd9a9fc807d270a865fc7f1166fb191bc16310b2eacea0; record id: perturbation-ahlmann-20261011-eval-replogle-k562-scgpt; fields: status; links; attributes.origin; attributes.comparison; sha256: effc43120db85590dd609357f03792db922c25f2a577e34eb46b07b019957813; record id: perturbation-ahlmann-20261011-eval-replogle-k562-uce; fields: status; links; attributes.origin; attributes.comparison; sha256: 09299a6bc7427d49c038c6728ed78aad7633f5e0bbff59439b5ddccb6a4a1121; record id: perturbation-ahlmann-20261011-protocol-replogle-k562-single; fields: status; name; description; facets; source_ids; links; attributes; sha256: 8d11d187ea3a57d5e85c605397805470e371f7618ddff274d5320678fbc2f336; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-gears; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 52796eee5d7934dcc35f5dd3dda408aad950e34b4a749b8b29aa6ba0a9a6132f; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-geneformer; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 361d16a1107c00dd66ea79035ebb7577d883475772695ea4aca586f6b1272be0; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-linear-model; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cc5ee418d0acb3e6a2ef99c7891b0621596617f96c3fbd371702bb3c4815b737; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-mean; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 48e729f36d887fe9697ffeb63003f8eeb7a4a799cb27bbc8c11b3f8ce0d44235; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-scbert; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 48e729f36d887fe9697ffeb63003f8eeb7a4a799cb27bbc8c11b3f8ce0d44235; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-scgpt; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: db3d6fd7c5117d5cc70cd5572734948f2e66824236fa8d3310248f041d351b66; record id: perturbation-ahlmann-20261011-result-ed8a-replogle-k562-uce; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5b4b5484388286b0db08b676188ddd89eb3cd9dfdfc72a0428d49775be08e99f; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-gears; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: db3f774273f8ce8cf1b2703b6033fae8dc85e973be422f07fba5ade7ab5ac6a7; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-geneformer; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a2c92a4bcc48c813197e75660d142b3e489252c8c617931ab9d66f565c9787d4; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-linear-model; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 717b8c429476f5d9b33cc6305b5ebf77625bf8894bbd079e86c720f64089f262; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-mean; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b76508ac04321b1ab324e2aff44f7c5685f6a7ce676b32ef96c5b547c907387c; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-scbert; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3addb2b3da4b91f1ff784e775209d23800efe16c39ad14b3ebd60a0db4eacda7; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-scgpt; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 227cef9771e624d6c1c23285f5a9c6b241537e9efa84967872618bbb75c86685; record id: perturbation-ahlmann-20261011-result-fig2a-replogle-k562-uce; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cc0e04b2aa93cac8936e5c1af20a257082ce213139ae4d5e44cb0aa319460855; record id: perturbation-ahlmann-20261011-source-article; fields: status; attributes.artifact_sha256; sha256: 4cc432f631d9f7012a62ce39d0f9ffc6b889422623700e3bb0e10b33d63fef61; record id: perturbation-ahlmann-20261011-source-data-ed-fig8; fields: status; attributes.artifact_sha256; sha256: a0520d5cb1369d0db08f9544dfd6d9de6f0fe5ffba5fafae44fb475fbccafc89; record id: perturbation-ahlmann-20261011-source-data-fig2; fields: status; attributes.artifact_sha256; sha256: fea2ac6068247d325bba31c43f03fafeb20ae2ab31fb5d7a986c58766d4b38fe; record id: use-case-genetic-perturbation-response; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 440f0f2e30cb7c828212bed728de2420ba8e13521280be58be7bdcdfe57ac973