Protocol

Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a)

Models are trained on all single and half of the double perturbations and predict expression for the 62 held-out doubles in each of five random splits. Error is the L2 distance, and agreement the Pearson delta, between predicted and observed expression over the 1,000 most highly expressed genes in the control condition, per perturbation.

9 evaluations · 17 results

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

9 recorded evaluations, 17 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

9 evaluations · 17 results. Different protocols are not a single leaderboard.

Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) · Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)

Sorted by L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits) (lower is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)	Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)
Additive (Ahlmann-Eltze et al. 2025)Evaluation	2.77Computed by Rewire from the authors' source data (best in column)	0.892Computed by Rewire from the authors' source data (best in column)
Geneformer (Ahlmann-Eltze et al. 2025)Evaluation	3.48Computed by Rewire from the authors' source data	0.773Computed by Rewire from the authors' source data
GEARS (Ahlmann-Eltze et al. 2025)Evaluation	4.1Computed by Rewire from the authors' source data	0.631Computed by Rewire from the authors' source data
scBERT (Ahlmann-Eltze et al. 2025)Evaluation	4.28Computed by Rewire from the authors' source data	0.61Computed by Rewire from the authors' source data
UCE (Ahlmann-Eltze et al. 2025)Evaluation	4.28Computed by Rewire from the authors' source data	0.61Computed by Rewire from the authors' source data
scGPT (Ahlmann-Eltze et al. 2025)Evaluation	4.62Computed by Rewire from the authors' source data	0.527Computed by Rewire from the authors' source data
scFoundation (Ahlmann-Eltze et al. 2025)Evaluation	4.72Computed by Rewire from the authors' source data	0.537Computed by Rewire from the authors' source data
No change (Ahlmann-Eltze et al. 2025)Evaluation	5.62Computed by Rewire from the authors' source data	Not reported
CPA (Ahlmann-Eltze et al. 2025)Evaluation	12.2Computed by Rewire from the authors' source data	0.146Computed by Rewire from the authors' source data

All 17 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: Additive (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.892 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Author-reported evaluation · Source checked Methods, coverage and source Additive on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "additive_model", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: CPA (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.146 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source CPA on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "cpa", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: GEARS (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.631 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source GEARS on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "gears", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: Geneformer (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.773 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source Geneformer on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "geneformer", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: scBERT (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.61 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source scBERT on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "scbert", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: scFoundation (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.537 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source scFoundation on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "scfoundation", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: scGPT (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.527 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source scGPT on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "scgpt", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: UCE (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	0.61 Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unitless · higher Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source UCE on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025 · Source Data Extended Data Fig. 2 (MOESM6), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "uce", column "r2_delta"; plotted as the red line in Extended Data Fig. 2a
Configuration: Additive (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	2.77 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Author-reported evaluation · Source checked Methods, coverage and source Additive on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "additive_model", column "l2"; plotted as the red line in Fig. 1a
Configuration: CPA (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	12.2 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source CPA on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "cpa", column "l2"; plotted as the red line in Fig. 1a
Configuration: GEARS (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	4.1 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source GEARS on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "gears", column "l2"; plotted as the red line in Fig. 1a
Configuration: Geneformer (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	3.48 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source Geneformer on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "geneformer", column "l2"; plotted as the red line in Fig. 1a
Configuration: No change (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	5.62 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Author-reported evaluation · Source checked Methods, coverage and source No change on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "no_change", column "l2"; plotted as the red line in Fig. 1a
Configuration: scBERT (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	4.28 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source scBERT on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "scbert", column "l2"; plotted as the red line in Fig. 1a
Configuration: scFoundation (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	4.72 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source scFoundation on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "scfoundation", column "l2"; plotted as the red line in Fig. 1a
Configuration: scGPT (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	4.62 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source scGPT on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "scgpt", column "l2"; plotted as the red line in Fig. 1a
Configuration: UCE (Ahlmann-Eltze et al. 2025)	Protocol: Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) Dataset: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)	4.28 L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)Computed by Rewire from the authors' source data unit-unreported · lower Uncertainty: Not reported by the source: No spread of the mean is stated; Rewire did not compute one Coverage: 310/310 perturbation-split rows scored	Independent external evaluation · Source checked Methods, coverage and source UCE on Norman doubles (Ahlmann-Eltze et al. 2025) perturbation-ahlmann-20261011-protocol-norman-double Aggregation: Not reported Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025 · Source Data Fig. 1 (MOESM3), sheet "Panel A", rows with dataset_name "norman_from_scfoundation" and method "uce", column "l2"; plotted as the red line in Fig. 1a

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 2 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

No execution recipe linked to this protocol. Recipe availability does not establish a completed evaluation.

Author-reported evaluations: 2
External evaluations: 7

Literature evidence is not a Rewire measurement. Executed but unpublished runs and private review status are not included.

Null control

Proposed control: requires review

No-change prediction under matched control conditions

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Conventional reference

Proposed control: requires review

Training-only mean-effect or linear prediction

Protocol-specific applicability, permitted inputs, access, split, evaluator and execution requirements need review before implementation or execution.

This is a suggested selection rule, not a validated method or a measured score.

Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)

Coverage is derived from release 2026-10-22-1b7cf027e4f6. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this protocol. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Strengths, limitations and unresolved questions

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance

No evidence rows match these filters. Choose another scope or clear the search.

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

1 source record and release history

Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines · Original source · Nature Methods 22(8):1657, published online 2025-08-04; PMC12328236 full-text XML

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: perturbation-ahlmann-20261011-protocol-norman-double

areas: cells-tissues; single-cell
contexts: research
protocol: Models are trained on all single and half of the double perturbations and predict expression for the 62 held-out doubles in each of five random splits. Error is the L2 distance, and agreement the Pearson delta, between predicted and observed expression over the 1,000 most highly expressed genes in the control condition, per perturbation.
scope note: Split: All 100 single perturbations and a random half (62) of the 124 doubles for training; the other 62 doubles held out (31 labelled test and 31 val in the Source Data); repeated with five random splits. 310 held-out perturbation-split rows per model in the Source Data.
denominator: 310
metric definition: L2 distance: square root of the summed squared differences over the 1,000 most highly expressed genes in the control condition. Pearson delta: correlation of predicted and observed expression after subtracting the control expression (Methods 'Evaluation metrics').
aggregation: Mean per model over held-out perturbations, pooling the rows labelled test and val and all splits ("The horizontal red lines show the mean per model").
source locator: Main text paragraphs 4 and 6; Fig. 1 legend; Extended Data Fig. 2 legend; Methods 'Double perturbation benchmark setup' and 'Evaluation metrics'
limitations: The paper prints no per-model value; the means are computed by Rewire from the authors' Source Data rows and have not been checked against a printed number.; Four datasets, all from cancer cell lines, chosen because the GEARS, scGPT and scFoundation publications used them; the original quality control was not improved, so perturbations that did not affect their own target gene are included (main text paragraph 22).; The authors ran every model themselves with default parameters as far as possible and asked the model developers to review their code (Methods 'Software versions and parameters').; UCE and scBERT post-perturbation embeddings use ground-truth expression of the perturbed genes; the authors note for UCE that this could leak test data (Methods 'Software versions and parameters').
missing metadata: uncertainty: reason: unreported; note: The figures show per-perturbation points and the mean; no spread of the mean is stated

Related records

uses data: Norman et al. 2019 Perturb-seq, K562 CRISPRa single and double perturbations (scFoundation reprocessing, Ahlmann-Eltze et al. 2025)
subject: reported_finding: perturbation-ahlmann-20261011-protocol-norman-double
subject: genetic_interaction_finding: perturbation-ahlmann-20261011-protocol-norman-double
assessment: Additive on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: CPA on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: GEARS on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: Geneformer on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: No change on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: scBERT on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: scFoundation on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: scGPT on Norman doubles (Ahlmann-Eltze et al. 2025)
assessment: UCE on Norman doubles (Ahlmann-Eltze et al. 2025)
assessed by: Assess models for genetic perturbation experiments

Suggest a correction