Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 8 results
• 2 evidence claims

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• Evidence summary: genetic perturbation response prediction: Summary The five sources whose comparisons are shown on this page score how well models predict expression changes after genetic perturbations they were not trained on. Each uses its own split, gene set and metric, so their values are not pooled. Ahlmann-Eltze et al. 2025, who did not develop the deep models and ran them all themselves, scored held-out two-gene CRISPRa perturbations in K562 cells (Norman), training on every single-gene perturbation and half of the pairs. A baseline that adds the two single-gene effects had a mean L2 distance of 2.766 and a Pearson delta of 0.892 over the 1,000 most highly expressed genes, against 3.478 and 0.773 for Geneformer with a linear decoder, 4.096 and 0.631 for GEARS, 4.279 and 0.610 for both UCE and scBERT, 4.616 and 0.527 for scGPT, 4.722 and 0.537 for scFoundation, and 12.205 and 0.146 for CPA; predicting no change gave an L2 distance of 5.616. Rewire computed these means from the authors' Source Data, no spread is stated, and the UCE and scBERT runs used ground-truth expression of the perturbed genes, a possible test leakage. In PerturBench v4, a preprint whose authors trained every model and built the Linear, Latent Additive and Decoder baselines, the cosine similarity of predicted log fold change on held-out Norman pairs ranged from 0.41 for Biolord and 0.44 for GEARS to 0.79 for the Latent Additive baseline, with CPA at 0.76 and the Linear baseline at 0.60, each a mean over five training seeds. When effects were transferred to melanoma co-culture (Frangieh21) every model scored between 0.01 and 0.22 and every printed DEG recall was 0.000; on held-out cell line and cytokine states (Jiang24) scores ranged from 0.17 for Linear to 0.64 for the Decoder baseline, and the paper describes the held-out states in two different ways. In Csendes et al. 2025, whose authors are employees of Turbine Ltd. and built the baselines themselves, predicting the mean training response gave a higher Pearson delta than scGPT and scFoundation, as re-run by the same authors, on all four Perturb-seq datasets: 0.711, 0.557, 0.373 and 0.628 for Adamson, Norman, Replogle K562 and Replogle RPE1, against 0.641, 0.554, 0.327 and 0.596 for scGPT and 0.552, 0.459, 0.269 and 0.471 for scFoundation. Each dataset there has one split and one run per model with no spread printed, so small gaps such as 0.557 against 0.554 cannot be tested. The GEARS developers report a mean squared error of 0.216 for GEARS on Norman, against 0.354 for CPA and 0.551 for a no-change control, with spreads of unstated type. PertEval-scFM, whose authors trained GEARS from scratch, reports an area under its distribution-shift curve of 0.815 for GEARS against 4.484 for an MLP and 4.612 for a mean baseline, lower being better. All of these scores measure agreement with observed expression in held-out perturbations; they do not show which experiments will succeed. (perturbation-ahlmann-20261011-source-data-fig1: sheet 'Panel A', Norman, column l2; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet 'Panel A', Norman, column r2_delta; perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends; Methods; lit-cells-networks-20261011-perturbench-v4-source-wu2025: Tables 3, 4 and 7, columns 'Cosine logFC' and 'DEG recall'; perturbation-response-20261009-source-csendes2025-supp: Supplementary Tables 1 and 2; perturbation-response-20261009-source-csendes2025: Results; Methods; Competing interests; coverage-source-gears-supp: Table 6, MSE; perteval-scfm-2025-source: Table 1, Norman single-gene AUSPC)
• Relevance of Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) to "Assess models for genetic perturbation experiments": Assessed by Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a) (perturbation-ahlmann-20261011-source-article: Fig. 1 and Extended Data Fig. 2 legends and Methods; perturbation-ahlmann-20261011-source-data-fig1: sheet "Panel A"; perturbation-ahlmann-20261011-source-data-ed-fig2: sheet "Panel A")

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.