rewirebio.iobenchmarks
Contested result

Geneformer and scGPT: Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx) reverses the order

  • Low severity
  • Not yet reviewed
  • Ordering reversal
  • unknown

Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx) ranks scGPT above Geneformer in 83 of 89 comparisons; 2 of 2 other studies rank Geneformer above scGPT.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

4 papers rank Geneformer above scGPT in 10 of their 19 comparisons and scGPT higher in 5; 4 are ties. Of those papers, Parameter-Efficient Fine-Tuning… ranks scGPT higher on its own (1 of 1). Kedzierska et al. 2025 ranks scGPT above Geneformer in 83 of its 97 comparisons and Geneformer higher in 6; 8 are ties.

Papers grouped by which system they rank higher: Geneformer or scGPTRank Geneformer higher: Wu et al. 2025, Deep-learning-based gene perturbation…. Rank scGPT higher: Kedzierska et al. 2025 (disagrees), Parameter-Efficient Fine-Tuning…. Mixed: HyCell CITATION.cff.Rank Geneformer higherMixedRank scGPT higherWu et al. 2025Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13)Deep-learning-basedgene perturbation…Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselinesHyCellCITATION.cffHyCell CITATION.cff (commit 0dcc533c)Kedzierska et al. 2025Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx)Parameter-EfficientFine-Tuning…Parameter-Efficient Fine-Tuning Enhances Adaptation of Single Cell Large Language Model for Cell Type Identification
  • Rank Geneformer higher (2): Wu et al. 2025, Deep-learning-based gene perturbation…
  • Rank scGPT higher (2): Kedzierska et al. 2025 (disagrees), Parameter-Efficient Fine-Tuning…
  • Mixed (1): HyCell CITATION.cff

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores Geneformer or scGPT higherKedzierska et al. 2025 (disagrees): Geneformer higher in 6, scGPT higher in 83, 8 ties. HyCell CITATION.cff: Geneformer higher in 5, scGPT higher in 3, 4 ties. Wu et al. 2025: Geneformer higher in 3, scGPT higher in 1, 0 ties. Deep-learning-based gene perturbation…: Geneformer higher in 2, scGPT higher in 0, 0 ties. Parameter-Efficient Fine-Tuning…: Geneformer higher in 0, scGPT higher in 1, 0 ties.Disagrees: Kedzierska et al. 2025Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx)6 Geneformer higher8 tie83 scGPT higher83HyCell CITATION.cffHyCell CITATION.cff (commit 0dcc533c)5 Geneformer higher54 tie43 scGPT higher3Wu et al. 2025Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13)3 Geneformer higher31 scGPT higher1Deep-learning-based gene perturbation…Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines2 Geneformer higher2Parameter-Efficient Fine-Tuning…Parameter-Efficient Fine-Tuning Enhances Adaptation of Single Cell Large Language Model for Cell Type Identification1 scGPT higher1
Geneformer higherTiescGPT higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in Kedzierska et al. 2025 (97)
ProtocolDatasetMetricGeneformerscGPTHigher
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (overall average ranking across datasets, based on mean biological-conservation and batch-mixing scores (1 = best); full embedding dimensions), unitless85scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median ARI score across datasets among the eight representations (1 = best); full embedding dimensions), unitless74scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median ASW score across datasets among the eight representations (1 = best); full embedding dimensions), unitless83scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median Average BIO score across datasets among the eight representations (1 = best); full embedding dimensions), unitless75scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median Average batch score across datasets among the eight representations (1 = best); full embedding dimensions), unitless87scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median Batch integration score across datasets among the eight representations (1 = best); full embedding dimensions), unitless74scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median NMI score across datasets among the eight representations (1 = best); full embedding dimensions), unitless85scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (rank of the median PCR score across datasets among the eight representations (1 = best); full embedding dimensions), unitless87scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.190.38scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions), unitless0.380.59scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions), unitless0.350.51scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.420.62scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions), unitless85scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions), unitless0.760.89scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; full embedding dimensions), unitless0.450.52scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsImmune (330k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions), unitless00.28scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.090.34scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions), unitless0.30.43scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions), unitless0.240.44scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.260.5scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions), unitless84scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions), unitless0.60.78scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; full embedding dimensions), unitless0.370.49scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (16k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions), unitless00.08scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.050.14scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions), unitless0.360.54scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions), unitless0.220.35scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.160.4scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions), unitless0.730.78scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; full embedding dimensions), unitless0.460.5scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPancreas (8k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions), unitless00.31scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.60.89scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions), unitless0.480.49scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions), unitless0.590.78scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.650.85scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions), unitless83scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions), unitless0.970.97Tie
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; full embedding dimensions), unitless0.510.61scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (12k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions), unitless00Tie
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (95k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.40.46scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (95k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions), unitless0.520.55scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (95k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.650.65Tie
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (95k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions), unitless63scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsPBMC (95k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; full embedding dimensions), unitless0.510.54scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.140.29scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions), unitless0.450.45Tie
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions), unitless0.30.44scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions), unitless0.350.55scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions), unitless86scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions), unitless0.90.89Geneformer
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; full embedding dimensions), unitless0.410.47scGPT
Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsTabula Sapiens (483k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions), unitless00Tie
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionEvaluated datasets, aggregated (Kedzierska et al. 2025)Rank (overall average ranking across datasets, based on mean biological-conservation and batch-mixing scores (1 = best); UMAP projection of the embedding), unitless85scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.120.35scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding), unitless0.240.48scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding), unitless0.290.47scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.340.59scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding), unitless84scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding), unitless0.490.59scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding), unitless0.40.47scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionImmune (330k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding), unitless00.36scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.070.26scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding), unitless0.180.24scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding), unitless0.220.4scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.240.46scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding), unitless84scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding), unitless0.370.47scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding), unitless0.350.48scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (16k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding), unitless00Tie
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.020.13scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding), unitless0.260.19Geneformer
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding), unitless0.20.29scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.160.37scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding), unitless0.530.38Geneformer
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding), unitless0.420.39Geneformer
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPancreas (8k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding), unitless00Tie
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.370.51scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding), unitless0.710.85scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding), unitless0.510.66scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.580.74scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding), unitless75scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding), unitless0.890.89Tie
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding), unitless0.60.74scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (12k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding), unitless0.530.81scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (95k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.30.43scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (95k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding), unitless0.450.54scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (95k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.480.59scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (95k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding), unitless68Geneformer
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionPBMC (95k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding), unitless0.580.61scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Adjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.110.33scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Average batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding), unitless0.350.37scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Avg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding), unitless0.220.4scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Normalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding), unitless0.30.52scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Rank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding), unitless83scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Scib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding), unitless0.710.65Geneformer
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Scib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding), unitless0.260.35scGPT
Kedzierska et al. 2025 cell embedding scoring after UMAP projectionTabula Sapiens (483k) (Kedzierska et al. 2025)Scib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding), unitless00.09scGPT
Comparisons in HyCell CITATION.cff (12)
ProtocolDatasetMetricGeneformerscGPTHigher
Annotation transfer with 15-nearest neighbours (HyCell report)Seven held-out atlases of the HyCell reportMacro F1 (Macro-F1 of a 15-nearest-neighbour classifier trained on the other batches, over the cell types of the held-out batch, averaged over held-out batches; per atlas or averaged over the seven atlases), fraction0.7820.765Geneformer
Annotation transfer with logistic regression (HyCell report)Seven held-out atlases of the HyCell reportMacro F1 (Macro-F1 of a logistic-regression classifier trained on the other batches, over the cell types of the held-out batch, averaged over held-out batches; per atlas or averaged over the seven atlases), fraction0.8340.825Geneformer
scIB batch correction (HyCell report)Seven held-out atlases of the HyCell reportScib batch correction score (Mean of five scIB batch-correction metrics (batch silhouette, iLISI, kBET, graph connectivity and principal-component regression), per atlas or averaged over the seven atlases), unitless0.4650.46Geneformer
scIB biological conservation (HyCell report)Seven held-out atlases of the HyCell reportScib bio conservation score (Mean of seven scIB biological-conservation metrics (isolated-label score, NMI and ARI of k-means and of Leiden clusters against the cell types, cell-type silhouette and cLISI), per atlas or averaged over the seven atlases), unitless0.6060.634scGPT
Composite score S (HyCell report)Asian Immune Diversity Atlas, sample (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.630.599Geneformer
Composite score S (HyCell report)MultiVINE brain vasculature, healthy donors (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.7560.756Tie
Composite score S (HyCell report)Cross-tissue immune atlas, stratified sample (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.6340.624Geneformer
Composite score S (HyCell report)scIB human immune atlas, integer-count part (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.6830.689Tie
Composite score S (HyCell report)scIB human lung atlas, integer-count part (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.6530.647Tie
Composite score S (HyCell report)scIB human pancreas atlas, integer-count part (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.5790.61scGPT
Composite score S (HyCell report)PBMC from two 10x Genomics runs (HyCell report)Weighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.8160.834scGPT
Composite score S (HyCell report)Seven held-out atlases of the HyCell reportWeighted composite score (S = 0.3 Bio + 0.2 Batch + 0.25 LR + 0.25 kNN, the authors' weighting, which gives the scIB total (0.6 Bio + 0.4 Batch) and annotation transfer equal weight; per atlas or averaged over the seven atlases), unitless0.6790.68Tie
Comparisons in Wu et al. 2025 (4)
ProtocolDatasetMetricGeneformerscGPTHigher
Cross-atlas annotation transfer, HLCA to Tabula Sapiens, 14 shared cell types (Wu et al. 2025 Table S3)Tabula Sapiens (CELLxGENE), 120 leaf cell types after processingMacro F1 (test cells of the 14 shared cell types; averaged over 'all classes' (Methods 'Macro-F1'), not stated as the 14 shared types or every source-atlas type), fraction0.2390.209Geneformer
Cross-atlas annotation transfer, HLCA to Tabula Sapiens, 14 shared cell types (Wu et al. 2025 Table S3)Tabula Sapiens (CELLxGENE), 120 leaf cell types after processingTop-1 accuracy (Accuracy@1 over cells of the 14 shared cell types), fraction0.7720.614Geneformer
Cross-atlas annotation transfer, Tabula Sapiens to HLCA, 14 shared cell types (Wu et al. 2025 Table S2)HLCA core (CELLxGENE), 37 leaf cell types after processingMacro F1 (test cells of the 14 shared cell types; averaged over 'all classes' (Methods 'Macro-F1'), not stated as the 14 shared types or every source-atlas type), fraction0.0870.081Geneformer
Cross-atlas annotation transfer, Tabula Sapiens to HLCA, 14 shared cell types (Wu et al. 2025 Table S2)HLCA core (CELLxGENE), 37 leaf cell types after processingTop-1 accuracy (Accuracy@1 over cells of the 14 shared cell types), fraction0.6650.674scGPT
Comparisons in Deep-learning-based gene perturbation… (2)
Comparisons in Parameter-Efficient Fine-Tuning… (1)
ProtocolDatasetMetricGeneformerscGPTHigher
Cell-type identificationM.S. single-cell datasetF1, unitless0.3880.734scGPT

What disagrees

Other studies
rank Geneformer above scGPT in 5 of 6 comparisons
This study
ranks scGPT above Geneformer in 83 of 89 comparisons
Compared
Geneformer, scGPT
Study that disagrees
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx), Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)
Other studies
6 studiesWu et al. 2025, Additional file 3 (Supplementary Tables S1-S13), Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines, Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025, Source Data Extended Data Fig. 8, Ahlmann-Eltze et al. 2025, Source Data Fig. 1, Ahlmann-Eltze et al. 2025, Source Data Fig. 2, Ahlmann-Eltze et al. 2025
Rule
Ordering reversal; unknown
Model identity
Matched across studies partly through links not yet reviewed

The compared numbers

Each value links to its result record and source. Compare values within one protocol; the rule compares rankings, not raw values across protocols.

In the study that disagrees (178)
StudyToolProtocol or benchmarkMetricValue
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (overall average ranking across datasets, based on mean biological-conservation and batch-mixing scores (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median ARI score across datasets among the eight representations (1 = best); full embedding dimensions)7
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median ASW score across datasets among the eight representations (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median Average batch score across datasets among the eight representations (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median Average BIO score across datasets among the eight representations (1 = best); full embedding dimensions)7
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median Batch integration score across datasets among the eight representations (1 = best); full embedding dimensions)7
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median NMI score across datasets among the eight representations (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median PCR score across datasets among the eight representations (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (overall average ranking across datasets, based on mean biological-conservation and batch-mixing scores (1 = best); full embedding dimensions)5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median ARI score across datasets among the eight representations (1 = best); full embedding dimensions)4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median ASW score across datasets among the eight representations (1 = best); full embedding dimensions)3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median Average batch score across datasets among the eight representations (1 = best); full embedding dimensions)7
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median Average BIO score across datasets among the eight representations (1 = best); full embedding dimensions)5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median Batch integration score across datasets among the eight representations (1 = best); full embedding dimensions)4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median NMI score across datasets among the eight representations (1 = best); full embedding dimensions)5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank of the median PCR score across datasets among the eight representations (1 = best); full embedding dimensions)7
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.19
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.45
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.38
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.76
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.42
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions)0
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.38
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.52
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.59
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.51
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.89
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.62
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions)0.28
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.09
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.37
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.24
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.6
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.26
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions)0
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.34
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.49
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.43
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.44
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.78
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions)0.08
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.05
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.46
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.36
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.22
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.73
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.16
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions)0
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.14
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.54
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.78
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib pcr batch (principal component regression on batch; whole dataset; full embedding dimensions)0.31
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.6
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.51
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.48
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.59
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.65
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.89
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.61
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; full embedding dimensions)0.49
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.78
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.85
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.51
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.52
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)6
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.46
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.54
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.55
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.14
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.41
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.9
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.29
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw label (cell-type ASW; whole dataset; full embedding dimensions)0.47
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; full embedding dimensions)0.44
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsScib asw batch (batch integration score (batch ASW per cell type); whole dataset; full embedding dimensions)0.89
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; full embedding dimensions)0.55
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring, full embedding dimensionsRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); full embedding dimensions)6
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (overall average ranking across datasets, based on mean biological-conservation and batch-mixing scores (1 = best); UMAP projection of the embedding)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (overall average ranking across datasets, based on mean biological-conservation and batch-mixing scores (1 = best); UMAP projection of the embedding)5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.12
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.24
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.29
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.49
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.34
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding)0
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.47
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.48
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.47
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.59
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.59
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding)0.36
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.07
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.18
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.22
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.37
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.24
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.26
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.48
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.24
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.47
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.46
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.02
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.42
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.26
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.2
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.53
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.16
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.13
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.39
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.19
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.29
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.38
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.37
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.37
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.6
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.71
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.51
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.58
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding)0.53
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)7
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.51
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.74
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.85
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.66
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.74
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding)0.81
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)5
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.58
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.45
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.48
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)6
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.43
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.61
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.54
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.59
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.11
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.26
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.22
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.71
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.3
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding)0
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)Geneformer (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)8
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAdjusted rand index (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.33
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw label (cell-type ASW; whole dataset; UMAP projection of the embedding)0.35
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAverage batch score (average batch score (mean of batch integration score and PCR batch score); whole dataset; UMAP projection of the embedding)0.37
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionAvg bio (AvgBIO (mean of cell-type ASW, NMI and ARI; NMI and ARI on Louvain clusters at the best-NMI resolution); whole dataset; UMAP projection of the embedding)0.4
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib asw batch (batch integration score (batch ASW per cell type); whole dataset; UMAP projection of the embedding)0.65
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionNormalized mutual information (Louvain clusters at best-NMI resolution versus cell-type labels; whole dataset; UMAP projection of the embedding)0.52
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionScib pcr batch (principal component regression on batch; whole dataset; UMAP projection of the embedding)0.09
Kedzierska et al. 2025, Additional file 1 (Supplementary Tables S1-S7, xlsx); Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)scGPT human (Kedzierska et al. 2025)Kedzierska et al. 2025 cell embedding scoring after UMAP projectionRank (rank among the eight representations by mean biological-conservation and batch-mixing scores on this dataset (1 = best); UMAP projection of the embedding)3
In the other studies (12)
StudyToolProtocol or benchmarkMetricValue
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsGeneformer zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, HLCA to Tabula Sapiens, 14 shared cell types (Wu et al. 2025 Table S3)Top-1 accuracy (Accuracy@1 over cells of the 14 shared cell types)0.772
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsGeneformer zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, HLCA to Tabula Sapiens, 14 shared cell types (Wu et al. 2025 Table S3)Macro F1 (test cells of the 14 shared cell types; averaged over 'all classes' (Methods 'Macro-F1'), not stated as the 14 shared types or every source-atlas type)0.239
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsscGPT zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, HLCA to Tabula Sapiens, 14 shared cell types (Wu et al. 2025 Table S3)Top-1 accuracy (Accuracy@1 over cells of the 14 shared cell types)0.614
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsscGPT zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, HLCA to Tabula Sapiens, 14 shared cell types (Wu et al. 2025 Table S3)Macro F1 (test cells of the 14 shared cell types; averaged over 'all classes' (Methods 'Macro-F1'), not stated as the 14 shared types or every source-atlas type)0.209
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsGeneformer zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, Tabula Sapiens to HLCA, 14 shared cell types (Wu et al. 2025 Table S2)Top-1 accuracy (Accuracy@1 over cells of the 14 shared cell types)0.665
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsGeneformer zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, Tabula Sapiens to HLCA, 14 shared cell types (Wu et al. 2025 Table S2)Macro F1 (test cells of the 14 shared cell types; averaged over 'all classes' (Methods 'Macro-F1'), not stated as the 14 shared types or every source-atlas type)0.087
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsscGPT zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, Tabula Sapiens to HLCA, 14 shared cell types (Wu et al. 2025 Table S2)Top-1 accuracy (Accuracy@1 over cells of the 14 shared cell types)0.674
Wu et al. 2025, Additional file 3 (Supplementary Tables S1-S13); Biology-driven insights into the power of single-cell foundation modelsscGPT zero-shot embeddings with OnClass (Wu et al. 2025)Cross-atlas annotation transfer, Tabula Sapiens to HLCA, 14 shared cell types (Wu et al. 2025 Table S2)Macro F1 (test cells of the 14 shared cell types; averaged over 'all classes' (Methods 'Macro-F1'), not stated as the 14 shared types or every source-atlas type)0.081
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025; Source Data Fig. 1, Ahlmann-Eltze et al. 2025Geneformer (Ahlmann-Eltze et al. 2025)Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a)Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)0.773
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Extended Data Fig. 2, Ahlmann-Eltze et al. 2025; Source Data Fig. 1, Ahlmann-Eltze et al. 2025scGPT (Ahlmann-Eltze et al. 2025)Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a)Pearson delta (1,000 most highly expressed control genes; mean over held-out perturbations and splits)0.527
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025Geneformer (Ahlmann-Eltze et al. 2025)Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a)L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)3.48
Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines; Source Data Fig. 1, Ahlmann-Eltze et al. 2025scGPT (Ahlmann-Eltze et al. 2025)Norman double perturbations: expression prediction for held-out double perturbations (Ahlmann-Eltze et al. 2025 Fig. 1a and Extended Data Fig. 2a)L2 distance (1,000 most highly expressed control genes; mean over held-out perturbations and splits)4.62

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-de4baabd59543adf · All contested results · How findings are made and followed up