Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101)
Genome Biology · 2025 · Version of record, Genome Biology 26:101, published 2025-04-18 · Peer-reviewed
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- Immune (330k): overlap with scGPT pretraining (Kedzierska et al. 2025): Pretraining overlap:scGPT The Immune (Cross-tissue Immune Cell Atlas) dataset was included in the scGPT pretraining data (CELLxGENE May 2023 census). (Additional file 1, sheet 'TableS2' cell H12 (column 'scGPT', row 'Immune (330k)'); page 9 'Datasets')
- Pancreas (8k) (Kedzierska et al. 2025): description: Description Row labelled 'Pancreas (8k)' in the score sheets: the Pancreas subset without GEO GSE84133, which was in Geneformer pretraining. The Methods (page 9) say both the complete Pancreas dataset and its non-overlapping subset were scored 'in Additional file 1: Tables S1 and S5', which are the two score sheets, and this is their only extra Pancreas row. The workbook itself does not name the subset. (Article page 9 'Datasets' paragraph 3; Additional file 1 sheets TableS6 and TableS7 row 'Pancreas (8k)')
- Pancreas (16k): overlap with Geneformer pretraining (Kedzierska et al. 2025): Pretraining overlap:Geneformer The Pancreas study GSE84133 (Baron et al.) was included in Geneformer pretraining; the other five Pancreas studies were not. (Additional file 1, sheet 'TableS2' cell G6 (column 'Geneformer', row 'GEO - GSE84133'); page 9 'Datasets')
- Tabula Sapiens (483k): overlap with scGPT pretraining (Kedzierska et al. 2025): Pretraining overlap:scGPT Tabula Sapiens was included in the scGPT pretraining data (CELLxGENE May 2023 census). (Additional file 1, sheet 'TableS2' cell H11 (column 'scGPT', row 'Tabula Sapiens (483k)'); page 9 'Datasets')
Models, benchmarks and other records
Methods (1)
Configurations (8)
- 2,000 highly variable genes (Kedzierska et al. 2025)
- Geneformer (Kedzierska et al. 2025)
- Harmony (Kedzierska et al. 2025)
- scGPT blood (Kedzierska et al. 2025)
- scGPT human (Kedzierska et al. 2025)
- scGPT kidney (Kedzierska et al. 2025)
- scGPT, randomly initialised (Kedzierska et al. 2025)
- scVI (Kedzierska et al. 2025)
Benchmarks (1)
Protocols (3)
Dataset subsets (7)
Evaluations (113)
- Geneformer cell embedding on evaluated datasets (Kedzierska et al. 2025)
- Geneformer cell embedding on Immune (330k) (Kedzierska et al. 2025)
- Geneformer cell embedding on Pancreas (8k) (Kedzierska et al. 2025)
- Geneformer cell embedding on Pancreas (16k) (Kedzierska et al. 2025)
- Geneformer cell embedding on PBMC (12k) (Kedzierska et al. 2025)
- Geneformer cell embedding on PBMC (95k) (Kedzierska et al. 2025)
- Geneformer cell embedding on Tabula Sapiens (483k) (Kedzierska et al. 2025)
- Geneformer pretraining reconstruction on evaluated datasets (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on evaluated datasets (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on Immune (330k) (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on Pancreas (8k) (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on Pancreas (16k) (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on PBMC (12k) (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on PBMC (95k) (Kedzierska et al. 2025)
- Geneformer UMAP-projected embedding on Tabula Sapiens (483k) (Kedzierska et al. 2025)
- Harmony cell embedding on evaluated datasets (Kedzierska et al. 2025)
- Harmony cell embedding on Immune (330k) (Kedzierska et al. 2025)
- Harmony cell embedding on Pancreas (8k) (Kedzierska et al. 2025)
- Harmony cell embedding on Pancreas (16k) (Kedzierska et al. 2025)
- Harmony cell embedding on PBMC (12k) (Kedzierska et al. 2025)
- Harmony cell embedding on PBMC (95k) (Kedzierska et al. 2025)
- Harmony cell embedding on Tabula Sapiens (483k) (Kedzierska et al. 2025)
- Harmony UMAP-projected embedding on evaluated datasets (Kedzierska et al. 2025)
- Harmony UMAP-projected embedding on Immune (330k) (Kedzierska et al. 2025)
- Harmony UMAP-projected embedding on Pancreas (8k) (Kedzierska et al. 2025)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
28 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 8efcc5fcb2e35b9a04b1ee9e03581e20fffeb5521a4a4c748a6f9b34f7d3c47f Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://genomebiology.biomedcentral.com/counter/pdf/10.1186/s13059-025-03574-x.pdf Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Doi 10.1186/s13059-025-03574-x Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Licence CC-BY-4.0 Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations2 values
| Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Media type application/pdf Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Publication status peer_reviewed Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-10-14T12:01:07Z Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Review: artifact sha256 8efcc5fcb2e35b9a04b1ee9e03581e20fffeb5521a4a4c748a6f9b34f7d3c47f Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Review: date 2026-10-14 Source metadata | Zero-shot evaluation reveals limitations of single-cell foundation models (Kedzierska et al. 2025, Genome Biology 26:101) No field-specific location recorded Version: Version of record, Genome Biology 26:101, published 2025-04-18 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-23-5347695500ed · Record review: source checked
Publisher PDF of the article, retrieved and hashed for the 2026-10-14 genomic foundation model guide fixes (issue rewire-benchmark-data#129).
Technical metadata and extraction receipts
Stable ID: gfm-guides-20261014-kedzierska2025-source-article
- areas
- single-cell
- contexts
- research
- url
- https://doi.org/10.1186/s13059-025-03574-x
- artifact url
- https://genomebiology.biomedcentral.com/counter/pdf/10.1186/s13059-025-03574-x.pdf
- version
- Version of record, Genome Biology 26:101, published 2025-04-18
- retrieved at
- 2026-10-14T12:01:07Z
- artifact sha256
- 8efcc5fcb2e35b9a04b1ee9e03581e20fffeb5521a4a4c748a6f9b34f7d3c47f
- doi
- 10.1186/s13059-025-03574-x
- publication status
- peer_reviewed
- licence
- CC-BY-4.0
- media type
- application/pdf
- venue
- Genome Biology
- year
- 2025
- limitations
- The headline comparisons (Fig. 1B average BIO score, Fig. 1E average batch score, Fig. 2 reconstruction) are figures without printed values; the per-dataset scores are in Additional file 1.; All five evaluation datasets were chosen by the authors; Tabula Sapiens and Immune were in scGPT pretraining and part of Pancreas in Geneformer pretraining (Additional file 1, sheet TableS2).
- review
- method: source-hash-verification; method note: A fresh download on 2026-10-14 returned a 3,038-byte bot-check HTML page instead of the PDF; the archived copy in artifacts/ hashes to the pinned SHA-256 and was used.; reviewer: claude; reviewer note: Claude (Opus 5.5) research agent, independent of the extractor; no human review claimed; date: 2026-10-14; artifact sha256: 8efcc5fcb2e35b9a04b1ee9e03581e20fffeb5521a4a4c748a6f9b34f7d3c47f; retrieval url: https://genomebiology.biomedcentral.com/counter/pdf/10.1186/s13059-025-03574-x.pdf; note: Independent review 2026-10-14.