Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation
arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) · Preprint
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- Relevance of Adamson K562 Perturb-seq to "Assess models for genetic perturbation experiments": Assessed by Chen et al. 2026 genetic perturbation prediction, Adamson (lit-cells-networks-20261011-chen2026-source-chen2026: Table 5; Results 'Perturbation prediction separates expression reconstruction from functional response modelling')
- Relevance of ATAA held-out annotation to "Transfer cell-type annotations to a new dataset": Assessed by Chen et al. 2026 supervised cell type annotation, ATAA (lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation')
- Relevance of BMMC held-out annotation to "Transfer cell-type annotations to a new dataset": Assessed by Chen et al. 2026 supervised cell type annotation, BMMC (lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation')
- Relevance of HS held-out annotation to "Transfer cell-type annotations to a new dataset": Assessed by Chen et al. 2026 supervised cell type annotation, HS (lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation')
- Relevance of Norman K562 Perturb-seq to "Assess models for genetic perturbation experiments": Assessed by Chen et al. 2026 genetic perturbation prediction, Norman (lit-cells-networks-20261011-chen2026-source-chen2026: Table 5; Results 'Perturbation prediction separates expression reconstruction from functional response modelling')
- Relevance of Replogle K562 Perturb-seq to "Assess models for genetic perturbation experiments": Assessed by Chen et al. 2026 genetic perturbation prediction, Replogle (lit-cells-networks-20261011-chen2026-source-chen2026: Table 5; Results 'Perturbation prediction separates expression reconstruction from functional response modelling')
- Relevance of SEAAD held-out annotation to "Transfer cell-type annotations to a new dataset": Assessed by Chen et al. 2026 supervised cell type annotation, SEAAD (lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation')
Models, benchmarks and other records
Models (6)
Configurations (21)
- CellPLM (continual, Chen et al. 2026)
- CellPLM (fine-tuned, Chen et al. 2026)
- CellPLM (gears-decoder, Chen et al. 2026)
- CellPLM (zero-shot, Chen et al. 2026)
- GenePT-w (fine-tuned, Chen et al. 2026)
- GenePT-w (gears-decoder, Chen et al. 2026)
- GenePT-w (kmeans, Chen et al. 2026)
- GenePT-w (zero-shot, Chen et al. 2026)
- Nicheformer (continual, Chen et al. 2026)
- Nicheformer (fine-tuned, Chen et al. 2026)
- Nicheformer (zero-shot, Chen et al. 2026)
- Novae (continual, Chen et al. 2026)
- Novae (fine-tuned, Chen et al. 2026)
- Novae (zero-shot, Chen et al. 2026)
- scELMo (fine-tuned, Chen et al. 2026)
- scELMo (gears-decoder, Chen et al. 2026)
- scELMo (zero-shot, Chen et al. 2026)
- scGPT-spatial (continual, Chen et al. 2026)
- scGPT-spatial (fine-tuned, Chen et al. 2026)
- scGPT-spatial (gears-decoder, Chen et al. 2026)
- scGPT-spatial (zero-shot, Chen et al. 2026)
Benchmarks (1)
Protocols (25)
- Chen et al. 2026 continually pretrained clustering, ATAA
- Chen et al. 2026 continually pretrained clustering, BC
- Chen et al. 2026 continually pretrained clustering, BMMC
- Chen et al. 2026 continually pretrained clustering, HS
- Chen et al. 2026 continually pretrained clustering, ME
- Chen et al. 2026 continually pretrained clustering, MH
- Chen et al. 2026 genetic perturbation prediction, Adamson
- Chen et al. 2026 genetic perturbation prediction, Norman
- Chen et al. 2026 genetic perturbation prediction, Replogle
- Chen et al. 2026 supervised cell type annotation, ATAA
- Chen et al. 2026 supervised cell type annotation, BMMC
- Chen et al. 2026 supervised cell type annotation, HS
- Chen et al. 2026 supervised cell type annotation, SEAAD
- Chen et al. 2026 zero-shot cell type and spatial domain clustering, ATAA
- Chen et al. 2026 zero-shot cell type and spatial domain clustering, BC
- Chen et al. 2026 zero-shot cell type and spatial domain clustering, BMMC
- Chen et al. 2026 zero-shot cell type and spatial domain clustering, HS
- Chen et al. 2026 zero-shot cell type and spatial domain clustering, ME
- Chen et al. 2026 zero-shot cell type and spatial domain clustering, MH
- Chen et al. 2026 zero-shot clustering with k-means, ATAA
- Chen et al. 2026 zero-shot clustering with k-means, BC
- Chen et al. 2026 zero-shot clustering with k-means, BMMC
- Chen et al. 2026 zero-shot clustering with k-means, HS
- Chen et al. 2026 zero-shot clustering with k-means, ME
- Chen et al. 2026 zero-shot clustering with k-means, MH
Dataset subsets (10)
Evaluations (93)
- CellPLM on Chen et al. 2026 continually pretrained clustering, ATAA
- CellPLM on Chen et al. 2026 continually pretrained clustering, BC
- CellPLM on Chen et al. 2026 continually pretrained clustering, BMMC
- CellPLM on Chen et al. 2026 continually pretrained clustering, HS
- CellPLM on Chen et al. 2026 continually pretrained clustering, ME
- CellPLM on Chen et al. 2026 continually pretrained clustering, MH
- CellPLM on Chen et al. 2026 genetic perturbation prediction, Adamson
- CellPLM on Chen et al. 2026 genetic perturbation prediction, Norman
- CellPLM on Chen et al. 2026 genetic perturbation prediction, Replogle
- CellPLM on Chen et al. 2026 supervised cell type annotation, ATAA
- CellPLM on Chen et al. 2026 supervised cell type annotation, BMMC
- CellPLM on Chen et al. 2026 supervised cell type annotation, HS
- CellPLM on Chen et al. 2026 supervised cell type annotation, SEAAD
- CellPLM on Chen et al. 2026 zero-shot cell type and spatial domain clustering, ATAA
- CellPLM on Chen et al. 2026 zero-shot cell type and spatial domain clustering, BC
- CellPLM on Chen et al. 2026 zero-shot cell type and spatial domain clustering, BMMC
- CellPLM on Chen et al. 2026 zero-shot cell type and spatial domain clustering, HS
- CellPLM on Chen et al. 2026 zero-shot cell type and spatial domain clustering, ME
- CellPLM on Chen et al. 2026 zero-shot cell type and spatial domain clustering, MH
- GenePT-w on Chen et al. 2026 genetic perturbation prediction, Adamson
- GenePT-w on Chen et al. 2026 genetic perturbation prediction, Norman
- GenePT-w on Chen et al. 2026 genetic perturbation prediction, Replogle
- GenePT-w on Chen et al. 2026 supervised cell type annotation, ATAA
- GenePT-w on Chen et al. 2026 supervised cell type annotation, BMMC
- GenePT-w on Chen et al. 2026 supervised cell type annotation, HS
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
24 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://arxiv.org/pdf/2607.17227v2 Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Doi 10.48550/arXiv.2607.17227 Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Evidence concerns1 values
| Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Licence CC-BY-SA-4.0 Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations3 values
| Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Media type application/pdf Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Publication status preprint Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-10-11T21:29:34Z Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Review: date 2026-10-12 Source metadata | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation No field-specific location recorded Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-23-5347695500ed · Record review: source checked
Primary source retrieved and hashed for the 2026-10-11 cells-spatial and networks-mechanistic literature pass.
Technical metadata and extraction receipts
Stable ID: lit-cells-networks-20261011-chen2026-source-chen2026
- areas
- single-cell; spatial-omics
- contexts
- research
- url
- https://arxiv.org/abs/2607.17227v2
- artifact url
- https://arxiv.org/pdf/2607.17227v2
- version
- arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
- retrieved at
- 2026-10-11T21:29:34Z
- artifact sha256
- 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da
- doi
- 10.48550/arXiv.2607.17227
- publication status
- preprint
- licence
- CC-BY-SA-4.0
- media type
- application/pdf
- limitations
- Preprint, not peer reviewed.; Single run per cell with a fixed seed (42); no spread is printed.; Spatial dataset codes BC, MH and ME are not defined in the tables; the Figure 2 legend defines them as BC/HBCA1 (human breast cancer), MH/MHPC (mouse hippocampus) and ME/MOSTA E9.5 (mouse embryo).
- evidence concerns
- source id: lit-cells-networks-20261011-chen2026-source-chen2026; message: Table 4 precision and recall equal accuracy in all eight Nicheformer and CellPLM rows, as micro (global) averaging would give, but recall differs from accuracy in every scGPT-spatial, GenePT-w, scELMo and Novae row. The Methods call precision and recall 'global' without defining the averaging, so they may be averaged differently between models and should not be compared across models.; source locator: Table 4, columns 'Accuracy', 'Precision' and 'Recall'; Methods 'Cell type annotation' (page 25); artifact sha256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da; review date: 2026-10-12
- review
- method: source-hash-verification; method note: Artifact re-downloaded from arXiv and SHA-256 matched; the archived gzip copy decompresses to the same bytes.; reviewer: claude; reviewer note: Claude review agent reviewer-cn-2, independent of the extractor lit-cells-networks; no human review claimed; date: 2026-10-12