Relevance of BMMC held-out annotation to "Transfer cell-type annotations to a new dataset"
- Use case
- Transfer cell-type annotations to a new dataset
- Assessed by
- Chen et al. 2026 supervised cell type annotation, BMMC
- Relevance
- Proxy evidence
- Endpoint
- Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of BMMC
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
5 reviewed evaluations
- CellPLM on Chen et al. 2026 supervised cell type annotation, BMMC
- GenePT-w on Chen et al. 2026 supervised cell type annotation, BMMC
- Nicheformer on Chen et al. 2026 supervised cell type annotation, BMMC
- scELMo on Chen et al. 2026 supervised cell type annotation, BMMC
- scGPT-spatial on Chen et al. 2026 supervised cell type annotation, BMMC
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
19 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group chen2026-harmonised-fm Context-only references | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026) Context-only references | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints2 values
| Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of BMMC Context-only references | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc Context-only references | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric macro-f1 Context-only references | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations6 values
| Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records38 values
| Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure. Context-only references | Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation' Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19) | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-23-5347695500ed · Record review: source checked
1 source record and release history
- Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation · Original source · arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Technical metadata and extraction receipts
Stable ID: use-case-mapping-lit-cells-networks-20261011-chen2026-annotation-bmmc
- field
- links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc
- value
- lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc
- relevance
- proxy
- endpoint
- Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of BMMC
- rationale
- Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
- constraints
- Inspect every linked evaluation's source locator before citing a result.; Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
- limitations
- Preprint; single run with seed 42 and no spread, so small differences are not testable.; The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.; Within-dataset stratified split, not cross-dataset transfer.; PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.; Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.; No rejection of unseen cell types or unassigned output is evaluated.
- citation locators
- source id: lit-cells-networks-20261011-chen2026-source-chen2026; locator: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
- source locator
- lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
- headline metric
- macro-f1
- comparison group
- chen2026-harmonised-fm
- comparison title
- Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
- stratum label
- BMMC held-out annotation
- stratum order
- 3
- reviewed evaluations
- lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-cellplm; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-genept-w; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-nicheformer; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scelmo; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scgpt-spatial
- reason
- Add a relevance judgement for a protocol from the 2026-10-11 cells-spatial and networks-mechanistic literature pass, reviewed in docs/reviews/2026-10-12-cells-networks-batches.md.
- review
- method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs, output and exclusions, the relevance rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-cn-2, independent of the extractor lit-cells-networks; no human review claimed; reviewed at: 2026-10-12T08:44:00Z; note: Automated review; not human scientific review.
- pins
- record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-cellplm; fields: status; links; attributes.origin; attributes.comparison; sha256: 5673376ce298b37dbdebcf3dc88b2777744b5d334742489b37d28c0f6dbf1599; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-genept-w; fields: status; links; attributes.origin; attributes.comparison; sha256: 05ef80fd43c60ef41ea4ede74339ce64c4fd8c3fb981295c73136ef6b5ff18c3; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-nicheformer; fields: status; links; attributes.origin; attributes.comparison; sha256: b896524e4ecfba89940388d377996e5e493728a8b8af32343fbf26c906fff3e1; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scelmo; fields: status; links; attributes.origin; attributes.comparison; sha256: a6c02755e2c10792d7e54ab9def9d3e1622801a5ec17c34752169a02d6a78a22; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scgpt-spatial; fields: status; links; attributes.origin; attributes.comparison; sha256: 0b9b07d2985cfbd6c2267ed8daad7413f1b09af85d2e8f819e357de18d2581e8; record id: lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc; fields: status; name; description; facets; source_ids; links; attributes; sha256: f9105c795b1858ed1c996c389d4b9d754f21f739a43df1b94475628bfd1a645f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0407c644baded7959689d9d2ae01a445612899f459d57101d4568ce3ebace978; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: dcc54dbf70e9defb7e7cd2524d773468e6c788c2f87584bf318ddc1a95d2b71c; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cfe0aa7a189a8642003de367cd16fa3e40c69ee37109f591e33c555c2b0384fb; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a805ebc965e51761c569a2e87a1f32562292518c16725356b992e67fdbe1e760; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5f40ea2990b91eca7c85eb36ca27cc94fedf094c4859184b576e83fd48c90395; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f7a53f605ce3631832bebb0e04d33769692b2c4730661f23e5fce023ebbf90d4; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f3a339288f9d8189835346aff6273cdbb5f59df90c5e281009e906ff24da6b08; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: bfa29e1a1cebba81ce9b4a6fbddbccf3dee7c45ff1d0d43cc0746b573572c097; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f9a009022e4e75bdc897a29e83317f5e067b731aa86766204b0ef163d4ce9d6f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 29058de0cc9d3c07f8f72d2b920a54f4b0d5accbce265ee300c4e323e05eba94; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6b564168511cd8ff4ddba29e07554415ac02f4380ac0e18385ac2de1f6bf9772; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 42097e0e96b6c871bed112b7d3333cb88d9e99967dae6c2375bdc28731056f8f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 15f7fc9992c37c5ce60e845cb9dbc0b5ddc2df2e6c2daa9a8f3177633dbbcf92; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 78bea2fba63a28916a10a0ef4b73d5a29773680f908640ffb3913630dc2acd37; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eefdde13b37b2a3c4dc78e759534caf0022c692dd201ce7845ccec745abbfc45; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 58d6f807a26b57dedadbf0ec6bde8f494688bba2e4e1cfc939c1183409da993d; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 32fc1ebcd9f5d964512a561b1d35db1b1e558eff6063d9e20fac3ea9edc5faa7; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d1eae90efd0c5f4468994842c4afac14d920f3b5437e3f906a8cfccc7b96ab20; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c9a9d51d2c3d4bbdd82793f6d4bbdc94edbcabb7db09aaf61e70013efb7bbf79; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 032c65655d77cdedddd8bbd913a6a75a633d398e4a7e4eb6872e7bbceeabc3b3; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 75c9afeb2d6429addafd3198dda5191f7ed1d62be4bdb1594f716b9d56a597ce; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a075d756ab88c6ab00600d3025d2720db9ea39316c57b93b600a8ebaa467e3e1; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 42097e0e96b6c871bed112b7d3333cb88d9e99967dae6c2375bdc28731056f8f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e1dc0befddaa50696f11771006d5873c209bd0ff7916c308e8512e21b0905499; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c55986af7ee5440144cdd4127b37c4afec4704614a35ef08ddd3005adf047420; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ba4221179cb8bc2d2615f96bdc0ba449eb798ecb6da8210f9d725a3c1a486d1a; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: dd5d7f219ea646e7c7975d9d8b5bf09212eae4b932dad8728ca714c26cfab8e1; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-source-chen2026; fields: status; attributes.artifact_sha256; sha256: 0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5; record id: use-case-cell-type-annotation-transfer; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9