rewirebio.iobenchmarks
Evidence claim

Relevance of ATAA held-out annotation to "Transfer cell-type annotations to a new dataset"

Use case
Transfer cell-type annotations to a new dataset
Assessed by
Chen et al. 2026 supervised cell type annotation, ATAA
Relevance
Proxy evidence
Endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of ATAA

Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.

5 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-23-5347695500ed
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    lit-cells-networks-20261011-chen2026-source-chen2026
    locator
    Table 4; Results on supervised annotation; Methods 'Cell type annotation'
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
chen2026-harmonised-fm
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
2 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.constraints

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of ATAA
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.endpoint

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-ataa
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.field

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
macro-f1
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
6 values
  • Preprint; single run with seed 42 and no spread, so small differences are not testable.
  • The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.
  • Within-dataset stratified split, not cross-dataset transfer.
  • PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.
  • Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.
  • No rejection of unseen cell types or unassigned output is evaluated.
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.limitations

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
38 values
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-ataa-cellplm
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    4b34f96ea7e7dfb34ae5634944988ff0b8395ca2095c026ba03bd1ceed4e0d64
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-ataa-genept-w
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    ea0b2819c2b4c8f99fc4912cefbff5349b245b44f798624fe91cf21348e53a89
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-ataa-nicheformer
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    ba6046d1c293fece0657af77587e19a797314a9df0873d83e302c5e9a4bd5ee2
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-ataa-scelmo
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    1b88b2fafd0d7f23e14adc1d266994bff644b780c0104b74b2dfed76e3169ace
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-ataa-scgpt-spatial
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    b723f4bdb2c553f0246d28d7b1b67c8b99cb57deaab8eef673ce93ccb5af4b23
  • record id
    lit-cells-networks-20261011-chen2026-protocol-annotation-ataa
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    a5ea95ecf162cf2f00661b8efd8bc43f3f811d38f8265f632285e94d797f02bd
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8f0ae506f565a83d0e86e0bc3b97835ccbf71a68bb365dda7aaabaa6cd2d512e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ae1d1d71e474cba34cdb923de9d05048da9be93a13143d5be5c9ebec03cf08cc
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9414af44e9c0d0c3c227f122711222b87234eadc68ada3632876deca804c018b
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    00907c66dca63f81089c38a6874df6068c458542f4d98c615b13980cab2a4316
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ee7c366f05d2396a3b2120bf8d5bd243a78315dcdeb973fcee37c435b33f833d
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f8fdd600a08042fd8158ecda74035929576c04153a1ef4a6dfa6f61ab53bf59a
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    4e9932f6503ee218730aaefebabc13782a861b784ef585ff389aba26c025cb63
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8cae0a3098d481a7c57dd7f9754db378b1dd15e0f0fc520ed619571769f60d72
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5fed8d144ff87ec3dede63fcd592d76040e62f758bdd14c3d8cb7925480c4028
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5b74e8570673f43c1efa2d12c5f7584f5f90979863f30c6c4f1878140652518d
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d9e527dda40f96508e6022f2573624f951f1a4fdcc4eb2578a1e2f82c5e1d49d
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8f0ae506f565a83d0e86e0bc3b97835ccbf71a68bb365dda7aaabaa6cd2d512e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7a358e5833fc3bf3b9bcd857e96502364a8a57af4be898ee921184c8cad6f025
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9414af44e9c0d0c3c227f122711222b87234eadc68ada3632876deca804c018b
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    00907c66dca63f81089c38a6874df6068c458542f4d98c615b13980cab2a4316
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ee7c366f05d2396a3b2120bf8d5bd243a78315dcdeb973fcee37c435b33f833d
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6c3efb411e95bddf84278cbb432c6a1430e96e794cbaa839a74f90c4b27bd0c8
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9e761e2f0b2b8ed4a77a36c87cab9f79dd4cab5c72ff4d7b2fe3addf5cbba63f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0c6a4c3361ab780e303f50d8ef52388ca017dd7ef0535668fd8aeaa892c0d18e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    e56ec6d36c32dbde0930509842dbfabac96e099c2494f60832b1c83e6d00bf7b
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5f40ea2990b91eca7c85eb36ca27cc94fedf094c4859184b576e83fd48c90395
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    8f0ae506f565a83d0e86e0bc3b97835ccbf71a68bb365dda7aaabaa6cd2d512e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7a358e5833fc3bf3b9bcd857e96502364a8a57af4be898ee921184c8cad6f025
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9414af44e9c0d0c3c227f122711222b87234eadc68ada3632876deca804c018b
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b3129bb9f0be1db2bbdd6c39b8ccc79840c56f38ef2faa4f8e62393e8e51cdb6
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    53ee1dff11e6096fefd865aceb6d413c7c069bed248c990798be7e67f43e24fe
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-source-chen2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5
  • record id
    use-case-cell-type-annotation-transfer
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.pins

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.rationale

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-23-5347695500ed · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-cells-networks-20261011-chen2026-annotation-ataa

field
links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-ataa
value
lit-cells-networks-20261011-chen2026-protocol-annotation-ataa
relevance
proxy
endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of ATAA
rationale
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
constraints
Inspect every linked evaluation's source locator before citing a result.; Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
limitations
Preprint; single run with seed 42 and no spread, so small differences are not testable.; The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.; Within-dataset stratified split, not cross-dataset transfer.; PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.; Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.; No rejection of unseen cell types or unassigned output is evaluated.
citation locators
source id: lit-cells-networks-20261011-chen2026-source-chen2026; locator: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
source locator
lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
headline metric
macro-f1
comparison group
chen2026-harmonised-fm
comparison title
Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
stratum label
ATAA held-out annotation
stratum order
1
reviewed evaluations
lit-cells-networks-20261011-chen2026-eval-annotation-ataa-cellplm; lit-cells-networks-20261011-chen2026-eval-annotation-ataa-genept-w; lit-cells-networks-20261011-chen2026-eval-annotation-ataa-nicheformer; lit-cells-networks-20261011-chen2026-eval-annotation-ataa-scelmo; lit-cells-networks-20261011-chen2026-eval-annotation-ataa-scgpt-spatial
reason
Add a relevance judgement for a protocol from the 2026-10-11 cells-spatial and networks-mechanistic literature pass, reviewed in docs/reviews/2026-10-12-cells-networks-batches.md.
review
method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs, output and exclusions, the relevance rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-cn-2, independent of the extractor lit-cells-networks; no human review claimed; reviewed at: 2026-10-12T08:44:00Z; note: Automated review; not human scientific review.
pins
record id: lit-cells-networks-20261011-chen2026-eval-annotation-ataa-cellplm; fields: status; links; attributes.origin; attributes.comparison; sha256: 4b34f96ea7e7dfb34ae5634944988ff0b8395ca2095c026ba03bd1ceed4e0d64; record id: lit-cells-networks-20261011-chen2026-eval-annotation-ataa-genept-w; fields: status; links; attributes.origin; attributes.comparison; sha256: ea0b2819c2b4c8f99fc4912cefbff5349b245b44f798624fe91cf21348e53a89; record id: lit-cells-networks-20261011-chen2026-eval-annotation-ataa-nicheformer; fields: status; links; attributes.origin; attributes.comparison; sha256: ba6046d1c293fece0657af77587e19a797314a9df0873d83e302c5e9a4bd5ee2; record id: lit-cells-networks-20261011-chen2026-eval-annotation-ataa-scelmo; fields: status; links; attributes.origin; attributes.comparison; sha256: 1b88b2fafd0d7f23e14adc1d266994bff644b780c0104b74b2dfed76e3169ace; record id: lit-cells-networks-20261011-chen2026-eval-annotation-ataa-scgpt-spatial; fields: status; links; attributes.origin; attributes.comparison; sha256: b723f4bdb2c553f0246d28d7b1b67c8b99cb57deaab8eef673ce93ccb5af4b23; record id: lit-cells-networks-20261011-chen2026-protocol-annotation-ataa; fields: status; name; description; facets; source_ids; links; attributes; sha256: a5ea95ecf162cf2f00661b8efd8bc43f3f811d38f8265f632285e94d797f02bd; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8f0ae506f565a83d0e86e0bc3b97835ccbf71a68bb365dda7aaabaa6cd2d512e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ae1d1d71e474cba34cdb923de9d05048da9be93a13143d5be5c9ebec03cf08cc; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9414af44e9c0d0c3c227f122711222b87234eadc68ada3632876deca804c018b; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 00907c66dca63f81089c38a6874df6068c458542f4d98c615b13980cab2a4316; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ee7c366f05d2396a3b2120bf8d5bd243a78315dcdeb973fcee37c435b33f833d; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-cellplm-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f8fdd600a08042fd8158ecda74035929576c04153a1ef4a6dfa6f61ab53bf59a; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4e9932f6503ee218730aaefebabc13782a861b784ef585ff389aba26c025cb63; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8cae0a3098d481a7c57dd7f9754db378b1dd15e0f0fc520ed619571769f60d72; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5fed8d144ff87ec3dede63fcd592d76040e62f758bdd14c3d8cb7925480c4028; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5b74e8570673f43c1efa2d12c5f7584f5f90979863f30c6c4f1878140652518d; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-genept-w-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d9e527dda40f96508e6022f2573624f951f1a4fdcc4eb2578a1e2f82c5e1d49d; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8f0ae506f565a83d0e86e0bc3b97835ccbf71a68bb365dda7aaabaa6cd2d512e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7a358e5833fc3bf3b9bcd857e96502364a8a57af4be898ee921184c8cad6f025; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9414af44e9c0d0c3c227f122711222b87234eadc68ada3632876deca804c018b; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 00907c66dca63f81089c38a6874df6068c458542f4d98c615b13980cab2a4316; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ee7c366f05d2396a3b2120bf8d5bd243a78315dcdeb973fcee37c435b33f833d; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-nicheformer-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6c3efb411e95bddf84278cbb432c6a1430e96e794cbaa839a74f90c4b27bd0c8; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9e761e2f0b2b8ed4a77a36c87cab9f79dd4cab5c72ff4d7b2fe3addf5cbba63f; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0c6a4c3361ab780e303f50d8ef52388ca017dd7ef0535668fd8aeaa892c0d18e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e56ec6d36c32dbde0930509842dbfabac96e099c2494f60832b1c83e6d00bf7b; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5f40ea2990b91eca7c85eb36ca27cc94fedf094c4859184b576e83fd48c90395; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scelmo-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 8f0ae506f565a83d0e86e0bc3b97835ccbf71a68bb365dda7aaabaa6cd2d512e; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7a358e5833fc3bf3b9bcd857e96502364a8a57af4be898ee921184c8cad6f025; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9414af44e9c0d0c3c227f122711222b87234eadc68ada3632876deca804c018b; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b3129bb9f0be1db2bbdd6c39b8ccc79840c56f38ef2faa4f8e62393e8e51cdb6; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 53ee1dff11e6096fefd865aceb6d413c7c069bed248c990798be7e67f43e24fe; record id: lit-cells-networks-20261011-chen2026-result-annotation-ataa-scgpt-spatial-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-source-chen2026; fields: status; attributes.artifact_sha256; sha256: 0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5; record id: use-case-cell-type-annotation-transfer; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Related records

Suggest a correction