rewirebio.iobenchmarks
Evidence claim

Relevance of HS held-out annotation to "Transfer cell-type annotations to a new dataset"

Use case
Transfer cell-type annotations to a new dataset
Assessed by
Chen et al. 2026 supervised cell type annotation, HS
Relevance
Proxy evidence
Endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of HS

Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.

5 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-23-5347695500ed
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    lit-cells-networks-20261011-chen2026-source-chen2026
    locator
    Table 4; Results on supervised annotation; Methods 'Cell type annotation'
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
chen2026-harmonised-fm
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
2 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.constraints

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of HS
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.endpoint

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-hs
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.field

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
macro-f1
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
6 values
  • Preprint; single run with seed 42 and no spread, so small differences are not testable.
  • The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.
  • Within-dataset stratified split, not cross-dataset transfer.
  • PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.
  • Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.
  • No rejection of unseen cell types or unassigned output is evaluated.
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.limitations

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
38 values
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-hs-cellplm
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    2cea2ab61b4c9e7005220f3a460b3ba78f3aa1abc2aa5fbfbfac81153871f15b
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-hs-genept-w
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    6570d9c746042e53216c92c505b71e87b678ce3d1b50265609a51170891ea1a4
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-hs-nicheformer
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    7905a1935743d7f3b1c445f1f6a8bac2ea0b903c1cbd317cc2ec1c5dc020f364
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-hs-scelmo
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    427458f0a512c29a0624e5b9dd0ac3d6073ff77781efe269e97629fae822f18b
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-hs-scgpt-spatial
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    aa3d86875bb06a710cee3f9462382ff1c342483a3237579ca7f1834a087b90a3
  • record id
    lit-cells-networks-20261011-chen2026-protocol-annotation-hs
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    9043ff65a1f731cedffbe39d3da426cb96489e2e060c23da2100c577d94ddd13
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ee8a8dcb4f61902ec7780c6efee9ed81dacfbf9b11165bdc88d6264e8d0c05b0
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    fa6b29002216ecb0462608594e36fc8abb5e578cfc278eddc9f5ea6407c5e4e4
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6135d2e5eacb3c957f5d82ea1bf8e6baa0e8e2b6a24b6568554306d92799edd7
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2fb9997f8efd5c242b315252224b99373d0bf63461bfb27dc36b2cc90bd49e4b
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7f1010a829e42cda7d98a885b7ec320157592024e41df5257466a20ce206bf82
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    106643c2d148204adabff87c4bd30a190122d134db43c4871af42c84b24150aa
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    15f7fc9992c37c5ce60e845cb9dbc0b5ddc2df2e6c2daa9a8f3177633dbbcf92
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    21c173d5940c0c012c3b8ee4f81855581d6c6f792aba5e0f955a69cd880d0a5f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ba4221179cb8bc2d2615f96bdc0ba449eb798ecb6da8210f9d725a3c1a486d1a
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ce72e1b0e82eb95bcd1442bfb135d57ae16b1e6a61c3b5ac40ae6576c309adb0
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    63661b670a080da998d29d4e583e918fd144df8f77901b129dfdac241c8fdbef
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5a3f5522bc8350be616e2f05b9319efdd3c868c4a1d969774a157347f9b6cf2a
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9abaf346815599c01a4efb93890651400ff485db263a105ef6d642130622e6e2
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    37375f9bdc9d88e3f1dc551118aada3fb1b87aa8807958297e1d107519821cd1
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f23cf5ca3404c70f8cf64db556364d99b11ab80764dd3d70471480be6c5afacd
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f14d5f0e543f820c3690a4971c97dfaf4a89cbd5429791da87800d7c7803919c
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6329a099520a390481c8bb8b54035d9c250a38fd6ae9b6a2ba8521748ef196c5
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0407c644baded7959689d9d2ae01a445612899f459d57101d4568ce3ebace978
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a19d6227c38e6f8068ea933b5a1dab95020c1daf7337e183473078e86d5ee804
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    e6f55b3ba142f7e8a66e01ddf4870629e11b0dcdc77697b552cda6c9740ff983
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7147bf88d5586da14fa716842c1238f823a3cc7555e370ba813631009b63309f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a4fec4945f210fde6d68f042f732fc53e35094ed2672d8ddc4f447ac6b4622ae
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2034fa09363fa80194e259d86a95a269d7685cdce629838d83e14875fbe75738
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0b6cdeceee5eb4673e2302b8943909cd5cccb55ff049ea22016b92d71a0aa1a3
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6135d2e5eacb3c957f5d82ea1bf8e6baa0e8e2b6a24b6568554306d92799edd7
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    7667281e09f9be7da4573caefa17cd5ca28bdd58587a6611354669f43659a1c3
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f14d5f0e543f820c3690a4971c97dfaf4a89cbd5429791da87800d7c7803919c
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-source-chen2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5
  • record id
    use-case-cell-type-annotation-transfer
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.pins

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.rationale

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-23-5347695500ed · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-cells-networks-20261011-chen2026-annotation-hs

field
links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-hs
value
lit-cells-networks-20261011-chen2026-protocol-annotation-hs
relevance
proxy
endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of HS
rationale
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
constraints
Inspect every linked evaluation's source locator before citing a result.; Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
limitations
Preprint; single run with seed 42 and no spread, so small differences are not testable.; The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.; Within-dataset stratified split, not cross-dataset transfer.; PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.; Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.; No rejection of unseen cell types or unassigned output is evaluated.
citation locators
source id: lit-cells-networks-20261011-chen2026-source-chen2026; locator: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
source locator
lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
headline metric
macro-f1
comparison group
chen2026-harmonised-fm
comparison title
Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
stratum label
HS held-out annotation
stratum order
2
reviewed evaluations
lit-cells-networks-20261011-chen2026-eval-annotation-hs-cellplm; lit-cells-networks-20261011-chen2026-eval-annotation-hs-genept-w; lit-cells-networks-20261011-chen2026-eval-annotation-hs-nicheformer; lit-cells-networks-20261011-chen2026-eval-annotation-hs-scelmo; lit-cells-networks-20261011-chen2026-eval-annotation-hs-scgpt-spatial
reason
Add a relevance judgement for a protocol from the 2026-10-11 cells-spatial and networks-mechanistic literature pass, reviewed in docs/reviews/2026-10-12-cells-networks-batches.md.
review
method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs, output and exclusions, the relevance rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-cn-2, independent of the extractor lit-cells-networks; no human review claimed; reviewed at: 2026-10-12T08:44:00Z; note: Automated review; not human scientific review.
pins
record id: lit-cells-networks-20261011-chen2026-eval-annotation-hs-cellplm; fields: status; links; attributes.origin; attributes.comparison; sha256: 2cea2ab61b4c9e7005220f3a460b3ba78f3aa1abc2aa5fbfbfac81153871f15b; record id: lit-cells-networks-20261011-chen2026-eval-annotation-hs-genept-w; fields: status; links; attributes.origin; attributes.comparison; sha256: 6570d9c746042e53216c92c505b71e87b678ce3d1b50265609a51170891ea1a4; record id: lit-cells-networks-20261011-chen2026-eval-annotation-hs-nicheformer; fields: status; links; attributes.origin; attributes.comparison; sha256: 7905a1935743d7f3b1c445f1f6a8bac2ea0b903c1cbd317cc2ec1c5dc020f364; record id: lit-cells-networks-20261011-chen2026-eval-annotation-hs-scelmo; fields: status; links; attributes.origin; attributes.comparison; sha256: 427458f0a512c29a0624e5b9dd0ac3d6073ff77781efe269e97629fae822f18b; record id: lit-cells-networks-20261011-chen2026-eval-annotation-hs-scgpt-spatial; fields: status; links; attributes.origin; attributes.comparison; sha256: aa3d86875bb06a710cee3f9462382ff1c342483a3237579ca7f1834a087b90a3; record id: lit-cells-networks-20261011-chen2026-protocol-annotation-hs; fields: status; name; description; facets; source_ids; links; attributes; sha256: 9043ff65a1f731cedffbe39d3da426cb96489e2e060c23da2100c577d94ddd13; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ee8a8dcb4f61902ec7780c6efee9ed81dacfbf9b11165bdc88d6264e8d0c05b0; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: fa6b29002216ecb0462608594e36fc8abb5e578cfc278eddc9f5ea6407c5e4e4; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6135d2e5eacb3c957f5d82ea1bf8e6baa0e8e2b6a24b6568554306d92799edd7; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2fb9997f8efd5c242b315252224b99373d0bf63461bfb27dc36b2cc90bd49e4b; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7f1010a829e42cda7d98a885b7ec320157592024e41df5257466a20ce206bf82; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-cellplm-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 106643c2d148204adabff87c4bd30a190122d134db43c4871af42c84b24150aa; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 15f7fc9992c37c5ce60e845cb9dbc0b5ddc2df2e6c2daa9a8f3177633dbbcf92; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 21c173d5940c0c012c3b8ee4f81855581d6c6f792aba5e0f955a69cd880d0a5f; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ba4221179cb8bc2d2615f96bdc0ba449eb798ecb6da8210f9d725a3c1a486d1a; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ce72e1b0e82eb95bcd1442bfb135d57ae16b1e6a61c3b5ac40ae6576c309adb0; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-genept-w-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 63661b670a080da998d29d4e583e918fd144df8f77901b129dfdac241c8fdbef; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5a3f5522bc8350be616e2f05b9319efdd3c868c4a1d969774a157347f9b6cf2a; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9abaf346815599c01a4efb93890651400ff485db263a105ef6d642130622e6e2; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 37375f9bdc9d88e3f1dc551118aada3fb1b87aa8807958297e1d107519821cd1; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f23cf5ca3404c70f8cf64db556364d99b11ab80764dd3d70471480be6c5afacd; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f14d5f0e543f820c3690a4971c97dfaf4a89cbd5429791da87800d7c7803919c; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-nicheformer-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6329a099520a390481c8bb8b54035d9c250a38fd6ae9b6a2ba8521748ef196c5; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0407c644baded7959689d9d2ae01a445612899f459d57101d4568ce3ebace978; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a19d6227c38e6f8068ea933b5a1dab95020c1daf7337e183473078e86d5ee804; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e6f55b3ba142f7e8a66e01ddf4870629e11b0dcdc77697b552cda6c9740ff983; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7147bf88d5586da14fa716842c1238f823a3cc7555e370ba813631009b63309f; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a4fec4945f210fde6d68f042f732fc53e35094ed2672d8ddc4f447ac6b4622ae; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scelmo-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2034fa09363fa80194e259d86a95a269d7685cdce629838d83e14875fbe75738; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0b6cdeceee5eb4673e2302b8943909cd5cccb55ff049ea22016b92d71a0aa1a3; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6135d2e5eacb3c957f5d82ea1bf8e6baa0e8e2b6a24b6568554306d92799edd7; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7667281e09f9be7da4573caefa17cd5ca28bdd58587a6611354669f43659a1c3; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f14d5f0e543f820c3690a4971c97dfaf4a89cbd5429791da87800d7c7803919c; record id: lit-cells-networks-20261011-chen2026-result-annotation-hs-scgpt-spatial-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-source-chen2026; fields: status; attributes.artifact_sha256; sha256: 0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5; record id: use-case-cell-type-annotation-transfer; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Related records

Suggest a correction