rewirebio.iobenchmarks
Evidence claim

Relevance of BMMC held-out annotation to "Transfer cell-type annotations to a new dataset"

Use case
Transfer cell-type annotations to a new dataset
Assessed by
Chen et al. 2026 supervised cell type annotation, BMMC
Relevance
Proxy evidence
Endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of BMMC

Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.

5 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-23-5347695500ed
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    lit-cells-networks-20261011-chen2026-source-chen2026
    locator
    Table 4; Results on supervised annotation; Methods 'Cell type annotation'
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
chen2026-harmonised-fm
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
2 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.constraints

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of BMMC
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.endpoint

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.field

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
macro-f1
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
6 values
  • Preprint; single run with seed 42 and no spread, so small differences are not testable.
  • The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.
  • Within-dataset stratified split, not cross-dataset transfer.
  • PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.
  • Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.
  • No rejection of unseen cell types or unassigned output is evaluated.
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.limitations

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
38 values
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-cellplm
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    5673376ce298b37dbdebcf3dc88b2777744b5d334742489b37d28c0f6dbf1599
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-genept-w
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    05ef80fd43c60ef41ea4ede74339ce64c4fd8c3fb981295c73136ef6b5ff18c3
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-nicheformer
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    b896524e4ecfba89940388d377996e5e493728a8b8af32343fbf26c906fff3e1
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scelmo
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    a6c02755e2c10792d7e54ab9def9d3e1622801a5ec17c34752169a02d6a78a22
  • record id
    lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scgpt-spatial
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    0b9b07d2985cfbd6c2267ed8daad7413f1b09af85d2e8f819e357de18d2581e8
  • record id
    lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    f9105c795b1858ed1c996c389d4b9d754f21f739a43df1b94475628bfd1a645f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0407c644baded7959689d9d2ae01a445612899f459d57101d4568ce3ebace978
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    dcc54dbf70e9defb7e7cd2524d773468e6c788c2f87584bf318ddc1a95d2b71c
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    cfe0aa7a189a8642003de367cd16fa3e40c69ee37109f591e33c555c2b0384fb
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a805ebc965e51761c569a2e87a1f32562292518c16725356b992e67fdbe1e760
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5f40ea2990b91eca7c85eb36ca27cc94fedf094c4859184b576e83fd48c90395
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f7a53f605ce3631832bebb0e04d33769692b2c4730661f23e5fce023ebbf90d4
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f3a339288f9d8189835346aff6273cdbb5f59df90c5e281009e906ff24da6b08
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    bfa29e1a1cebba81ce9b4a6fbddbccf3dee7c45ff1d0d43cc0746b573572c097
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f9a009022e4e75bdc897a29e83317f5e067b731aa86766204b0ef163d4ce9d6f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    29058de0cc9d3c07f8f72d2b920a54f4b0d5accbce265ee300c4e323e05eba94
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    6b564168511cd8ff4ddba29e07554415ac02f4380ac0e18385ac2de1f6bf9772
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    42097e0e96b6c871bed112b7d3333cb88d9e99967dae6c2375bdc28731056f8f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    15f7fc9992c37c5ce60e845cb9dbc0b5ddc2df2e6c2daa9a8f3177633dbbcf92
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    78bea2fba63a28916a10a0ef4b73d5a29773680f908640ffb3913630dc2acd37
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    eefdde13b37b2a3c4dc78e759534caf0022c692dd201ce7845ccec745abbfc45
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    58d6f807a26b57dedadbf0ec6bde8f494688bba2e4e1cfc939c1183409da993d
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    32fc1ebcd9f5d964512a561b1d35db1b1e558eff6063d9e20fac3ea9edc5faa7
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d1eae90efd0c5f4468994842c4afac14d920f3b5437e3f906a8cfccc7b96ab20
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    c9a9d51d2c3d4bbdd82793f6d4bbdc94edbcabb7db09aaf61e70013efb7bbf79
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    032c65655d77cdedddd8bbd913a6a75a633d398e4a7e4eb6872e7bbceeabc3b3
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    75c9afeb2d6429addafd3198dda5191f7ed1d62be4bdb1594f716b9d56a597ce
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a075d756ab88c6ab00600d3025d2720db9ea39316c57b93b600a8ebaa467e3e1
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-accuracy
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    42097e0e96b6c871bed112b7d3333cb88d9e99967dae6c2375bdc28731056f8f
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-macro-f1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    e1dc0befddaa50696f11771006d5873c209bd0ff7916c308e8512e21b0905499
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-pr-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    c55986af7ee5440144cdd4127b37c4afec4704614a35ef08ddd3005adf047420
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-precision
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    ba4221179cb8bc2d2615f96bdc0ba449eb798ecb6da8210f9d725a3c1a486d1a
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-recall
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    dd5d7f219ea646e7c7975d9d8b5bf09212eae4b932dad8728ca714c26cfab8e1
  • record id
    lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-roc-auc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e
  • record id
    lit-cells-networks-20261011-chen2026-source-chen2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5
  • record id
    use-case-cell-type-annotation-transfer
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.pins

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
Context-only references
Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation

Original source ↗

lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'

Version: arXiv 2607.17227v2, updated 2026-09-17 (v1 posted 2026-07-19)
Retrieved: 2026-10-11T21:29:34Z

not individually reviewed

No individual claim review recorded

Source has a recorded evidence concern. Consult its source page before using the claim.

Audit details

Field: attributes.rationale

Source artifact SHA-256: 8137338c6146f570a6546f30707d4244baca9dab848b32c51539665a5ba539da

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-23-5347695500ed · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-lit-cells-networks-20261011-chen2026-annotation-bmmc

field
links:assessed_by:lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc
value
lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc
relevance
proxy
endpoint
Accuracy, precision, recall, macro-F1, PR-AUC and ROC-AUC of supervised cell type annotation on a stratified held-out split of BMMC
rationale
Scores foundation models fine-tuned on reference labels and tested on held-out cells of the same dataset. Proxy: the use case transfers labels to a new dataset, while this split keeps train and test cells from the same study and batch structure.
constraints
Inspect every linked evaluation's source locator before citing a result.; Prefer macro-F1 and PR-AUC; accuracy and ROC-AUC saturate on ATAA (Results).
limitations
Preprint; single run with seed 42 and no spread, so small differences are not testable.; The authors did not develop any of the six models (independent comparison), but no conventional baseline is in the tables.; Within-dataset stratified split, not cross-dataset transfer.; PR-AUC and ROC-AUC are macro-averaged (Methods); precision and recall are called global without a definition and appear to be averaged differently between models (source evidence concern), so do not compare them across models.; Reference labels are the datasets' curated marker-gene and expert annotations, not independent ground truth.; No rejection of unseen cell types or unassigned output is evaluated.
citation locators
source id: lit-cells-networks-20261011-chen2026-source-chen2026; locator: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
source locator
lit-cells-networks-20261011-chen2026-source-chen2026: Table 4; Results on supervised annotation; Methods 'Cell type annotation'
headline metric
macro-f1
comparison group
chen2026-harmonised-fm
comparison title
Harmonised single-cell and spatial foundation model benchmark (Chen et al. 2026)
stratum label
BMMC held-out annotation
stratum order
3
reviewed evaluations
lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-cellplm; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-genept-w; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-nicheformer; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scelmo; lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scgpt-spatial
reason
Add a relevance judgement for a protocol from the 2026-10-11 cells-spatial and networks-mechanistic literature pass, reviewed in docs/reviews/2026-10-12-cells-networks-batches.md.
review
method: ai-assisted-source-review; scope-review; method note: Reviewer checked the endpoint against the use case's inputs, output and exclusions, the relevance rating and the limitations; every evaluation on the protocol had its values checked in the batch review. Corrections listed in review.md were applied by the extractor.; reviewer: claude; reviewer note: Claude review agent reviewer-cn-2, independent of the extractor lit-cells-networks; no human review claimed; reviewed at: 2026-10-12T08:44:00Z; note: Automated review; not human scientific review.
pins
record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-cellplm; fields: status; links; attributes.origin; attributes.comparison; sha256: 5673376ce298b37dbdebcf3dc88b2777744b5d334742489b37d28c0f6dbf1599; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-genept-w; fields: status; links; attributes.origin; attributes.comparison; sha256: 05ef80fd43c60ef41ea4ede74339ce64c4fd8c3fb981295c73136ef6b5ff18c3; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-nicheformer; fields: status; links; attributes.origin; attributes.comparison; sha256: b896524e4ecfba89940388d377996e5e493728a8b8af32343fbf26c906fff3e1; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scelmo; fields: status; links; attributes.origin; attributes.comparison; sha256: a6c02755e2c10792d7e54ab9def9d3e1622801a5ec17c34752169a02d6a78a22; record id: lit-cells-networks-20261011-chen2026-eval-annotation-bmmc-scgpt-spatial; fields: status; links; attributes.origin; attributes.comparison; sha256: 0b9b07d2985cfbd6c2267ed8daad7413f1b09af85d2e8f819e357de18d2581e8; record id: lit-cells-networks-20261011-chen2026-protocol-annotation-bmmc; fields: status; name; description; facets; source_ids; links; attributes; sha256: f9105c795b1858ed1c996c389d4b9d754f21f739a43df1b94475628bfd1a645f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0407c644baded7959689d9d2ae01a445612899f459d57101d4568ce3ebace978; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: dcc54dbf70e9defb7e7cd2524d773468e6c788c2f87584bf318ddc1a95d2b71c; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: cfe0aa7a189a8642003de367cd16fa3e40c69ee37109f591e33c555c2b0384fb; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a805ebc965e51761c569a2e87a1f32562292518c16725356b992e67fdbe1e760; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5f40ea2990b91eca7c85eb36ca27cc94fedf094c4859184b576e83fd48c90395; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-cellplm-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f7a53f605ce3631832bebb0e04d33769692b2c4730661f23e5fce023ebbf90d4; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f3a339288f9d8189835346aff6273cdbb5f59df90c5e281009e906ff24da6b08; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: bfa29e1a1cebba81ce9b4a6fbddbccf3dee7c45ff1d0d43cc0746b573572c097; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f9a009022e4e75bdc897a29e83317f5e067b731aa86766204b0ef163d4ce9d6f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 29058de0cc9d3c07f8f72d2b920a54f4b0d5accbce265ee300c4e323e05eba94; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-genept-w-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6b564168511cd8ff4ddba29e07554415ac02f4380ac0e18385ac2de1f6bf9772; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 42097e0e96b6c871bed112b7d3333cb88d9e99967dae6c2375bdc28731056f8f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 15f7fc9992c37c5ce60e845cb9dbc0b5ddc2df2e6c2daa9a8f3177633dbbcf92; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 78bea2fba63a28916a10a0ef4b73d5a29773680f908640ffb3913630dc2acd37; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: eefdde13b37b2a3c4dc78e759534caf0022c692dd201ce7845ccec745abbfc45; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 58d6f807a26b57dedadbf0ec6bde8f494688bba2e4e1cfc939c1183409da993d; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-nicheformer-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 32fc1ebcd9f5d964512a561b1d35db1b1e558eff6063d9e20fac3ea9edc5faa7; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d1eae90efd0c5f4468994842c4afac14d920f3b5437e3f906a8cfccc7b96ab20; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c9a9d51d2c3d4bbdd82793f6d4bbdc94edbcabb7db09aaf61e70013efb7bbf79; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 032c65655d77cdedddd8bbd913a6a75a633d398e4a7e4eb6872e7bbceeabc3b3; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 75c9afeb2d6429addafd3198dda5191f7ed1d62be4bdb1594f716b9d56a597ce; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scelmo-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a075d756ab88c6ab00600d3025d2720db9ea39316c57b93b600a8ebaa467e3e1; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-accuracy; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 42097e0e96b6c871bed112b7d3333cb88d9e99967dae6c2375bdc28731056f8f; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-macro-f1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e1dc0befddaa50696f11771006d5873c209bd0ff7916c308e8512e21b0905499; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-pr-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c55986af7ee5440144cdd4127b37c4afec4704614a35ef08ddd3005adf047420; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-precision; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ba4221179cb8bc2d2615f96bdc0ba449eb798ecb6da8210f9d725a3c1a486d1a; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-recall; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: dd5d7f219ea646e7c7975d9d8b5bf09212eae4b932dad8728ca714c26cfab8e1; record id: lit-cells-networks-20261011-chen2026-result-annotation-bmmc-scgpt-spatial-roc-auc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9b999931a75aafb766fe8efc8145bb499ffdaa2b17fac4206f62df642c37341e; record id: lit-cells-networks-20261011-chen2026-source-chen2026; fields: status; attributes.artifact_sha256; sha256: 0b42f541d5ca376dc4e14f043f2eef02b5f307d7820685e0a1e3ef4a31def2b5; record id: use-case-cell-type-annotation-transfer; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Related records

Suggest a correction