rewirebio.iobenchmarks
Evidence claim

Relevance of scTab deep-ensemble uncertainty ROC-AUC: known-type vs. absent-type (Group 1 vs. Group 3) to "Transfer cell-type annotations to a new dataset"

Use case
Transfer cell-type annotations to a new dataset
Assessed by
scTab deep-ensemble uncertainty ROC-AUC: known-type vs. absent-type (Group 1 vs. Group 3)
Relevance
Proxy evidence
Endpoint
Deep-ensemble (5-model) uncertainty-score ROC-AUC distinguishing correctly predicted known cell types from cell types entirely absent from training, within one donor-held-out split of scTab's pooled 249-dataset CELLxGENE corpus.

Directly on the use case's unknown-type-recognition endpoint, but single-corpus and donor-level only; no cross-study or cross-platform transfer of this capability is established by this source.

1 reviewed evaluation

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

15 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    ucc-research-source-sctab-paper
    locator
    Methods, "Uncertainty quantification for scTab model"
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
2 values
  • Donor-level holdout within one pooled 249-dataset corpus (10x-related assays only; rare types excluded); not a whole-study or whole-platform holdout. Dataset/study overlap between training and test splits is not directly confirmed in the retrieved text.
  • Single ROC-AUC value; no rejection-at-coverage curve or demonstrated calibration.
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Deep-ensemble (5-model) uncertainty-score ROC-AUC distinguishing correctly predicted known cell types from cell types entirely absent from training, within one donor-held-out split of scTab's pooled 249-dataset CELLxGENE corpus.
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:ucc-research-protocol-sctab-uncertainty-absent
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
4 values
  • This is unknown/absent-type detection only; it must not be combined with or treated as evidence for scTab's separate known-type error-detection ROC-AUC (0.891, see use-case-mapping-20261006-349-6a9d63555f43).
  • Exact number of cell types and cells comprising the absent-type comparison group is not stated in the retrieved main text; held here as an explicit missing denominator, not invented.
  • Reference/training labels are author annotations, not independent ground truth.
  • Automated source review only; independent human scientific review remains outstanding.
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
5 values
  • record id
    ucc-research-eval-sctab-uncertainty-absent
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    44c2b8c932b96ff5395e772de09b2dac18d6ca9c632ec23cd2ed5e8717725664
  • record id
    ucc-research-protocol-sctab-uncertainty-absent
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    342b65ed776b1ad1e5bc08afa5e499ee3b33bb0522f0f1451d1056e78361d18e
  • record id
    ucc-research-result-sctab-uncertainty-absent-rocauc
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    38b03250a64279dc2f567fdac6c10c04a8efef62aa7a4d64f6245982ff819c30
  • record id
    ucc-research-source-sctab-paper
    fields
    • status
    • attributes.artifact_sha256
    sha256
    12a4f59860912c8db02b1d56bf33636bdaccf99273cfcca9e9375ace19e54cc4
  • record id
    use-case-cell-type-annotation-transfer
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Directly on the use case's unknown-type-recognition endpoint, but single-corpus and donor-level only; no cross-study or cross-platform transfer of this capability is established by this source.
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Reason
Add scTab's own unknown/absent-type detection ROC-AUC (deep-ensemble uncertainty score, correct known-type vs. absent-type), independently source-checked in the additive 2026-10-06 intake (issue #349). Kept as a separate mapping from the existing Table 1a known-type macro-F1 mapping and from scTab's own known-type error-detection ROC-AUC, which must not be conflated with unknown-type detection.
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.reason

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relevance
proxy
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.relevance

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Revision
1
Context-only references
scTab: Scaling cross-tissue single-cell annotation models

Original source ↗

ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"

Version: 10.1038/s41467-024-51059-5; published article XML retrieved 2026-09-30
Retrieved: 2026-09-30T21:23:28Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.revision

Source artifact SHA-256: e61b11cd873f3d13e891e9c33bac0847070826200f8fbe0b3969d994f1c52290

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-20261006-349-2cfbe5c7674a

field
links:assessed_by:ucc-research-protocol-sctab-uncertainty-absent
value
ucc-research-protocol-sctab-uncertainty-absent
relevance
proxy
endpoint
Deep-ensemble (5-model) uncertainty-score ROC-AUC distinguishing correctly predicted known cell types from cell types entirely absent from training, within one donor-held-out split of scTab's pooled 249-dataset CELLxGENE corpus.
rationale
Directly on the use case's unknown-type-recognition endpoint, but single-corpus and donor-level only; no cross-study or cross-platform transfer of this capability is established by this source.
constraints
Donor-level holdout within one pooled 249-dataset corpus (10x-related assays only; rare types excluded); not a whole-study or whole-platform holdout. Dataset/study overlap between training and test splits is not directly confirmed in the retrieved text.; Single ROC-AUC value; no rejection-at-coverage curve or demonstrated calibration.
limitations
This is unknown/absent-type detection only; it must not be combined with or treated as evidence for scTab's separate known-type error-detection ROC-AUC (0.891, see use-case-mapping-20261006-349-6a9d63555f43).; Exact number of cell types and cells comprising the absent-type comparison group is not stated in the retrieved main text; held here as an explicit missing denominator, not invented.; Reference/training labels are author annotations, not independent ground truth.; Automated source review only; independent human scientific review remains outstanding.
revision
1
reason
Add scTab's own unknown/absent-type detection ROC-AUC (deep-ensemble uncertainty score, correct known-type vs. absent-type), independently source-checked in the additive 2026-10-06 intake (issue #349). Kept as a separate mapping from the existing Table 1a known-type macro-F1 mapping and from scTab's own known-type error-detection ROC-AUC, which must not be conflated with unknown-type detection.
citation locators
source id: ucc-research-source-sctab-paper; locator: Methods, "Uncertainty quantification for scTab model"
source locator
ucc-research-source-sctab-paper: Methods, "Uncertainty quantification for scTab model"
review
method: automated-source-review; reviewer: claude-sonnet; reviewer note: Claude Sonnet cell-type-annotation-transfer evidence-research worker; reviewed at: 2026-10-06T22:39:40Z; note: Source-backed primary-text transcription of the 2026-10-06 cell-type-annotation-transfer research dossier (docs/omics/evidence-research/cell-type-annotation-transfer-2026-10-06.md), corrected after an independent source check; bound by data/omics/use-case-coverage-20261006/review.json. Revised same day after a second independent review corrected the uncertainty-averaging order, removed an unsupported 'temperature-scaled' claim, replaced n_runs with ensemble_size, and removed an unsupported 'data augmentation enabled' assertion; evidence_sha256 refreshed accordingly. No ROC-AUC value or endpoint scoping changed. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.
reviewed evaluations
ucc-research-eval-sctab-uncertainty-absent
pins
record id: ucc-research-eval-sctab-uncertainty-absent; fields: status; links; attributes.origin; attributes.comparison; sha256: 44c2b8c932b96ff5395e772de09b2dac18d6ca9c632ec23cd2ed5e8717725664; record id: ucc-research-protocol-sctab-uncertainty-absent; fields: status; name; description; facets; source_ids; links; attributes; sha256: 342b65ed776b1ad1e5bc08afa5e499ee3b33bb0522f0f1451d1056e78361d18e; record id: ucc-research-result-sctab-uncertainty-absent-rocauc; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 38b03250a64279dc2f567fdac6c10c04a8efef62aa7a4d64f6245982ff819c30; record id: ucc-research-source-sctab-paper; fields: status; attributes.artifact_sha256; sha256: 12a4f59860912c8db02b1d56bf33636bdaccf99273cfcca9e9375ace19e54cc4; record id: use-case-cell-type-annotation-transfer; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Related records

Suggest a correction