rewire.itbenchmarks

Use caseCells and tissuesResearch only

Transfer cell-type annotations to a new dataset

Which annotation workflow can label my new dataset reliably and recognise unsupported cell populations?

In this release 0 evaluated endpoints · 0 tested configurations · No current evaluated evidence · 3 recorded evidence gaps

Evidence collection plan

Collection planned

Evidence collection is planned for this question. The plan defines a comparison to investigate; it does not establish model performance or suitability.

Comparison question

Does a method improve known-type annotation and unknown-type recognition at matched coverage over conventional methods across independent studies?

Baselines to include

  • Marker-based and nearest-reference annotation
  • Appropriate pinned Azimuth, CellTypist or scANVI workflows
  • Embeddings with a simple classifier where applicable

Outcomes to measure

  • Class-specific and hierarchy-aware errors
  • Rare-type and absent-reference performance
  • Calibration and accuracy as uncertain cells are left unassigned
  • Resource use and transfer across studies and platforms

Validation requirements

  • Hold out donors, whole studies and platforms as required by the intended query.
  • Audit duplicated cells, donors and labels across references and pretraining data.
  • Use independent expert or orthogonal label checks and preserve uncertainty in reference labels.
  • Match ontology level and coverage; keep integration and disease-state discovery separate.

Next collection task

Build an annotation-transfer evidence table with reference versions, donor/study independence, label hierarchy and coverage-aware outcomes.

Question and applicability

Choose an annotation workflow and reference, accept supported labels, and identify cells that require expert review or additional measurements.

Who this is for
  • Single-cell analysts
  • Cell atlas researchers
Research setting

Research

Research only. Annotation accuracy does not establish the validity of a clinical diagnostic classifier.

Biological setting
Research annotation of a new single-cell or single-nucleus cohort. Transfer across donors, studies, disease contexts and technologies is assessed separately.

Outside this use case

  • Atlas labels are not infallible ground truth.
  • Embedding separation and batch mixing do not establish correct cell identity.
  • Annotation, batch integration and discovery of new disease states require separate evaluations.

Inputs and expected output

Inputs you need

  • Query count data with tissue, disease, donor and assay metadata
  • A versioned reference and a defined cell-label hierarchy
  • Independent marker, protein or expert evidence where available
  • Requirements for coverage, uncertainty and resource use

Expected output

Cell-type labels at the requested ontology level, confidence estimates and an explicit unassigned group.

Evaluated evidence

Evidence is grouped by its protocol. Relevance refers to the stated endpoint and context; it is separate from clinical validation and from the review method. Limits specific to each evaluation are listed with it.

No model comparison has been collected for this question yet.

Relevant methods and studies may exist outside this collection.

View the evidence plan and next collection task

Limitations and missing evidence

These gaps apply to the question as a whole. Absence of evidence is not a zero score.

  • Comparisons need independent study and platform holdouts with reference and pretraining provenance.
  • Rare and absent-reference populations need explicit evaluation alongside common cell types.
  • Human domain review remains unassigned.

Contribute evidence or propose a correction

Sources and review

Automated source review · 2026-09-28 · Codex

Automated review of the workflow definition, cited primary-source scope and collection plan. No model evaluation or applicability mapping was added. Human domain review remains unassigned.

Release provenance and downloads

Release 2026-09-29-06401fd5b220

Use-case input digest a0dd27a5f430ec387d309fd6e8615083250873ce4cff5e882c6fb8458ef80e95

Download questions, collection plans and review metadata (JSON) · Verify release checksums

Question use-case-cell-type-annotation-transfer. Any numerical results on this page come from this release's existing evaluation records.