Task

Donor-held-out reference mapping

Donor-held-out reference mapping asks whether a reference learned from some donors remains useful for cells from a new donor.

Sources (2)

scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

No reviewed evaluations are linked here in this release. See the sources and separately identified configurations below.

0 evaluations · 0 results

Overview

Datasets

No single dataset is fixed by this guide. Select a linked protocol and its versioned data release.

inapplicable

Metrics

Choose label-transfer metrics for annotations and biological-conservation metrics for representations. The exact metric, averaging and unknown-cell handling belong to the concrete protocol.

Allowed inputs

Reference and query molecular measurements, compatible feature identities, donor metadata and labels reserved for assessment.

Sources (2)

Evaluation procedure diagram

How it worksConceptual evaluation workflow

Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol.

Sources (2)

Source reviewed · Automated source review, 2026-09-16. All specifications and missing details

Results

All evaluations

0 evaluations · 0 results. Different protocols are not a single leaderboard.

Applied filters: All linked evaluations

No evaluations linked in this release.

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Choosing an evaluation

Keep donor identity visible throughout the evaluation. The reference and any fitted annotation rules belong to the training data; a query donor supplies the held-out assessment. Cell representations and transferred labels are different outputs and require different scoring. This is a proposed task definition, not a claim that every reference-mapping paper uses donor holdout.

Sources (2)

Task scope

This is a task guide, not a single versioned benchmark protocol. The connected resources provide examples or concrete procedures. A candidate method or proposed control is not evidence that an evaluation has been completed.

Sources (2)

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths supported by sources

Separating donors makes the intended generalisation question explicit and avoids confusing held-out cells with a new biological replicate.
Sources (2)
scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Limitations and conditions

A donor split alone does not control every difference in assay, tissue or cell composition. This guide does not define a particular cohort.
Sources (2)
scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Profile review details

Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol.

Stable record: catalog-task-cell-reference-mapping

Specifications

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Entity type	Task guide; concrete protocol identities remain separate. Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Datasets	No single dataset is fixed by this guide. Select a linked protocol and its versioned data release. · Not applicable Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Organisms	No shared organism population is defined at this guide level. Record it for each selected dataset. · Not applicable Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Assays	No single measurement assay is fixed by this guide; the endpoint and assay belong to the selected protocol. · Not applicable Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Splits	This proposed guide requires donor separation; no particular donor assignment or cohort is fixed here. · Not applicable Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Allowed inputs	Reference and query molecular measurements, compatible feature identities, donor metadata and labels reserved for assessment. Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Adaptation	No common fitting regime is imposed here. Keep pretrained, frozen, probed, fine-tuned and conventional methods distinct where applicable. · Not applicable Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Metrics	Choose label-transfer metrics for annotations and biological-conservation metrics for representations. The exact metric, averaging and unknown-cell handling belong to the concrete protocol. Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.
Baselines	Specify a reference-only comparator with the same available labels and features. No measured baseline is assigned here. Sources (2) scverse/scvi-tools: docs/user_guide/models/scvi.md; theislab/scib official source · scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.

Applicable tests and references

Applicability is distinct from a completed evaluation.

scVI · proposed

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.

Paper or primary resource	Version	Reference
Probabilistic harmonization and annotation of single‐cell transcriptomics data with deep generative models	PMC full-text XML snapshot PMC7829634 at retrieval; byte-pinned by SHA-256	Read source DOI: 10.15252/msb.20209620
Defining and benchmarking open problems in single-cell analysis	PMC full-text XML snapshot PMC11030530 at retrieval; byte-pinned by SHA-256	Read source DOI: 10.21203/rs.3.rs-4181617/v1
openproblems-label primary benchmark evidence	v1.0.0	Read source

Historical gaps recorded on 2026-09-17

A split by batch/reference is not automatically a donor-held-out split. A chart needs the exact dataset donor manifest.
Open Problems controls and normalization choices must remain separate from raw label accuracy/F1.

Search and extraction details

broad task primary protocol linked

Searches

Open Problems single cell label projection benchmark v1.0.0 paper
CAPRI assessment rounds 46 54 protein docking results
GEARS predicting transcriptional outcomes multigene perturbations 2023
scVI scANVI reference mapping benchmark primary paper
Boltz-2 affinity benchmark paper
ESMFold monomer structure benchmark Science 2023
"ProkBERT" "paper" "2024"
"scPertEval"
ProkBERT family prokaryotic language models microbe paper
Probabilistic harmonization annotation single-cell transcriptomics scANVI Nature Methods 2021
Evolutionary-scale prediction atomic-level protein structure language model ESMFold Science Lin 2023
Towards Principled Evaluation Single-Cell Perturbation Prediction Models Schäfer 2026

Evidence locations

scANVI Results and evaluation metrics
Open Problems: Label projection
Versioned label-projection task page

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

34 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol. Individual claims	scverse/scvi-tools: docs/user_guide/models/scvi.md Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 73b28e44223621470e582a81a102c107bb22678b Retrieved: 2026-09-16T19:46:20.137176+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef` Hash scope: SHA-256 of retrieved original artifact bytes Format: original_artifact Inspected artifact
Diagram caption Conceptual task guide. Dataset preparation, parameters and scoring must come from a separately identified protocol. Individual claims	theislab/scib official source Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: cd67913396b4c0430710b3d90f1d1841f5fa4468 Retrieved: 2026-09-16T10:31:32.426513+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa` Hash scope: Hash scope not separately documented; inspect source record
Diagram steps Define reference and query donors Fit reference using permitted data Map held-out query cells Score labels and biological structure Individual claims	scverse/scvi-tools: docs/user_guide/models/scvi.md Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 73b28e44223621470e582a81a102c107bb22678b Retrieved: 2026-09-16T19:46:20.137176+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef` Hash scope: SHA-256 of retrieved original artifact bytes Format: original_artifact Inspected artifact
Diagram steps Define reference and query donors Fit reference using permitted data Map held-out query cells Score labels and biological structure Individual claims	theislab/scib official source Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: cd67913396b4c0430710b3d90f1d1841f5fa4468 Retrieved: 2026-09-16T10:31:32.426513+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa` Hash scope: Hash scope not separately documented; inspect source record
Diagram title Conceptual evaluation workflow Individual claims	scverse/scvi-tools: docs/user_guide/models/scvi.md Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 73b28e44223621470e582a81a102c107bb22678b Retrieved: 2026-09-16T19:46:20.137176+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef` Hash scope: SHA-256 of retrieved original artifact bytes Format: original_artifact Inspected artifact
Diagram title Conceptual evaluation workflow Individual claims	theislab/scib official source Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: cd67913396b4c0430710b3d90f1d1841f5fa4468 Retrieved: 2026-09-16T10:31:32.426513+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa` Hash scope: Hash scope not separately documented; inspect source record
Entity type Task guide; concrete protocol identities remain separate. Individual claims	scverse/scvi-tools: docs/user_guide/models/scvi.md Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 73b28e44223621470e582a81a102c107bb22678b Retrieved: 2026-09-16T19:46:20.137176+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef` Hash scope: SHA-256 of retrieved original artifact bytes Format: original_artifact Inspected artifact
Entity type Task guide; concrete protocol identities remain separate. Individual claims	theislab/scib official source Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: cd67913396b4c0430710b3d90f1d1841f5fa4468 Retrieved: 2026-09-16T10:31:32.426513+00:00	source checked automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa` Hash scope: Hash scope not separately documented; inspect source record
Datasets No single dataset is fixed by this guide. Select a linked protocol and its versioned data release. Individual claims	scverse/scvi-tools: docs/user_guide/models/scvi.md Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 73b28e44223621470e582a81a102c107bb22678b Retrieved: 2026-09-16T19:46:20.137176+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `c81a5cfb30db1eca8c03af3d0db1a36a447a1bb5a27b270de5633ba73ebeeeef` Hash scope: SHA-256 of retrieved original artifact bytes Format: original_artifact Inspected artifact
Datasets No single dataset is fixed by this guide. Select a linked protocol and its versioned data release. Individual claims	theislab/scib official source Original source ↗ scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default. Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: cd67913396b4c0430710b3d90f1d1841f5fa4468 Retrieved: 2026-09-16T10:31:32.426513+00:00	inapplicable automated source review · 2026-09-16 Audit details Reviewed named primary documentation and existing protocol records. Editorial task scope is distinguished from source-reported procedures. No model execution or new numerical result; a reviewed guide is not a complete runnable protocol. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `db7aa3a701778d541bf47d8214b50e1ce9f73e92dc3e810bfd740270e9e353aa` Hash scope: Hash scope not separately documented; inspect source record

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: discovered

9 source records and release history

Geneformer official resource · Original source · Not reported
scGPT official resource · Original source · Not reported
scFoundation official resource · Original source · Not reported
scVI official resource · Original source · Not reported
openproblems-label primary benchmark evidence · Original source · v1.0.0
Defining and benchmarking open problems in single-cell analysis · Original source · PMC full-text XML snapshot PMC11030530 at retrieval; byte-pinned by SHA-256
Probabilistic harmonization and annotation of single‐cell transcriptomics data with deep generative models · Original source · PMC full-text XML snapshot PMC7829634 at retrieval; byte-pinned by SHA-256
scverse/scvi-tools: docs/user_guide/models/scvi.md · Original source · 73b28e44223621470e582a81a102c107bb22678b
theislab/scib official source · Original source · cd67913396b4c0430710b3d90f1d1841f5fa4468

Download this release

Technical metadata and extraction receipts

Stable ID: catalog-task-cell-reference-mapping

areas: cells-tissues
entity level: task
version: Not reported
task: Donor-held-out reference mapping
scope note: Map unseen donors to a labelled cell-type reference.
benchmark research: review date: 2026-09-17; status: broad_task_primary_protocol_linked; primary sources: expansion-p3-scanvi; expansion-p3-open-problems-paper; expansion-p3-open-problems; inspected locators: scANVI Results and evaluation metrics; Open Problems: Label projection; Versioned label-projection task page; searched queries: Open Problems single cell label projection benchmark v1.0.0 paper; CAPRI assessment rounds 46 54 protein docking results; GEARS predicting transcriptional outcomes multigene perturbations 2023; scVI scANVI reference mapping benchmark primary paper; Boltz-2 affinity benchmark paper; ESMFold monomer structure benchmark Science 2023; "ProkBERT" "paper" "2024"; "scPertEval"; ProkBERT family prokaryotic language models microbe paper; Probabilistic harmonization annotation single-cell transcriptomics scANVI Nature Methods 2021; Evolutionary-scale prediction atomic-level protein structure language model ESMFold Science Lin 2023; Towards Principled Evaluation Single-Cell Perturbation Prediction Models Schäfer 2026; gaps: A split by batch/reference is not automatically a donor-held-out split. A chart needs the exact dataset donor manifest.; Open Problems controls and normalization choices must remain separate from raw label accuracy/F1.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
historical missing metadata: protocol version: not_yet_extracted
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
legacy kinds: benchmark
entity classification: review date: 2026-09-17; rationale: This record identifies the biological prediction question or a suite-specific task, rather than a uniquely fixed evaluated procedure. Preserve its task identity and leave split, model adaptation and scoring details on linked protocols/evaluations.; source ids: evidence-official-0cbab80c2d1769287a1f; src-discovery-theislab-scib; source locator: scVI model guide: generative model and latent representation; scIB README: biological-conservation metrics. Donor holdout is this guide’s evaluation requirement, not an asserted scVI default.; ambiguities: None recorded

Related records

applicable to: scVI

Suggest a correction