rewirebio.iobenchmarks
Evidence claim

Relevance of Abdelaal et al. 2019 intra-dataset 5-fold CV: Baron Human pancreatic dataset to "Transfer cell-type annotations to a new dataset"

Use case
Transfer cell-type annotations to a new dataset
Assessed by
Abdelaal et al. 2019 intra-dataset 5-fold CV: Baron Human pancreatic dataset
Relevance
Proxy evidence
Endpoint
Intra-dataset (same single Baron Human dataset, stratified 5-fold cross-validation, identical folds across classifiers) median F1-score and, for rejection-capable classifiers, percentage of cells left unlabeled.

A conventional-classifier rejection-option baseline directly relevant to the use case's 'identify cells that require expert review' endpoint, but intra-dataset only; it does not establish cross-study or cross-platform transfer, and the compared classifiers (SVM, SVMrejection, scmapcell, scPred) predate scTab, CellTypist, Azimuth, scANVI and scParadise.

4 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

15 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    ucc-research-source-abdelaal-2019-paper
    locator
    Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
2 values
  • INTRA-dataset (same single dataset, stratified 5-fold CV, identical folds across classifiers); NOT cross-study or cross-platform. Confirmed from the source's own Methods, which explicitly distinguishes this design from a separate inter-dataset design.
  • Baron Human dataset only: 8,569 cells, 17,499 genes, 14 cell populations (13 after <10-cell filtering), inDrop protocol (Table 2).
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Intra-dataset (same single Baron Human dataset, stratified 5-fold cross-validation, identical folds across classifiers) median F1-score and, for rejection-capable classifiers, percentage of cells left unlabeled.
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:ucc-research-protocol-abdelaal-baron-human-intra
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
8 values
  • Does not establish cross-study or cross-platform transfer; the source's own inter-dataset (cross-platform/cross-study) experiments (Fig. 4 brain, Fig. 5 pancreatic) are not ingested in this mapping.
  • Median F1-score is a per-dataset summary statistic across cell populations, not a per-population breakdown.
  • SVM (no rejection) is quoted as classifying "100% of the cells" (0% unlabeled by implication) but is not given its own rejection-percentage result record in this intake.
  • Reference labels are the original Baron et al. study's author-assigned cell-population annotations, not independently adjudicated ground truth.
  • Exact per-fold cell counts and the fold manifest are not stated in the retrieved main text.
  • 8,569 cells is the Table 2 Baron Human dataset-size figure; it is not independently confirmed as the exact number of cells scored for any individual classifier's median F1 or unlabeled-percentage result.
  • Percentage unlabeled is a coverage/rejection-rate tradeoff against the same classifier's own median F1-score, not a standalone performance figure with a universal better/worse direction; it must be read jointly with that classifier's F1, not in isolation. The corresponding result records carry metric_direction: "unknown" for this reason, per the schema's higher/lower/unknown contract.
  • Automated source review only; independent human scientific review remains outstanding.
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
14 values
  • record id
    ucc-research-eval-abdelaal-baron-human-scmapcell
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    9affc03fa8d6d155ceb3fc55151569a575201d45a853284a8577d5952fab081d
  • record id
    ucc-research-eval-abdelaal-baron-human-scpred
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    054a876c3926680682c0fefc453cf0cc0f38116fc6b468c227c4cc021ca81d12
  • record id
    ucc-research-eval-abdelaal-baron-human-svm
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    8e99f8d51eeb0ea479929273c864bc7ed23fdf95c836fba14f32fb3a716aa146
  • record id
    ucc-research-eval-abdelaal-baron-human-svm-rejection
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    9a575a548e45a2c094f06ca1808020aeb897c1ca8928cb05adb4f671963448e7
  • record id
    ucc-research-protocol-abdelaal-baron-human-intra
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    9da4500703d2dfe403d0ee802dd244b7e3ac96321c98a5bf5c86c91ac1b1dfec
  • record id
    ucc-research-result-abdelaal-baron-human-scmapcell-medianf1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    63f56178164deba38929031d3e4aa4a33f80669fedb7b030ae6eb83507e8e0ac
  • record id
    ucc-research-result-abdelaal-baron-human-scmapcell-unlabeled
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    df7c6aaa025fb8fa89d261b2edcad9a853a8990e874d8a22f697287695a8254b
  • record id
    ucc-research-result-abdelaal-baron-human-scpred-medianf1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    4ea19ff48e93dcb096eaecf545d2479259dab9dc42dbb1eda37d1f2f865a3215
  • record id
    ucc-research-result-abdelaal-baron-human-scpred-unlabeled
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    b308a96f4f2c018bb3a3342d4584d0f59398894acd1c8b8606e8e382aa4f65df
  • record id
    ucc-research-result-abdelaal-baron-human-svm-medianf1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f1e7cdd1b9505675a83bdd5f88414e53dd309a6145c3a108a4615ea4f3496acd
  • record id
    ucc-research-result-abdelaal-baron-human-svm-rejection-medianf1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    c88881d69c19e279f0968340a48aad83b235463071cf4c2c2ea184538c5f291f
  • record id
    ucc-research-result-abdelaal-baron-human-svm-rejection-unlabeled
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    a404e31229a302ade2e1710f41d246741707d99ab06fed8a57d760bde244a700
  • record id
    ucc-research-source-abdelaal-2019-paper
    fields
    • status
    • attributes.artifact_sha256
    sha256
    e7c72be59b6152bcd5c356c70d58744cf8ec978a5be40d011422ee8f4c2b5f18
  • record id
    use-case-cell-type-annotation-transfer
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
A conventional-classifier rejection-option baseline directly relevant to the use case's 'identify cells that require expert review' endpoint, but intra-dataset only; it does not establish cross-study or cross-platform transfer, and the compared classifiers (SVM, SVMrejection, scmapcell, scPred) predate scTab, CellTypist, Azimuth, scANVI and scParadise.
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Reason
Add Abdelaal et al. 2019 Baron Human INTRA-dataset rejection-option baseline (same-day continuation of the additive 2026-10-06 intake, issue #349): four median F1 values and three cells-left-unlabeled percentages for conventional classifiers, re-verified directly against the cached primary source. This is explicitly NOT a cross-study or cross-platform result; the source's own genuine inter-dataset experiments (Fig. 4, Fig. 5) are not ingested here.
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.reason

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relevance
proxy
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.relevance

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Revision
1
Context-only references
A comparison of automatic cell identification methods for single-cell RNA sequencing data

Original source ↗

ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2

Version: 10.1186/s13059-019-1795-z; published article XML
Retrieved: 2026-10-06T23:28:54Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.revision

Source artifact SHA-256: 6df0937c5a2d8ba06465ba356c2321fb9f52d9ecf23456876acd82f828b41d4c

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-20261006-349-967caca02e7d

field
links:assessed_by:ucc-research-protocol-abdelaal-baron-human-intra
value
ucc-research-protocol-abdelaal-baron-human-intra
relevance
proxy
endpoint
Intra-dataset (same single Baron Human dataset, stratified 5-fold cross-validation, identical folds across classifiers) median F1-score and, for rejection-capable classifiers, percentage of cells left unlabeled.
rationale
A conventional-classifier rejection-option baseline directly relevant to the use case's 'identify cells that require expert review' endpoint, but intra-dataset only; it does not establish cross-study or cross-platform transfer, and the compared classifiers (SVM, SVMrejection, scmapcell, scPred) predate scTab, CellTypist, Azimuth, scANVI and scParadise.
constraints
INTRA-dataset (same single dataset, stratified 5-fold CV, identical folds across classifiers); NOT cross-study or cross-platform. Confirmed from the source's own Methods, which explicitly distinguishes this design from a separate inter-dataset design.; Baron Human dataset only: 8,569 cells, 17,499 genes, 14 cell populations (13 after <10-cell filtering), inDrop protocol (Table 2).
limitations
Does not establish cross-study or cross-platform transfer; the source's own inter-dataset (cross-platform/cross-study) experiments (Fig. 4 brain, Fig. 5 pancreatic) are not ingested in this mapping.; Median F1-score is a per-dataset summary statistic across cell populations, not a per-population breakdown.; SVM (no rejection) is quoted as classifying "100% of the cells" (0% unlabeled by implication) but is not given its own rejection-percentage result record in this intake.; Reference labels are the original Baron et al. study's author-assigned cell-population annotations, not independently adjudicated ground truth.; Exact per-fold cell counts and the fold manifest are not stated in the retrieved main text.; 8,569 cells is the Table 2 Baron Human dataset-size figure; it is not independently confirmed as the exact number of cells scored for any individual classifier's median F1 or unlabeled-percentage result.; Percentage unlabeled is a coverage/rejection-rate tradeoff against the same classifier's own median F1-score, not a standalone performance figure with a universal better/worse direction; it must be read jointly with that classifier's F1, not in isolation. The corresponding result records carry metric_direction: "unknown" for this reason, per the schema's higher/lower/unknown contract.; Automated source review only; independent human scientific review remains outstanding.
revision
1
reason
Add Abdelaal et al. 2019 Baron Human INTRA-dataset rejection-option baseline (same-day continuation of the additive 2026-10-06 intake, issue #349): four median F1 values and three cells-left-unlabeled percentages for conventional classifiers, re-verified directly against the cached primary source. This is explicitly NOT a cross-study or cross-platform result; the source's own genuine inter-dataset experiments (Fig. 4, Fig. 5) are not ingested here.
citation locators
source id: ucc-research-source-abdelaal-2019-paper; locator: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2
source locator
ucc-research-source-abdelaal-2019-paper: Results, "All classifiers perform well in intra-dataset experiments"; Methods, "Intra-dataset classification"; Fig. 1a,b; Table 1; Table 2
review
method: automated-source-review; reviewer: claude-sonnet; reviewer note: Claude Sonnet cell-type-annotation-transfer evidence-research worker; reviewed at: 2026-10-06T23:32:17Z; note: Source-backed primary-text transcription of Abdelaal et al. 2019, continuing the bounded 2026-10-06 cell-type-annotation-transfer research dossier and same-day additive intake; bound by data/omics/use-case-coverage-20261006/review.json. Revised after an independent review: the source's retrieved_at is now the real re-fetch timestamp 2026-10-06T23:28:54Z UTC (byte-identical; a bounded-window estimate was used previously), its bytes are archived as a committed artifact (not only the gitignored workbench copy), the three unlabeled-percentage results' metric_direction is now "unknown" (was "lower", an unsupported universal claim) with an explicit F1/rejection-tradeoff note, and the 8,569-cell figure is now explicitly scoped as the Table 2 dataset size, not a confirmed scored denominator. No printed value changed; evidence_sha256 refreshed accordingly. scArches's ~84% accuracy figure was re-checked in the same pass and found unresolved (accuracy-formula denominator not stated for the reporting figure); no record for it exists in this release. No new model execution, independent experimental replication, qualified human scientific review or clinical validation.
reviewed evaluations
ucc-research-eval-abdelaal-baron-human-scmapcell; ucc-research-eval-abdelaal-baron-human-scpred; ucc-research-eval-abdelaal-baron-human-svm; ucc-research-eval-abdelaal-baron-human-svm-rejection
pins
record id: ucc-research-eval-abdelaal-baron-human-scmapcell; fields: status; links; attributes.origin; attributes.comparison; sha256: 9affc03fa8d6d155ceb3fc55151569a575201d45a853284a8577d5952fab081d; record id: ucc-research-eval-abdelaal-baron-human-scpred; fields: status; links; attributes.origin; attributes.comparison; sha256: 054a876c3926680682c0fefc453cf0cc0f38116fc6b468c227c4cc021ca81d12; record id: ucc-research-eval-abdelaal-baron-human-svm; fields: status; links; attributes.origin; attributes.comparison; sha256: 8e99f8d51eeb0ea479929273c864bc7ed23fdf95c836fba14f32fb3a716aa146; record id: ucc-research-eval-abdelaal-baron-human-svm-rejection; fields: status; links; attributes.origin; attributes.comparison; sha256: 9a575a548e45a2c094f06ca1808020aeb897c1ca8928cb05adb4f671963448e7; record id: ucc-research-protocol-abdelaal-baron-human-intra; fields: status; name; description; facets; source_ids; links; attributes; sha256: 9da4500703d2dfe403d0ee802dd244b7e3ac96321c98a5bf5c86c91ac1b1dfec; record id: ucc-research-result-abdelaal-baron-human-scmapcell-medianf1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 63f56178164deba38929031d3e4aa4a33f80669fedb7b030ae6eb83507e8e0ac; record id: ucc-research-result-abdelaal-baron-human-scmapcell-unlabeled; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: df7c6aaa025fb8fa89d261b2edcad9a853a8990e874d8a22f697287695a8254b; record id: ucc-research-result-abdelaal-baron-human-scpred-medianf1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4ea19ff48e93dcb096eaecf545d2479259dab9dc42dbb1eda37d1f2f865a3215; record id: ucc-research-result-abdelaal-baron-human-scpred-unlabeled; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: b308a96f4f2c018bb3a3342d4584d0f59398894acd1c8b8606e8e382aa4f65df; record id: ucc-research-result-abdelaal-baron-human-svm-medianf1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f1e7cdd1b9505675a83bdd5f88414e53dd309a6145c3a108a4615ea4f3496acd; record id: ucc-research-result-abdelaal-baron-human-svm-rejection-medianf1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c88881d69c19e279f0968340a48aad83b235463071cf4c2c2ea184538c5f291f; record id: ucc-research-result-abdelaal-baron-human-svm-rejection-unlabeled; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a404e31229a302ade2e1710f41d246741707d99ab06fed8a57d760bde244a700; record id: ucc-research-source-abdelaal-2019-paper; fields: status; attributes.artifact_sha256; sha256: e7c72be59b6152bcd5c356c70d58744cf8ec978a5be40d011422ee8f4c2b5f18; record id: use-case-cell-type-annotation-transfer; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: f4ef70c0d86c1d3ea8ff8600431eb0792da0bd5f355e7137bc364a42aa284bd9
Related records

Suggest a correction