Task

differentially expressed gene identification

Differential-expression evaluation compares predicted gene responses against observed differential-expression labels.

Sources

AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103

1 evaluation · 1 result

Overview

Datasets

Cell-type–perturbation combinations with responses available across donors.

Metrics

Gene-expression R-squared is contrasted with precision–recall evaluation of differentially expressed genes.

Allowed inputs

Cell type, perturbation identity and expression information permitted by the chosen model.

Sources

Evaluation procedure diagram

How it worksComputational evaluation flow

Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.

Sources

Source reviewed · Automated source review, 2026-09-16. All specifications and missing details

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

1 evaluation · 1 result. Different protocols are not a single leaderboard.

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: scGen	Task: differentially expressed gene identification Dataset: stimulated immune PBMC	0.91 precision at 50% recall fraction · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible	Independent external evaluation · Source checked Methods, coverage and source scGen: differentially expressed gene identification In-silico perturbation assessment with precision sampled at fixed 50% recall Aggregation: Not reported AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Table 3, CD14+Mono section, scGen row, Precision at 50% Recall column

Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

Cell-type–perturbation combinations with responses available across donors. A subset of complete combinations is held out for testing; models are fitted separately for each donor. Gene-expression R-squared is contrasted with precision–recall evaluation of differentially expressed genes. Cell-type-only, perturbation-only and two-factor linear models, plus SI-A. Complete cell-type–perturbation combinations are held out. Models are fitted separately for each donor, so this is not an unseen-donor generalization test.

Sources

Recorded evaluations

Each evaluation records what was tested and under which conditions.

scGen: differentially expressed gene identification

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths supported by sources

Holding out complete cell-type–perturbation combinations tests compositional response prediction.
Sources
AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103

Limitations and conditions

Holding out cell-type–perturbation combinations is not donor-held-out evaluation. Distributions across testing pairs quantify heterogeneity across prediction problems, not confidence intervals for generalization to new donors.
Sources
AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Results: In silico models and evaluation of performance; Figure 6 caption; Results: DEG prediction on population-level responses under multiple perturbations across multiple cell types

Profile review details

Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change.

Stable record: reported-task-003d746a129c9b

Specifications

Inputs, training, access and other details

Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Datasets	Cell-type–perturbation combinations with responses available across donors. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Splits	A subset of complete combinations is held out for testing; models are fitted separately for each donor. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Metrics	Gene-expression R-squared is contrasted with precision–recall evaluation of differentially expressed genes. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Baselines	Cell-type-only, perturbation-only and two-factor linear models, plus SI-A. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Leakage controls	Complete cell-type–perturbation combinations are held out. Models are fitted separately for each donor, so this is not an unseen-donor generalization test. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Uncertainty	Figure 6 shows the distribution of AUPRC across 100 held-out cell-type–perturbation pairs and marks their averages. These are between-pair distributions, not confidence intervals for an aggregate score; no such interval is specified in this evaluation passage. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Results: In silico models and evaluation of performance; Figure 6 caption
Entity type	Paper-specific computational evaluation protocol. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Organisms	Human peripheral blood mononuclear cells from three healthy donors, measured 24 hours after compound treatment. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Results: DEG prediction on population-level responses under multiple perturbations across multiple cell types
Assays	Measured expression responses and differential-expression targets. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Allowed inputs	Cell type, perturbation identity and expression information permitted by the chosen model. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103
Adaptation	Models fit separately per donor with complete cell-type–perturbation combinations held out. Sources AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Methods: In silico models and evaluation of performance; cached text lines 100–103

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.

Paper or primary resource	Version	Reference
AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes	PMC archival version PMC12400816.1	Read source DOI: 10.1093/bib/bbaf426

Historical gaps recorded on 2026-09-17

The catalogue now holds 1 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

Do not pool cell types or recall thresholds; table gives no uncertainty for these cells.

Search and extraction details

primary comparison table screened

Searches

"Normalized compression distance for DNA classification"
"AUPRC" "in-silico perturbation"
"mouse" "Geneformer" "PMC11964219"
"antibody flexibility" "PMC12530544"
"scXDR" cross dataset drug response
"Hi-enhancer" 2025
"PlantCaduceus2" 2025
"COBRA" "RNA" "PMC12790621"
"PMC12425018"
"PMC12758598"
"DeepInterAware" 2025
"BarcodeBERT" unseen species
"PMC8207588"
"PMC10723403"
"PMC11167433"
"PMC12826486"
"PMC11565894"
"PMC11695915"
"PMC9178954"
"PMC13418759"
"PMC12516880"
"PMC11815853"
"PMC12417085"
"PMC12889687"
"PMC12635123"
"PMC12957212"
"PMC12453675"
"PMC12493982"
"PMC12798952"
"PMC13132462"
"PMC11785235"
"PMC8763943"
"PMC13182013"
"PMC12619997"
ProkBERT promoter benchmark
scPertEval benchmark paper
GlycanML benchmark 2405.16206
AMBER metagenome binning assessment 2018 PMC6022608
BEELINE gene regulatory network benchmark 2020 Pratapa
FLIP benchmark protein fitness landscape inference 2021
Genomic Benchmarks collection genomic sequence classification PMC10150520
mRNABench PMC12265608
PFMBench 2506.14796
ProteinBench 2409.06744

Evidence locations

Table 3
Cell-level responses under single stimulus

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

18 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram steps Input: Cell type, perturbation identity and expression information permitted by the chosen model. Evaluation: A subset of complete combinations is held out for testing; models are fitted separately for each donor. Readout: Gene-expression R-squared is contrasted with precision–recall evaluation of differentially expressed genes. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram title Computational evaluation flow Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Datasets Cell-type–perturbation combinations with responses available across donors. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Splits A subset of complete combinations is held out for testing; models are fitted separately for each donor. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Adaptation Models fit separately per donor with complete cell-type–perturbation combinations held out. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.10.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Metrics Gene-expression R-squared is contrasted with precision–recall evaluation of differentially expressed genes. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.2.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Baselines Cell-type-only, perturbation-only and two-factor linear models, plus SI-A. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.3.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Leakage controls Complete cell-type–perturbation combinations are held out. Models are fitted separately for each donor, so this is not an unseen-donor generalization test. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Methods: In silico models and evaluation of performance; cached text lines 100–103 Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.4.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Uncertainty Figure 6 shows the distribution of AUPRC across 100 held-out cell-type–perturbation pairs and marks their averages. These are between-pair distributions, not confidence intervals for an aggregate score; no such interval is specified in this evaluation passage. Individual claims	AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes Original source ↗ Results: In silico models and evaluation of performance; Figure 6 caption Version: PMC archival version PMC12400816.1 Retrieved: 2026-09-16T10:41:06Z	source checked automated source review · 2026-09-16 Audit details Targeted full-paper and supplement review of the outstanding task fields, with original dataset metadata checked where accessible. Source-scoped omissions are explicit; no independent benchmark reproduction or numerical-result change. Field: `attributes.profile.facts.5.value` Source artifact SHA-256: `2715709d94f84744afa32cafdcaa72efd206d63af8c60afe7619b2cb90108b6b` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: needs review

2 source records and release history

AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Original source · PMC archival version PMC12400816.1
AUPRC: a metric for evaluating the performance of in-silico perturbation methods in identifying differentially expressed genes · Original source · PMC archival version PMC12400816.1

Download this release

Technical metadata and extraction receipts

Stable ID: reported-task-003d746a129c9b

areas: cells-tissues
tasks: differentially expressed gene identification
entity level: task
version: Not reported
task: differentially expressed gene identification
scope note: Paper-specific evaluation task; protocol completeness requires further extraction.
benchmark research: review date: 2026-09-17; status: primary_comparison_table_screened; primary sources: expansion-p3-insilico-perturbation-auprc-2025; inspected locators: Table 3; Cell-level responses under single stimulus; searched queries: "Normalized compression distance for DNA classification"; "AUPRC" "in-silico perturbation"; "mouse" "Geneformer" "PMC11964219"; "antibody flexibility" "PMC12530544"; "scXDR" cross dataset drug response; "Hi-enhancer" 2025; "PlantCaduceus2" 2025; "COBRA" "RNA" "PMC12790621"; "PMC12425018"; "PMC12758598"; "DeepInterAware" 2025; "BarcodeBERT" unseen species; "PMC8207588"; "PMC10723403"; "PMC11167433"; "PMC12826486"; "PMC11565894"; "PMC11695915"; "PMC9178954"; "PMC13418759"; "PMC12516880"; "PMC11815853"; "PMC12417085"; "PMC12889687"; "PMC12635123"; "PMC12957212"; "PMC12453675"; "PMC12493982"; "PMC12798952"; "PMC13132462"; "PMC11785235"; "PMC8763943"; "PMC13182013"; "PMC12619997"; ProkBERT promoter benchmark; scPertEval benchmark paper; GlycanML benchmark 2405.16206; AMBER metagenome binning assessment 2018 PMC6022608; BEELINE gene regulatory network benchmark 2020 Pratapa; FLIP benchmark protein fitness landscape inference 2021; Genomic Benchmarks collection genomic sequence classification PMC10150520; mRNABench PMC12265608; PFMBench 2506.14796; ProteinBench 2409.06744; gaps: Do not pool cell types or recall thresholds; table gives no uncertainty for these cells.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
historical missing metadata: protocol version: not_reported_in_legacy_extract
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
legacy kinds: benchmark
entity classification: review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: insilico-perturbation-auprc-2025; source locator: Methods: In silico models and evaluation of performance; cached text lines 100–103; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.

Related records

dataset: stimulated immune PBMC
benchmark: scGen: differentially expressed gene identification

Suggest a correction