Benchmark

Virtual Cell Challenge 2026

The 2026 Virtual Cell Challenge evaluates perturbation-response prediction in unseen cellular contexts.

Sources

vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring

1,048 evaluations · 1,048 results

Overview

Datasets

Challenge-specific cellular-context evaluation data; the previous year’s released dataset is a separate resource.

Sources

vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring

Metrics

Six calibrated components: perturbation discrimination, expression accuracy, differential-expression log-fold-change accuracy, direction fidelity, direction reach and significance overlap. Their unweighted mean is the overall score; each result also needs its partition, perturbation panel and anchor-set identity.

Sources

vcc-guide primary benchmark evidence · VCC CLI guide: score interpretation and partition/panel/anchor stamp

Allowed inputs

Released challenge inputs and the submission schema for the 2026 edition.

Sources

vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring

Evaluation procedure diagram

How it worksEvaluation procedure

Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.

Sources (2)

vcc2026 official source; vcc-guide primary benchmark evidence · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp

limited source coverage · Automated source review, 2026-09-16. All specifications and missing details

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

Find a protocol, dataset or metric

Evaluation scope and datasetMetric and reported setup

vcc2026-val-1 · 2026 validation contexts A, B and C: score_avg (source entries 1–80 of 1048)

score_avg (score) · Higher values are better.

vcc2026-val-1 · 2026 validation contexts A, B and C · 2026 validation contexts A, B and C

Evidence origin: Author-reported evaluation.

Virtual Cell Challenge 2026: 2026-09-19 snapshot; panel vcc2026-val-1 · JSON entries[0].score_avg; submission fslq6ov1ijfW0MlMkAii through JSON entries[79].score_avg; submission v6KFhMYYu9nNj9choway

Provisional overall validation score only. Team/model labels do not establish a catalogue model identity; do not infer family aliases. Final test results unavailable at snapshot date.
Missing source cells and quarantined conflicts are recorded in acquisition and audit tables. Per-result scoring denominators may be unreported.

Comparison details and limitations

Complete selected source table is retained across source-order panels. These point estimates do not establish statistical significance or a universal ranking.

Source-specific evaluation. No equivalence to other releases, protocols or model families is inferred.

Automated source review: 2026-09-19. Numerical source review does not establish independent reproduction.

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Find a tested model or method

Showing 12 of 80 matching rows.

Zoom to observed range

Methods and evaluation design

Procedure, tasks and evaluated configurations

How it works

Evaluation methodology

The 2026 challenge predicts CRISPRi responses in cellular contexts with no challenge-specific perturbation training set. Participants receive unperturbed cells and target-gene identifiers; three contexts support validation and three different contexts support final testing. Scores are calibrated against baseline and split-half experimental references.

Sources

vcc-guide primary benchmark evidence · VCC CLI guide: context labels and interpretation of the six metrics

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Protocols

vcc2026-val-1 · 2026 validation contexts A, B and C

These source-backed links do not make different protocols or scores interchangeable.

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 2 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

Baseline status by linked protocol

vcc2026-val-1 · 2026 validation contexts A, B and C · 0/2 roles measured

Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums

Coverage is derived from release 2026-09-29-06401fd5b220. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run this benchmark

Choose a concrete protocol before running an evaluation. Its inputs, split and scoring rules determine which results can be compared.

vcc2026-val-1 · 2026 validation contexts A, B and C

Run this benchmark

The official CLI guide explains prediction-file validation and challenge submission, and links the evaluation software. It does not supply a predictive model or establish that a local result matches the hidden challenge assessment. Authentication and explicit submission are separate actions; no submission command has been executed.

A maintained rewire runner has not been verified for this benchmark. Check data access, weights, licences, dependencies and hardware in the linked official documentation; requirements have not been fully extracted.

virtual-cell-challenge-2026 official run documentation · Official VCC CLI documentation: setup, prediction validation, submission and cell-eval2 link

Strengths, limitations and unresolved questions

Strengths and limitations

Strengths and considerations

A dated challenge identity prevents historical datasets being mistaken for the current evaluation.
Sources
vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring

Limitations and conditions

Validation and final phases use different contexts, perturbation panels and scoring bundles. Their scores are not directly comparable, and a replicate-calibrated value is neither a percentage correct nor an absolute biological ceiling.
Sources
vcc-guide primary benchmark evidence · VCC CLI guide: context labels and interpretation of the six metrics

Profile review details

Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.

Stable record: discovery-benchmark-virtual-cell-challenge-2026

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Data, procedure and scoring
Property	Description and evidence
Datasets	Challenge-specific cellular-context evaluation data; the previous year’s released dataset is a separate resource. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Splits	Validation contexts support the live leaderboard and other contexts are held for final testing; there is no challenge-specific training set. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Metrics	Six calibrated components: perturbation discrimination, expression accuracy, differential-expression log-fold-change accuracy, direction fidelity, direction reach and significance overlap. Their unweighted mean is the overall score; each result also needs its partition, perturbation panel and anchor-set identity. Sources vcc-guide primary benchmark evidence · VCC CLI guide: score interpretation and partition/panel/anchor stamp
Baselines	Participants may choose modeling strategies and train on public or their own datasets. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Leakage controls	Unseen-context generalization defines the challenge; exact admissibility and overlap rules require the detailed competition protocol. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Uncertainty	The score scale uses a mean-effect baseline and a split-half replicate of the real experiment as reference points. Replicate noise is therefore represented in calibration, but the guide does not define a universal confidence interval on the final leaderboard score. Sources vcc-guide primary benchmark evidence · VCC CLI guide: context labels and interpretation of the six metrics
Entity type	Dated cellular perturbation prediction challenge. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Organisms	The released contexts are deliberately anonymized cell lines A/B/C for validation and D/E/F for final testing. The inspected public announcement and CLI guide do not provide their exact line identities or a per-context organism manifest; do not infer them from the 2025 H1 dataset. · Not reported in inspected sources Sources vcc-guide primary benchmark evidence · VCC CLI guide: context labels and interpretation of the six metrics
Assays	Challenge-specific perturbation response data; prior-year data form a different resource. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Allowed inputs	Released challenge inputs and the submission schema for the 2026 edition. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring
Adaptation	Rules and deadlines belong to this challenge edition; prior-year adaptation conditions are not automatically inherited. Sources vcc2026 official source · Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring

Applicable tests and references

Applicability is distinct from a completed evaluation.

STATE · Proposed association

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Papers and result coverage

Last literature check: 2026-09-17. Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.

Paper or primary resource	Version	Reference
Submitting with the VCC CLI \| VCC CLI	Retrieved 2026-09-16	Read source

Historical gaps recorded on 2026-09-17

The catalogue now holds 1,048 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.

Challenge scope found; this pass does not establish a final published 2026 result table. Do not copy 2025 rankings into 2026 challenge.

Search and extraction details

protocol found final result table not established

Searches

Virtual Cell Challenge 2026 primary paper benchmark results

Evidence locations

Official2026 challenge guide and task description

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

21 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims	vcc2026 official source Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e Retrieved: 2026-09-16T10:31:54.531595+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims	vcc-guide primary benchmark evidence Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved 2026-09-16 Retrieved: 2026-09-16T21:16:29.975126+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.caption` Source artifact SHA-256: `a2a085fd104ecf6b97e125ebc5714c642e8140db4f549ff84abd3bce1d2a2c80` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram steps Allowed inputs: Released challenge inputs and the submission schema for the 2026 edition. Splits: Validation contexts support the live leaderboard and other contexts are held for final testing; there is no challenge-specific training set. Metrics: Six calibrated components: perturbation discrimination, expression accuracy, differential-expression log-fold-change accuracy, direction fidelity, direction reach and significance overlap. Their unweighted mean is the overall score; each result also needs its partition, perturbation panel and anchor-set identity. Individual claims	vcc2026 official source Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e Retrieved: 2026-09-16T10:31:54.531595+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram steps Allowed inputs: Released challenge inputs and the submission schema for the 2026 edition. Splits: Validation contexts support the live leaderboard and other contexts are held for final testing; there is no challenge-specific training set. Metrics: Six calibrated components: perturbation discrimination, expression accuracy, differential-expression log-fold-change accuracy, direction fidelity, direction reach and significance overlap. Their unweighted mean is the overall score; each result also needs its partition, perturbation panel and anchor-set identity. Individual claims	vcc-guide primary benchmark evidence Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved 2026-09-16 Retrieved: 2026-09-16T21:16:29.975126+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.steps` Source artifact SHA-256: `a2a085fd104ecf6b97e125ebc5714c642e8140db4f549ff84abd3bce1d2a2c80` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram title Evaluation procedure Individual claims	vcc2026 official source Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e Retrieved: 2026-09-16T10:31:54.531595+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Diagram title Evaluation procedure Individual claims	vcc-guide primary benchmark evidence Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; VCC CLI guide: score interpretation and partition/panel/anchor stamp Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: Retrieved 2026-09-16 Retrieved: 2026-09-16T21:16:29.975126+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.diagram.title` Source artifact SHA-256: `a2a085fd104ecf6b97e125ebc5714c642e8140db4f549ff84abd3bce1d2a2c80` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Datasets Challenge-specific cellular-context evaluation data; the previous year’s released dataset is a separate resource. Individual claims	vcc2026 official source Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring Version: Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e Retrieved: 2026-09-16T10:31:54.531595+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.0.value` Source artifact SHA-256: `57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Splits Validation contexts support the live leaderboard and other contexts are held for final testing; there is no challenge-specific training set. Individual claims	vcc2026 official source Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring Version: Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e Retrieved: 2026-09-16T10:31:54.531595+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.1.value` Source artifact SHA-256: `57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Adaptation Rules and deadlines belong to this challenge edition; prior-year adaptation conditions are not automatically inherited. Individual claims	vcc2026 official source Original source ↗ Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring Version: Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e Retrieved: 2026-09-16T10:31:54.531595+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.10.value` Source artifact SHA-256: `57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Metrics Six calibrated components: perturbation discrimination, expression accuracy, differential-expression log-fold-change accuracy, direction fidelity, direction reach and significance overlap. Their unweighted mean is the overall score; each result also needs its partition, perturbation panel and anchor-set identity. Individual claims	vcc-guide primary benchmark evidence Original source ↗ VCC CLI guide: score interpretation and partition/panel/anchor stamp Version: Retrieved 2026-09-16 Retrieved: 2026-09-16T21:16:29.975126+00:00	source checked automated source review · 2026-09-16 Audit details Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: `attributes.profile.facts.2.value` Source artifact SHA-256: `a2a085fd104ecf6b97e125ebc5714c642e8140db4f549ff84abd3bce1d2a2c80` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: discovered

5 source records and release history

vcc2026 official source · Original source · Not reported
Submitting with the VCC CLI | VCC CLI · Original source · Retrieved 2026-09-16
vcc2026 official source · Original source · Retrieved website snapshot sha256:57dd9cbd5e6b62ed665eebd909d82ffec580a55c3bd2f48a928f807a8c983b1e
virtual-cell-challenge-2026 official run documentation · Original source · Official documentation snapshot 2026-09-17; mutable website
vcc-guide primary benchmark evidence · Original source · Retrieved 2026-09-16

Download this release

Technical metadata and extraction receipts

Stable ID: discovery-benchmark-virtual-cell-challenge-2026

areas: single-cell
entity level: challenge
scope note: Specialist molecular or omics evaluation; protocol details require review before numerical comparison.
task: Zero-shot perturbation prediction in unseen cellular contexts
version: Not reported
benchmark research: review date: 2026-09-17; status: protocol_found_final_result_table_not_established; primary sources: evidence-expansion-p2-evidence-discovery-final-vcc-guide-a2a085fd104e; inspected locators: Official2026 challenge guide and task description; searched queries: Virtual Cell Challenge 2026 primary paper benchmark results; gaps: Challenge scope found; this pass does not establish a final published 2026 result table. Do not copy 2025 rankings into 2026 challenge.; claim scope: Primary-source discovery and table/protocol screening; source checked is not independently reproduced. Raw acquisitions not automatically numerical publication approval.
historical missing metadata: dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted
metadata review scope: historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
entity classification: review date: 2026-09-17; rationale: The cited profile describes an organized challenge with category- or round-specific evaluation rules; retain it as a top-level benchmark, without conflating different editions or protocols.; source ids: evidence-benchmark-vcc2026-snapshot; source locator: Arc Institute 2026 Virtual Cell Challenge announcement: task; context holdouts; inputs; scoring; ambiguities: None recorded
run documentation: record id: discovery-benchmark-virtual-cell-challenge-2026; source ids: run-doc-virtual-cell-challenge-2026-official-20260917; status: official_documentation_linked; summary: The official CLI guide explains prediction-file validation and challenge submission, and links the evaluation software. It does not supply a predictive model or establish that a local result matches the hidden challenge assessment. Authentication and explicit submission are separate actions; no submission command has been executed.; source locator: Official VCC CLI documentation: setup, prediction validation, submission and cell-eval2 link

Related records

part of: vcc2026-val-1 · 2026 validation contexts A, B and C
applicable to: STATE

Suggest a correction