CPPC Table S2 original and reimplemented submission comparisons
Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- CPPC original submission row 2 (PPI_Weighted): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, PPI_Weighted: features averaged over STRING-network neighbours, then a fitted predictor)
- CPPC original submission row 3 (GWPS_K562 (H2M)): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, GWPS_K562_H2M: "An MLP regressor with a single hidden layer of latent dimension 10 is then trained")
- CPPC original submission row 4 (14Features): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, 14Features: seven expression statistics on control and perturbed cells, then a fitted predictor)
- CPPC original submission row 5 (scBERT): facets.method_types: Facets: method types ["foundation_model"] (CPPC Supplementary Note S1, scBERT: "extracts pretrained human gene embeddings from scBERT", reference 7 is titled "scBERT as a large-scale pretrained deep language model")
- CPPC original submission row 6 (scVI): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, scVI: gene embeddings from an LDVAE trained on the challenge data, then a fitted predictor)
- CPPC original submission row 7 (Expression_Conditioned): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, Expression_Conditioned: each gene represented by its expression across training conditions, then a fitted predictor)
- CPPC original submission row 8 (Exon): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, Exon: exon-length summary statistics, then a fitted predictor)
- CPPC original submission row 9 (Gene2Vec): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, Gene2Vec: pretrained gene2vec gene embeddings, then a fitted predictor)
- CPPC original submission row 12 (5Features): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, 5Features: expression summary statistics and cell-state marker expression, then a fitted predictor)
- CPPC original submission row 16 (scETM): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, scETM: gene embeddings from scETM trained on the challenge data, then a fitted predictor)
- CPPC original submission row 17 (scTenifoldNet): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, "Implementation of additional methods": scTenifoldNet virtual knockout, then "trains a random forest classifier" to select genes)
- CPPC original submission row 18 (GO_knn): facets.method_types: Facets: method types ["conventional_pipeline"] (CPPC Supplementary Note S1, GO_knn: the weighted average of the cell state proportions of the three most similar training perturbations; main-text Methods: "does not use an explicit predictor")
- CPPC original submission row 19 (LowExp): facets.method_types: Facets: method types ["conventional_pipeline"] (CPPC Supplementary Note S1, LowExp: proportions taken from the state annotations of the 5,000 training cells with the lowest expression of the gene)
- CPPC original submission row 21 (interv): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, Interv: LASSO cell-state classification and local LiNGAM regression)
- CPPC original submission row 22 (GO-node2vec): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, GO-node2vec: Node2Vec embeddings of the gene ontology, then a fitted predictor)
- CPPC original submission row 24 (GeneVector): facets.method_types: Facets: method types ["supervised_machine_learning"] (CPPC Supplementary Note S1, GeneVector: pretrained GeneVector gene embeddings, then a fitted predictor)
- Evidence summary: Select perturbations for a defined cellular response: Summary The evidence comes from one prospective challenge and two retrospective replays, which measure different things. In the Cancer Immunotherapy Data Science Challenge, scored prospectively in mouse T cells, the 20 original Challenge 2 submissions had a top-k overlap AUC of 0.452 to 0.566, with 17 of the 20 between 0.50 and 0.57; a random ordering is expected to score 0.5. That score is not a ROC AUC, and the genes it scores were drawn from the leading Challenge 1 nominations. In the follow-up, two nominated targets exceeded the Screen 1 targets on the immune checkpoint blockade objective; the source reports 61 nominations, 57 selected targets, 59 screen-2 genes and 50 perturbations passing quality control without reconciling them, so no success fraction follows. AssayBench-Loop replays 20 genome-wide CRISPR screens published after 2021, with each policy choosing 10 rounds of 100 genes. One random draw found 4.9% of each screen's hits on average (enrichment 1.04 over the random expectation). Language models used directly as the policy, with hit labels, reached enrichment of 1.39 to 4.81, BPMF 4.49, and the authors' handoffs from a language model to their AssayFormer up to 5.67, with the Gemini-3.1-Pro handoff finding 27.7% of hits. Many recovered hits are common-essential genes: 30.1% for that handoff, 18.6% for the random draw and 99.1% for a baseline that picks earlier screens' hits. Test screens were kept only if the best language model already passed a performance floor on them, and the paper does not say whether the models saw the screens' publications in training. The authors ran every row, and no uncertainty is printed. For two-gene combinations, NAIAD's replay of four viability screens counted how many of each screen's 200 strongest pairs the model ranked in its top 200 after four rounds: 143.0, 141.7, 99.7 and 150.0 when it acquired the pairs with the largest predicted effect, against 93.3, 70.3, 38.7 and 81.3 when it acquired pairs uniformly at random (means of three replicates). That count includes pairs the model predicted but never measured, and the uniform arm still uses the model's ranking, so it is not the yield of testing random pairs. The challenge's submissions scored close to a random ordering, while in both replays the leading model-guided policies scored above the random or uniform arm; the two differ in endpoint, candidate set, setting and who ran them, so their results cannot be compared directly. No source compares a model shortlist with random selection prospectively at the same budget. (ucc-research-source-challenge-supp-s2: Supplementary Table S2, media-3.xlsx, sheet 'Challenge 1 & 2', column D rows 2 to 24; ucc-research-source-challenge-paper: Abstract and Results Figure 5; ucpheno-20261022-source-edwards2026: Table 1, rows 'Random', 'GPT-5.6 Sol', 'BPMF [41]', 'Gemini-3.1-Pro → AssayFormer', 'GPT-5.6 Sol → AssayFormer' and 'Prior hit baseline', columns 'EF', 'FH (%)' and '%ess (%)'; Methods 4.2 and Appendix A (test-screen criteria); Supplementary Table 4, Base LLMs group, column 'EF'; pheno-20261022-qin2025-source-pmlr: Table 4, rows 'NAIAD MPE' and 'NAIAD Uniform')
- Relevance of CPPC original Challenge 2 prospective target ranking score to "Select perturbations for a defined cellular response": Assessed by CPPC original Challenge 2 prospective target ranking score (ucc-research-source-challenge-paper: Table S2 sheet Challenge 1 & 2 column D; Challenge setup and results; ucc-research-source-challenge-supp-s2: Table S2 sheet Challenge 1 & 2 column D; Challenge setup and results)
- Relevance of CPPC original Challenge 2 prospective target ranking score to "Select therapeutic targets for validation": Assessed by CPPC original Challenge 2 prospective target ranking score (ucc-research-source-challenge-paper: Table S2 sheet Challenge 1 & 2 column D; Challenge setup and results; ucc-research-source-challenge-supp-s2: Table S2 sheet Challenge 1 & 2 column D; Challenge setup and results)
- Source membership: ucc-research-protocol-cppc-challenge1-original: Part of Cancer Immunotherapy Data Science Challenge (Challenge setup and results; Supplementary Table S2)
- Source membership: ucc-research-protocol-cppc-challenge2-original: Part of Cancer Immunotherapy Data Science Challenge (Challenge setup and results; Supplementary Table S2)
- Source membership: ucc-research-protocol-cppc-challenge2-reimplemented-best: Part of Cancer Immunotherapy Data Science Challenge (Challenge setup and results; Supplementary Table S2)
- Source membership: ucc-research-protocol-cppc-prospective-selection: Part of Cancer Immunotherapy Data Science Challenge (Challenge setup and results; Supplementary Table S2)
- Source membership: ucc-research-protocol-cppc-response: Part of Cancer Immunotherapy Data Science Challenge (Challenge setup and results; Supplementary Table S2)
Models, benchmarks and other records
Configurations (36)
- CPPC original submission row 2 (PPI_Weighted)
- CPPC original submission row 3 (GWPS_K562 (H2M))
- CPPC original submission row 4 (14Features)
- CPPC original submission row 5 (scBERT)
- CPPC original submission row 6 (scVI)
- CPPC original submission row 7 (Expression_Conditioned)
- CPPC original submission row 8 (Exon)
- CPPC original submission row 9 (Gene2Vec)
- CPPC original submission row 10 (unnamed method)
- CPPC original submission row 11 (unnamed method)
- CPPC original submission row 12 (5Features)
- CPPC original submission row 13 (unnamed method)
- CPPC original submission row 14 (unnamed method)
- CPPC original submission row 15 (unnamed method)
- CPPC original submission row 16 (scETM)
- CPPC original submission row 17 (scTenifoldNet)
- CPPC original submission row 18 (GO_knn)
- CPPC original submission row 19 (LowExp)
- CPPC original submission row 20 (unnamed method)
- CPPC original submission row 21 (interv)
- CPPC original submission row 22 (GO-node2vec)
- CPPC original submission row 23 (Matrix_completion)
- CPPC original submission row 24 (GeneVector)
- CPPC reimplemented best model row 2 (PPI_Weighted)
- CPPC reimplemented best model row 3 (GWPS_K562 (H2M))
Benchmarks (1)
Protocols (3)
Evaluations (53)
- cppc table s2 c row 2
- cppc table s2 c row 3
- cppc table s2 c row 4
- cppc table s2 c row 5
- cppc table s2 c row 6
- cppc table s2 c row 7
- cppc table s2 c row 8
- cppc table s2 c row 9
- cppc table s2 c row 10
- cppc table s2 c row 11
- cppc table s2 c row 12
- cppc table s2 c row 13
- cppc table s2 c row 14
- cppc table s2 c row 15
- cppc table s2 c row 16
- cppc table s2 c row 17
- cppc table s2 c row 18
- cppc table s2 c row 19
- cppc table s2 c row 20
- cppc table s2 c row 21
- cppc table s2 d row 2
- cppc table s2 d row 3
- cppc table s2 d row 4
- cppc table s2 d row 5
- cppc table s2 d row 6
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
22 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Archive sha256 b79ab187f02aa1f265227a1003dce5a217d3b7480bd9bfe475b0f02596057cda Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Artifact member media-3.xlsx Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Artifact SHA-256 ed480039c8008fdfcedaa50a8248d32df6068a9230a18ab6953ada5535865105 Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Artifact url https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13228547/supplementaryFiles Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Excluded fields Contributor contact names/emails and personal submission links excluded from records. Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Retrieval method Europe PMC supplementaryFiles ZIP; extracted member media-3.xlsx; SHA-256 identifies member bytes; archive digest recorded separately Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Retrieved at 2026-09-30T21:40:24Z Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
Review method1 values
| CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Review note Primary-source table transcription and scope review; no independent reproduction or human scientific review. Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
| Review date 2026-09-30T21:43:56Z Source metadata | CPPC Table S2 original and reimplemented submission comparisons No field-specific location recorded Version: Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record Archive member: media-3.xlsx |
Sources and history
Release 2026-11-21-c8bea25e5abb · Record review: source checked
Primary research source retrieved and inspected for this coverage audit.
Technical metadata and extraction receipts
Stable ID: ucc-research-source-challenge-supp-s2
- url
- https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13228547/supplementaryFiles
- version
- Table S2 media-3.xlsx, 10.64898/2026.05.21.726863v1
- retrieved at
- 2026-09-30T21:40:24Z
- artifact url
- https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13228547/supplementaryFiles
- source locator
- Workbook media-3.xlsx; sheet Challenge 1 & 2; C2:D24 and J2:J24 (numeric cells only)
- review
- method: automated-source-review; reviewed at: 2026-09-30T21:43:56Z; note: Primary-source table transcription and scope review; no independent reproduction or human scientific review.; reviewer note: role: research coverage worker; reviewer: codex
- retrieval method
- Europe PMC supplementaryFiles ZIP; extracted member media-3.xlsx; SHA-256 identifies member bytes; archive digest recorded separately
- artifact member
- media-3.xlsx
- artifact sha256
- ed480039c8008fdfcedaa50a8248d32df6068a9230a18ab6953ada5535865105
- archive sha256
- b79ab187f02aa1f265227a1003dce5a217d3b7480bd9bfe475b0f02596057cda
- excluded fields
- Contributor contact names/emails and personal submission links excluded from records.