Effective number of groups (EP-D: Hill number of order 1 at dataset scope, rarefied to 6,000 annotated genes across the complete test set, one random group per gene per draw, 300 Monte Carlo draws; not comparable with EP-B or EP-S) of AssayFormer, random embeddings, supervised only (AssayBench-Loop) on AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)
51.5 Effective number of groups (EP-D: Hill number of order 1 at dataset scope, rarefied to 6,000 annotated genes across the complete test set, one random group per gene per draw, 300 Monte Carlo draws; not comparable with EP-B or EP-S)
Methods
- Tested configuration
- AssayFormer, random embeddings, supervised only (AssayBench-Loop)
- Protocol
- AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens
- Dataset subset
- AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)
- Procedure
- AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens
- Evaluation
- AssayFormer, random embeddings, supervised only on AssayBench-Loop test (Edwards et al. 2026)
- Coverage
- 20 test screens scored (One value pooled over the 20 test screens (Appendix H), not a per-screen mean as the Table 1 caption says of all metrics. Appendix H also excludes sample sets with fewer annotated genes than the rarefaction size; how many is not reported.)
- Uncertainty
- Not reported by the source: Table 1 and Supplementary Table 4 print means without spread.
- Evidence
- Author-reported evaluation · source checkedBiology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Table 1, row "AssayFormer (random embs.)" (group 'AssayFormer (Ours)'), column "EP-D"
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Temporal: train before 2021 (1,349 screens), validation 2021 (20), test after 2021 (20)
- Adaptation
- Method-specific; training on the 1,349 training screens permitted
- Scoring implementation
- Methods 4.3 and Appendix H of arXiv:2609.11877v1
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 51.5 Individual claims | Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Table 1, row "AssayFormer (random embs.)" (group 'AssayFormer (Ours)'), column "EP-D" Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed | source checked ["source-hash-verification","pdf-text-parse","independent-cell-check","rendered-source-visual-check","scope-review"] · 2026-10-22 author reported Audit detailsExtracted by deterministic parse of the pdftotext -layout text layer of the pinned PDF (extract/edwards2026assayloop.py), asserting the row labels and group headings of Table 1 (26 rows) and Supplementary Table 4 (57 rows) in order and that every repeated printing of the same run is identical. Same value printed identically in Supplementary Table 4, row 'AssayFormer (random embs.)' (AssayFormer (Ours)); printed identically in Supplementary Table 4, row 'AssayFormer (Random)' (Ablations); stored once. Independent review: printed value, numeric value, locator, metric, qualifier, unit, direction and the linked configuration, protocol and test split match the source. Field: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Extraction artifact SHA-256: |
Sources and history
Release 2026-10-23-5347695500ed · Record review: source checked
1 source record and release history
- Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Original source · arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed
Technical metadata and extraction receipts
Stable ID: ucpheno-20261022-result-edwards2026-assayformer-random-emb-sft-epd
- metric
- effective-number-of-groups
- metric qualifier
- EP-D: Hill number of order 1 at dataset scope, rarefied to 6,000 annotated genes across the complete test set, one random group per gene per draw, 300 Monte Carlo draws; not comparable with EP-B or EP-S
- metric direction
- unknown
- unit
- unitless
- printed value
- 51.5
- numeric value
- 51.5
- source locator
- Table 1, row "AssayFormer (random embs.)" (group 'AssayFormer (Ours)'), column "EP-D"
- coverage
- scored: 20; unit: test screens; note: One value pooled over the 20 test screens (Appendix H), not a per-screen mean as the Table 1 caption says of all metrics. Appendix H also excludes sample sets with fewer annotated genes than the rarefaction size; how many is not reported.
- review
- method: source-hash-verification; pdf-text-parse; independent-cell-check; rendered-source-visual-check; scope-review; reviewer: claude; reviewer note: Claude (Opus 5.5) review agent in the evidence-pass workflow, independent of the extraction agent; no human review claimed; date: 2026-10-22; artifact sha256: adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28; retrieval url: https://arxiv.org/pdf/2609.11877v1; method note: Re-hashed the pinned PDF (SHA-256 matches the source record). Ran scripts/omics/batch.py check-values on the pdftotext -layout text layer: 470 in_row, 60 row_not_found (the last six Supplementary Table 4 rows sit beyond the script's 80-line caption window). A separate reviewer script (review/edwards2026assayloop-check.py) re-parsed Table 1 (26 rows) and Supplementary Table 4 (57 rows), resolved each locator to one cell by group, row label, the row above it for repeated labels, and column position, and matched 530 of 530 printed and numeric values, with no cell used twice. It also confirmed that all 26 Table 1 rows equal their Supplementary Table 4 printings in the 8 shared columns and that the four repeated Supplementary Table 4 runs are identical. A seeded sample of 36 results, plus every row of the Base LLM, Adaptive Experimental Design and Ablations blocks, was compared by eye with the rendered PDF pages 8 and 31, including column order. Metric definitions, arrows and aggregation were checked against the captions, Methods 4.3 and Appendix H.; note: Extracted by deterministic parse of the pdftotext -layout text layer of the pinned PDF (extract/edwards2026assayloop.py), asserting the row labels and group headings of Table 1 (26 rows) and Supplementary Table 4 (57 rows) in order and that every repeated printing of the same run is identical. Same value printed identically in Supplementary Table 4, row 'AssayFormer (random embs.)' (AssayFormer (Ours)); printed identically in Supplementary Table 4, row 'AssayFormer (Random)' (Ablations); stored once. Independent review: printed value, numeric value, locator, metric, qualifier, unit, direction and the linked configuration, protocol and test split match the source.
- missing metadata
- uncertainty: reason: unreported; note: Table 1 and Supplementary Table 4 print means without spread.
- unit detail
- effective number of Reactome level-2 pathway groups (186 in the vocabulary)