Proportion (shortfall: share of acquired genes outside the screen library, including hallucinated gene names; mean over the 20 AssayBench-Loop test screens) of LLMNN (AssayBench-Loop) on AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)
0.1% Proportion (shortfall: share of acquired genes outside the screen library, including hallucinated gene names; mean over the 20 AssayBench-Loop test screens)
Methods
- Tested configuration
- LLMNN (AssayBench-Loop)
- Protocol
- AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens
- Dataset subset
- AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)
- Procedure
- AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens
- Evaluation
- LLMNN on AssayBench-Loop test (Edwards et al. 2026)
- Coverage
- 20 test screens scored
- Uncertainty
- Not reported by the source: Table 1 and Supplementary Table 4 print means without spread.
- Evidence
- Independent external evaluation · source checkedBiology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Table 1, row "LLMNN [17]" (group 'Agent Harnesses'), column "SF (%)(↓)"
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Temporal: train before 2021 (1,349 screens), validation 2021 (20), test after 2021 (20)
- Adaptation
- Method-specific; training on the 1,349 training screens permitted
- Scoring implementation
- Methods 4.3 and Appendix H of arXiv:2609.11877v1
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 0.1 Individual claims | Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Table 1, row "LLMNN [17]" (group 'Agent Harnesses'), column "SF (%)(↓)" Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed | source checked ["source-hash-verification","pdf-text-parse","independent-cell-check","rendered-source-visual-check","scope-review"] · 2026-10-22 independent paper Audit detailsExtracted by deterministic parse of the pdftotext -layout text layer of the pinned PDF (extract/edwards2026assayloop.py), asserting the row labels and group headings of Table 1 (26 rows) and Supplementary Table 4 (57 rows) in order and that every repeated printing of the same run is identical. Same value printed identically in Supplementary Table 4, row 'LLMNN [17]' (Agent Harnesses); stored once. Independent review: printed value, numeric value, locator, metric, qualifier, unit, direction and the linked configuration, protocol and test split match the source. Field: Source artifact SHA-256: Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Extraction artifact SHA-256: |
Sources and history
Release 2026-10-23-5347695500ed · Record review: source checked
1 source record and release history
- Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Original source · arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed
Technical metadata and extraction receipts
Stable ID: ucpheno-20261022-result-edwards2026-llmnn-sf
- metric
- proportion
- metric qualifier
- shortfall: share of acquired genes outside the screen library, including hallucinated gene names; mean over the 20 AssayBench-Loop test screens
- metric direction
- lower
- unit
- percent
- printed value
- 0.1
- numeric value
- 0.1
- source locator
- Table 1, row "LLMNN [17]" (group 'Agent Harnesses'), column "SF (%)(↓)"
- coverage
- scored: 20; unit: test screens
- review
- method: source-hash-verification; pdf-text-parse; independent-cell-check; rendered-source-visual-check; scope-review; reviewer: claude; reviewer note: Claude (Opus 5.5) review agent in the evidence-pass workflow, independent of the extraction agent; no human review claimed; date: 2026-10-22; artifact sha256: adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28; retrieval url: https://arxiv.org/pdf/2609.11877v1; method note: Re-hashed the pinned PDF (SHA-256 matches the source record). Ran scripts/omics/batch.py check-values on the pdftotext -layout text layer: 470 in_row, 60 row_not_found (the last six Supplementary Table 4 rows sit beyond the script's 80-line caption window). A separate reviewer script (review/edwards2026assayloop-check.py) re-parsed Table 1 (26 rows) and Supplementary Table 4 (57 rows), resolved each locator to one cell by group, row label, the row above it for repeated labels, and column position, and matched 530 of 530 printed and numeric values, with no cell used twice. It also confirmed that all 26 Table 1 rows equal their Supplementary Table 4 printings in the 8 shared columns and that the four repeated Supplementary Table 4 runs are identical. A seeded sample of 36 results, plus every row of the Base LLM, Adaptive Experimental Design and Ablations blocks, was compared by eye with the rendered PDF pages 8 and 31, including column order. Metric definitions, arrows and aggregation were checked against the captions, Methods 4.3 and Appendix H.; note: Extracted by deterministic parse of the pdftotext -layout text layer of the pinned PDF (extract/edwards2026assayloop.py), asserting the row labels and group headings of Table 1 (26 rows) and Supplementary Table 4 (57 rows) in order and that every repeated printing of the same run is identical. Same value printed identically in Supplementary Table 4, row 'LLMNN [17]' (Agent Harnesses); stored once. Independent review: printed value, numeric value, locator, metric, qualifier, unit, direction and the linked configuration, protocol and test split match the source.
- missing metadata
- uncertainty: reason: unreported; note: Table 1 and Supplementary Table 4 print means without spread.
- unit detail
- acquired genes