Evaluation

Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026)

Published comparison; transcribed, not reproduced.

Evaluation results

1 evaluation · 10 results. Different protocols are not a single leaderboard.

Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens · Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)

Sorted by Hit enrichment factor (hits found divided by hits expected under random selection, after 10 rounds of 100 genes; hallucinated genes and unfilled slots count in the denominator, valid genes outside the screen library do not; mean over the 20 AssayBench-Loop test screens) (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configuration	Hit enrichment factor (hits found divided by hits expected under random selection, after 10 rounds of 100 genes; hallucinated genes and unfilled slots count in the denominator, valid genes outside the screen library do not; mean over the 20 AssayBench-Loop test screens)	Normalized cumulative hits AUC (area under the fraction-of-hits against fraction-of-library-queried curve over 10 rounds, divided by an oracle's area over the same effective budget; mean over the 20 AssayBench-Loop test screens)	Proportion (shortfall: share of acquired genes outside the screen library, including hallucinated gene names; mean over the 20 AssayBench-Loop test screens)	Recall (fraction of the screen's hits acquired within 10 rounds of 100 genes (FH); mean over the 20 AssayBench-Loop test screens)	Annotation overlap ratio (mean pairwise Jaccard overlap of GO biological-process annotations within a batch, divided by that of an equal-size random batch from the screen library, averaged over batches; mean over the 20 AssayBench-Loop test screens)	Effective number of groups (EP-B: Hill number of order 1 at batch scope, rarefied to 30 annotated genes per batch, one random group per gene per draw, 400 Monte Carlo draws, averaged within each screen then across the 20 test screens; not comparable with EP-S or EP-D)	Effective number of groups (EP-D: Hill number of order 1 at dataset scope, rarefied to 6,000 annotated genes across the complete test set, one random group per gene per draw, 300 Monte Carlo draws; not comparable with EP-B or EP-S)	Effective number of groups (EP-S: Hill number of order 1 at screen scope, rarefied to 200 annotated genes per screen, one random group per gene per draw, 400 Monte Carlo draws, averaged across the 20 test screens; not comparable with EP-B or EP-D)	Proportion (share of acquired hits that are DepMap common-essential genes (1,827 genes); mean over the 20 AssayBench-Loop test screens)	Vendi ratio (Vendi score of each batch under a cosine kernel on GenePT embeddings, divided by batch size and averaged over batches; mean over the 20 AssayBench-Loop test screens)
Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)Evaluation	3.06	13.3	29.7%	16%	5.34	14.1	65.1	34.4	29%	51.2

All 10 result rows with coverage, uncertainty and sources

Filter evaluations

Tested model or methodProtocolDatasetEvaluation setupMetricEvidence origin

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configuration	Protocol and dataset	Finding	Evidence and details
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	3.06 Hit enrichment factor (hits found divided by hits expected under random selection, after 10 rounds of 100 genes; hallucinated genes and unfilled slots count in the denominator, valid genes outside the screen library do not; mean over the 20 AssayBench-Loop test screens) unitless · higher Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "EF"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	14.1 Effective number of groups (EP-B: Hill number of order 1 at batch scope, rarefied to 30 annotated genes per batch, one random group per gene per draw, 400 Monte Carlo draws, averaged within each screen then across the 20 test screens; not comparable with EP-S or EP-D) unitless · unknown Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored (Appendix H excludes batches or screens with fewer annotated genes than the rarefaction size; how many were excluded is not reported.)	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "EP-B"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	65.1 Effective number of groups (EP-D: Hill number of order 1 at dataset scope, rarefied to 6,000 annotated genes across the complete test set, one random group per gene per draw, 300 Monte Carlo draws; not comparable with EP-B or EP-S) unitless · unknown Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored (One value pooled over the 20 test screens (Appendix H), not a per-screen mean as the Table 1 caption says of all metrics. Appendix H also excludes sample sets with fewer annotated genes than the rarefaction size; how many is not reported.)	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "EP-D"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	34.4 Effective number of groups (EP-S: Hill number of order 1 at screen scope, rarefied to 200 annotated genes per screen, one random group per gene per draw, 400 Monte Carlo draws, averaged across the 20 test screens; not comparable with EP-B or EP-D) unitless · unknown Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored (Appendix H excludes batches or screens with fewer annotated genes than the rarefaction size; how many were excluded is not reported.)	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "EP-S"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	29% Proportion (share of acquired hits that are DepMap common-essential genes (1,827 genes); mean over the 20 AssayBench-Loop test screens) percent · unknown Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "Ess. (%)"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	16% Recall (fraction of the screen's hits acquired within 10 rounds of 100 genes (FH); mean over the 20 AssayBench-Loop test screens) percent · higher Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "Frac. hits (%)"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	13.3 Normalized cumulative hits AUC (area under the fraction-of-hits against fraction-of-library-queried curve over 10 rounds, divided by an oracle's area over the same effective budget; mean over the 20 AssayBench-Loop test screens) unitless · higher Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "nAUC (%)"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	5.34 Annotation overlap ratio (mean pairwise Jaccard overlap of GO biological-process annotations within a batch, divided by that of an equal-size random batch from the screen library, averaged over batches; mean over the 20 AssayBench-Loop test screens) unitless · unknown Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "Path. Ov."
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	29.7% Proportion (shortfall: share of acquired genes outside the screen library, including hallucinated gene names; mean over the 20 AssayBench-Loop test screens) percent · lower Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "Shortfall (%)"
Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)	Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)	51.2 Vendi ratio (Vendi score of each batch under a cosine kernel on GenePT embeddings, divided by batch size and averaged over batches; mean over the 20 AssayBench-Loop test screens) unitless · unknown Uncertainty: Not reported by the source: Table 1 and Supplementary Table 4 print means without spread. Coverage: 20 test screens scored	Independent external evaluation · Source checked Methods, coverage and source Claude Sonnet 4.6 acquisition policy on AssayBench-Loop test (Edwards et al. 2026) ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Aggregation: Not reported Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Supplementary Table 4, row "Claude Sonnet-4.6" (group 'Base LLMs'), column "Vendi (%)"

Source checking is not independent reproduction. Release 2026-10-23-5347695500ed.

Research readiness

0 of 4 readiness checks met. These checks assess whether the evidence supports a reproducible investigation; a source-checked score alone does not meet them.

Readiness checks, gaps and artifacts

Release 2026-10-23-5347695500ed · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
Metrics are recomputed from the saved predictions: not yet verified

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
Metrics are recomputed from the saved predictions: not yet verified
Sample annotations are recorded: not yet verified
Dependence between samples is assessed: not yet verified

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
A pinned run recipe exists: not yet verified
Compute requirements are estimated: not yet verified

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

No verified artifact manifest is linked to this exact record.
File checksums match the recorded files: not yet verified
Predictions are matched to the right samples: not yet verified
Score meaning and direction are confirmed: not yet verified
Independent validation data exist: not yet verified
Overlap with training data is checked: not yet verified

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Evaluation procedure

AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens

Configuration: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)
Protocol: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens
Dataset subset: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)

origin: Independent external evaluation
configuration: arXiv:2609.11877v1 as retrieved 2026-10-22
protocol id: ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100
dataset version: AssayBench-Loop test split, 20 screens (arXiv:2609.11877v1)
split: Temporal: train before 2021 (1,349 screens), validation 2021 (20), test after 2021 (20)
population: 20 genome-wide CRISPR test screens; candidate pool of about 20,000 genes
inputs: Screen description, genes acquired in earlier rounds with their hit labels, and the training screens
adaptation: Method-specific; training on the 1,349 training screens permitted
metric implementation: Methods 4.3 and Appendix H of arXiv:2609.11877v1
aggregation: Per-screen metric averaged equally over the 20 test screens
budget: 10 rounds x 100 genes (1,000 acquisitions per screen)

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split: Temporal: train before 2021 (1,349 screens), validation 2021 (20), test after 2021 (20)
Adaptation: Method-specific; training on the 1,349 training screens permitted
Scoring implementation: Methods 4.3 and Appendix H of arXiv:2609.11877v1

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

19 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-23-5347695500ed
Property and statement	Original source and location	Review and provenance
Comparison: adaptation Method-specific; training on the 1,349 training screens permitted Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.adaptation` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: aggregation Per-screen metric averaged equally over the 20 test screens Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.aggregation` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: budget 10 rounds x 100 genes (1,000 acquisitions per screen) Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.budget` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: dataset version AssayBench-Loop test split, 20 screens (arXiv:2609.11877v1) Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.dataset_version` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: inputs Screen description, genes acquired in earlier rounds with their hit labels, and the training screens Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.inputs` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: metric implementation Methods 4.3 and Appendix H of arXiv:2609.11877v1 Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.metric_implementation` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: population 20 genome-wide CRISPR test screens; candidate pool of about 20,000 genes Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.population` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: protocol id ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100 Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.protocol_id` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Comparison: split Temporal: train before 2021 (1,349 screens), validation 2021 (20), test after 2021 (20) Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.comparison.split` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact
Limitations 1 values Run by the AssayLoop authors on a third-party model with their own prompt. Context-only references	Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens Original source ↗ Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs) Version: arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed Retrieved: 2026-10-22T13:57:26Z	not individually reviewed No individual claim review recorded independent paper Audit details Field: `attributes.limitations` Source artifact SHA-256: `adcaaa5b7f9bd0e3bc1f91d94d26c8d4924c5c0b3064b6625c01ba92c6eaef28` Hash scope: pdftotext -layout text layer, parsed by data/omics/use-case-refresh-phenotype-20261022/extract/edwards2026assayloop.py Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-23-5347695500ed · Record review: source checked

1 source record and release history

Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens · Original source · arXiv:2609.11877 version 1, submitted 2026-09-10; not peer reviewed

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: ucpheno-20261022-eval-edwards2026-claude-sonnet-4-6

areas: cells-tissues
contexts: research
origin: independent_paper
protocol: ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100
version: arXiv:2609.11877v1 as retrieved 2026-10-22
comparison: protocol id: ucpheno-20261022-protocol-edwards2026-assaybench-loop-test-10x100; dataset version: AssayBench-Loop test split, 20 screens (arXiv:2609.11877v1); split: Temporal: train before 2021 (1,349 screens), validation 2021 (20), test after 2021 (20); population: 20 genome-wide CRISPR test screens; candidate pool of about 20,000 genes; inputs: Screen description, genes acquired in earlier rounds with their hit labels, and the training screens; adaptation: Method-specific; training on the 1,349 training screens permitted; metric implementation: Methods 4.3 and Appendix H of arXiv:2609.11877v1; aggregation: Per-screen metric averaged equally over the 20 test screens; budget: 10 rounds x 100 genes (1,000 acquisitions per screen)
source locator: Supplementary Table 4, row 'Claude Sonnet-4.6' (Base LLMs)
limitations: Run by the AssayLoop authors on a third-party model with their own prompt.
missing metadata: n runs: reason: unreported; note: The tables do not say how many runs or seeds each row averages; only Random is described as a single instantiation.

Related records

system: Claude Sonnet 4.6 acquisition policy (AssayBench-Loop)
assessment: AssayBench-Loop adaptive hit discovery: 10 rounds of 100 genes on 20 held-out CRISPR screens
data: AssayBench-Loop test split (20 genome-wide CRISPR screens published after 2021)
evaluation: Claude Sonnet 4.6 acquisition policy EF on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy EPB on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy EPD on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy EPS on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy ESS on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy FH on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy NAUC on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy PO on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy SF on AssayBench-Loop test
evaluation: Claude Sonnet 4.6 acquisition policy VENDI on AssayBench-Loop test

Suggest a correction