Result

0.072 recall

roohani2025-claude-3-haiku-schmidt2022-il2 recall

Tested configuration: BioDiscoveryAgent (No-Tools), Claude 3 Haiku (Roohani et al. 2025)
Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes
Dataset: Schmidt et al. 2022 screen, interleukin-2 production in primary human T cells
Procedure: tgtval-20261009-protocol-roohani2025-hitratio-round5
Evaluation: Claude 3 Haiku on Schmidt2 (Roohani et al. 2025)
Coverage: Not reported scored / Not reported eligible
Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass.
Evidence: Author-reported evaluation · source checkedBioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Haiku', column 'All' under 'Schmidt2'

A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.

Reproduction

Split: No training split; 5 sequential selection rounds
Adaptation: Selection only; no model fitting for the agent rows
Scoring implementation: Not reported

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-10-10-7fcc3e48a123
Property and statement	Original source and location	Review and provenance
Reported result 0.072 Individual claims	BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments Original source ↗ Table 1, row 'Claude 3 Haiku', column 'All' under 'Schmidt2' Version: arXiv:2405.17631 version 3, updated 2025-03-09; published as a conference paper at ICLR 2025 Retrieved: 2026-10-09T20:51:27Z	source checked ["source-hash-verification","pdf-text-parse","independent-cell-check"] · 2026-10-09T21:22:54Z author reported Audit details Extracted by deterministic parse of the PDF text layer (pdftotext -layout) of Table 1, asserting the two header rows, the two section headers, the 20 row labels and 12 values per row. Independent review 2026-10-09: value, metric, unit, direction, locator and configuration, protocol and dataset identity match the source. Field: `attributes.printed_value` Source artifact SHA-256: `dd94d80ec75c9bb84ec3989c910a313ef1a7aec57c772d3dd35ecadd250132a7` Hash scope: pdftotext -layout text layer, parsed by extract/extract_target_validation.py Inspected artifact Extraction artifact SHA-256: `dd94d80ec75c9bb84ec3989c910a313ef1a7aec57c772d3dd35ecadd250132a7` Extraction artifact

All evidence (JSONL) (gzip)How to read the evidence

Sources and history

Release 2026-10-10-7fcc3e48a123 · Record review: source checked

1 source records and release history

BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Original source · arXiv:2405.17631 version 3, updated 2025-03-09; published as a conference paper at ICLR 2025

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: tgtval-20261009-result-roohani2025-claude-3-haiku-schmidt2022-il2-all

metric: recall
metric qualifier: hit ratio after round 5, all genes; mean of 10 runs
metric direction: higher
unit: fraction
printed value: 0.072
numeric value: 0.072
source locator: Table 1, row 'Claude 3 Haiku', column 'All' under 'Schmidt2'
review: method: source-hash-verification; pdf-text-parse; independent-cell-check; method note: Re-downloaded the PDF and matched its SHA-256. Converted with pdftotext -layout and parsed Table 1 with a separate script written for this review (the extractor's script was not imported or run), asserting the screen header, the 'All N/E' header, the two section headings and the 20 row labels in order. Checked printed and numeric value, metric, qualifier (round, scoring population), unit, direction, locator, and the linked configuration, protocol and dataset (screen mapped by the caption: Schmidt1 is IFN-gamma, Schmidt2 is IL-2).; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-09T21:22:54Z; artifact sha256: dd94d80ec75c9bb84ec3989c910a313ef1a7aec57c772d3dd35ecadd250132a7; retrieval url: https://arxiv.org/pdf/2405.17631v3; note: Extracted by deterministic parse of the PDF text layer (pdftotext -layout) of Table 1, asserting the two header rows, the two section headers, the 20 row labels and 12 values per row. Independent review 2026-10-09: value, metric, unit, direction, locator and configuration, protocol and dataset identity match the source.
missing metadata: uncertainty: reason: unextracted; note: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass.
scope note: Scoring population: all genes.

Related records

evaluation: Claude 3 Haiku on Schmidt2 (Roohani et al. 2025)

Suggest a correction