BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 9 models
• 11 methods
• 20 configurations
• 1 protocol
• 6 datasets
• 120 evaluations
• 240 results
• 14 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• Baseline: Select perturbations for a defined cellular response: Baseline Random selection (tgtval-20261009-source-roohani2025: Table 2 (Random baseline by round) and baselines description)
• Baseline: Select therapeutic targets for validation: Baseline Random and pathway-based selection (tgtval-20261009-source-roohani2025: Appendix (human pathway baseline) and Table 6 (pathways), Table 2 (Random); tgtval-20261009-source-gupta2025: baselines description and Table 3 (Random))
• Claude 3 Haiku: description: Description Large multimodal model with vision capabilities, the fastest and least expensive of Anthropic's Claude 3 family. (qa-descriptions-20261011-b-source-claude3: model card abstract, p.1 ('Claude 3 Haiku, our fastest and least expensive model'))
• Claude 3 Opus: description: Description Large multimodal model with vision capabilities, the most capable of Anthropic's Claude 3 family. (qa-descriptions-20261011-b-source-claude3: model card abstract, p.1 ('Claude 3 Opus, our most capable offering'; 'All new models have vision capabilities'))
• Claude 3 Sonnet: description: Description Large multimodal model with vision capabilities in Anthropic's Claude 3 family, positioned between Opus and Haiku in skills and speed. (qa-descriptions-20261011-b-source-claude3: model card abstract, p.1 ('Claude 3 Sonnet, which provides a combination of skills and speed'))
• Claude 3.5 Sonnet: description: Description Large multimodal model from Anthropic, an evolution of the Claude 3 family with improved coding and visual processing. (qa-descriptions-20261011-b-source-claude3: Claude 3.5 Sonnet Model Card Addendum, section 1 Introduction ('Claude 3.5 Sonnet offers improved capabilities, including better coding and visual processing. Since it is an evolution of the Claude 3 model family'))
• cross_source_consistency: tgtval-20261009-protocol-gupta2025-cumulative-hits-round5: Cross source consistency The 'BDA (Reported Numbers)' row divided by the printed ground-truth hit counts reproduces the all-gene hit ratios of Roohani et al. 2025 Table 1 for Claude 3.5 Sonnet to the printed rounding: IL2 68.01/654 = 0.104 against 0.104, IFNG 87.4/920 = 0.095 against 0.095, Carnevale 39.6/943 = 0.042 against 0.042 and Sanchez 60.72/924 = 0.0657 against 0.066. This indicates that both sources score against the same screens and the same hit sets, and that the row converts the original results rather than reporting a new run. The conversion used values more precise than Roohani Table 1 prints: 0.066 x 924 = 60.98, while the row prints 60.72. Checked during extraction; no value was altered. (Gupta et al. 2025 Tables 1 and 2 against Roohani et al. 2025 Table 1)
• Evidence summary: therapeutic target validation: Summary Three sources compare target-nomination methods against measured CRISPR screen outcomes, and all three are retrospective replays of published screens: a method picks genes and the screen's existing measurement is looked up. Non-hits are genes the screen assayed that did not meet its criterion; genes a screen never assayed are outside the candidate pool or, in AssayBench's headline metric, removed before scoring, so these results say nothing about an untested target. Roohani et al. 2025 (ICLR) were run by the developers of the agent they test, two of whom are authors of two of the screens, and they state that patent applications have been filed on the findings. They report the fraction of each screen's hits found after five rounds of 128 genes. With Claude 3.5 Sonnet the agent reached 0.095 on the interferon-gamma screen, 0.104 on interleukin-2, 0.130 on CAR-T proliferation, 0.326 on lysosomal choline recycling, 0.042 on T-cell inhibition resistance and 0.066 on tau level, against random selection at 0.037, 0.031, 0.033, 0.160, 0.036 and 0.034. Conventional baselines were close on some screens: Coreset reached 0.102 on interleukin-2 and 0.047 on T-cell inhibition resistance, above the agent's 0.042 there. Gupta et al. 2025 (Findings of EMNLP), independent of the agent's authors, replayed five of the same screens and report cumulative hits. Their re-run of the Claude 3.5 Sonnet agent found fewer hits than the original numbers on three of four shared screens, for example 31.6 against 60.72 on tau level. Giving the agent randomly permuted feedback changed little: the permuted variant scored at least as high on three of five screens with each of the three backbones tested, for example 51 against 44 on interferon-gamma with Llama-3.1-8B, and the authors conclude that the models they tested do not perform in-context experimental design. Gaussian process optimisation over the same embeddings reached 147.8 hits on interleukin-2 against the agent's 39.4 with Llama-3.1-8B, from 654 ground-truth hits; the source prints identical Gaussian process values under both backbones. De Brouwer et al. 2026, a preprint that is not peer reviewed, asked 16 systems to rank 100 genes for each of 571 benchmark entries, each a described screen. On the 334 test entries the best system was an evolved ensemble of language models at 0.1631 adjusted NDCG@100 and the best single model, Gemini 3 Pro, 0.1570, against 0.1334 for a baseline ranking genes by how often they are hits in training screens of the same phenotype and 0.2918 for a retrieval upper bound that uses the answer. Most systems scored lower on the 19 entries published after the models' training cutoff, the ensemble falling to 0.1088, which that source reads as partly prior exposure to the literature. None of these sources tests a prospective validation batch, a target's usefulness, dependency, tractability or therapeutic window, or whether modulating a target benefits patients. (tgtval-20261009-source-roohani2025: Table 1, sections 2 and 4.1, conflicts of interest; tgtval-20261009-source-gupta2025: Tables 1 and 2, abstract, section 5; tgtval-20261009-source-debrouwer2026: Tables 2 and 3, sections 2 to 5)
• GPT-4o: description: Description Autoregressive omni model from OpenAI, trained end to end across text, vision and audio, that accepts text, audio, image and video input and generates text, audio and image output. (qa-descriptions-20261011-b-source-gpt-4o: abstract)
• metric_definition: tgtval-20261009-protocol-roohani2025-hitratio-round5: Metric definition A hit is a gene whose perturbation produces a phenotypic response above a threshold tau, f(g) > tau. The hit ratio after T rounds is the fraction of all such genes that the method selected, which the source describes as similar to recall. (Section 2 (problem setting), equations for hit ratio)
• o1-mini: description: Description Model in OpenAI's o1 series, trained with large-scale reinforcement learning to reason using a chain of thought before answering. (qa-descriptions-20261011-b-source-o1: abstract ('The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought'))
• o1-preview: description: Description Model in OpenAI's o1 series, trained with large-scale reinforcement learning to reason using a chain of thought before answering. (qa-descriptions-20261011-b-source-o1: abstract ('The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought'))
• Relevance of tgtval-20261009-protocol-roohani2025-hitratio-round5 to "Which targets should I test to change a defined disease-relevant phenotype?": Assessed by BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes (tgtval-20261009-source-roohani2025: Sections 2 and 4.1; Table 1; appendix Table 7)
• scoring_population: tgtval-20261009-protocol-roohani2025-hitratio-round5: Scoring population The 'N/E' columns count only non-essential genes among the hits. The source argues essential genes are likely to be detected as hits under many phenotypes, so the non-essential subset is the harder and more biologically useful task. The source does not print which gene list defines essentiality. (Section 5 (results, non-essential genes) and Table 1 caption)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.