LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 2 models
• 4 methods
• 11 configurations
• 1 protocol
• 5 datasets
• 55 evaluations
• 55 results
• 8 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• Baseline: Select therapeutic targets for validation: Baseline Random and pathway-based selection (tgtval-20261009-source-roohani2025: Appendix (human pathway baseline) and Table 6 (pathways), Table 2 (Random); tgtval-20261009-source-gupta2025: baselines description and Table 3 (Random))
• control_result: tgtval-20261009-protocol-gupta2025-cumulative-hits-round5: Control result The source states that replacing true outcomes with randomly permuted labels has no impact on performance, and concludes that the language models tested fail to perform in-context experimental design. It performs two levels of randomisation: random measurement values, and random hit or not-hit feedback. In Table 1 the permuted variant scores at least as high as the agent on three of the five screens with each of the three backbones: Llama-3.1-8B, Qwen-2-7B, and Claude 3.5 Sonnet compared with the replicated row. (Abstract; section 5; Table 1 and its caption)
• cross_source_consistency: tgtval-20261009-protocol-gupta2025-cumulative-hits-round5: Cross source consistency The 'BDA (Reported Numbers)' row divided by the printed ground-truth hit counts reproduces the all-gene hit ratios of Roohani et al. 2025 Table 1 for Claude 3.5 Sonnet to the printed rounding: IL2 68.01/654 = 0.104 against 0.104, IFNG 87.4/920 = 0.095 against 0.095, Carnevale 39.6/943 = 0.042 against 0.042 and Sanchez 60.72/924 = 0.0657 against 0.066. This indicates that both sources score against the same screens and the same hit sets, and that the row converts the original results rather than reporting a new run. The conversion used values more precise than Roohani Table 1 prints: 0.066 x 924 = 60.98, while the row prints 60.72. Checked during extraction; no value was altered. (Gupta et al. 2025 Tables 1 and 2 against Roohani et al. 2025 Table 1)
• denominator: tgtval-20261009-protocol-gupta2025-cumulative-hits-round5: Denominator Printed ground-truth hit counts |C_gt| per screen: IL2 654, IFNG 920, Carnevale 943, Sanchez 924, Sanchez Down 924. The same five values are printed in Tables 1, 2 and 3. (Tables 1, 2 and 3, row 'Ground truth (| Cgt |)')
• Evidence summary: therapeutic target validation: Summary Three sources compare target-nomination methods against measured CRISPR screen outcomes, and all three are retrospective replays of published screens: a method picks genes and the screen's existing measurement is looked up. Non-hits are genes the screen assayed that did not meet its criterion; genes a screen never assayed are outside the candidate pool or, in AssayBench's headline metric, removed before scoring, so these results say nothing about an untested target. Roohani et al. 2025 (ICLR) were run by the developers of the agent they test, two of whom are authors of two of the screens, and they state that patent applications have been filed on the findings. They report the fraction of each screen's hits found after five rounds of 128 genes. With Claude 3.5 Sonnet the agent reached 0.095 on the interferon-gamma screen, 0.104 on interleukin-2, 0.130 on CAR-T proliferation, 0.326 on lysosomal choline recycling, 0.042 on T-cell inhibition resistance and 0.066 on tau level, against random selection at 0.037, 0.031, 0.033, 0.160, 0.036 and 0.034. Conventional baselines were close on some screens: Coreset reached 0.102 on interleukin-2 and 0.047 on T-cell inhibition resistance, above the agent's 0.042 there. Gupta et al. 2025 (Findings of EMNLP), independent of the agent's authors, replayed five of the same screens and report cumulative hits. Their re-run of the Claude 3.5 Sonnet agent found fewer hits than the original numbers on three of four shared screens, for example 31.6 against 60.72 on tau level. Giving the agent randomly permuted feedback changed little: the permuted variant scored at least as high on three of five screens with each of the three backbones tested, for example 51 against 44 on interferon-gamma with Llama-3.1-8B, and the authors conclude that the models they tested do not perform in-context experimental design. Gaussian process optimisation over the same embeddings reached 147.8 hits on interleukin-2 against the agent's 39.4 with Llama-3.1-8B, from 654 ground-truth hits; the source prints identical Gaussian process values under both backbones. De Brouwer et al. 2026, a preprint that is not peer reviewed, asked 16 systems to rank 100 genes for each of 571 benchmark entries, each a described screen. On the 334 test entries the best system was an evolved ensemble of language models at 0.1631 adjusted NDCG@100 and the best single model, Gemini 3 Pro, 0.1570, against 0.1334 for a baseline ranking genes by how often they are hits in training screens of the same phenotype and 0.2918 for a retrieval upper bound that uses the answer. Most systems scored lower on the 19 entries published after the models' training cutoff, the ensemble falling to 0.1088, which that source reads as partly prior exposure to the literature. None of these sources tests a prospective validation batch, a target's usefulness, dependency, tractability or therapeutic window, or whether modulating a target benefits patients. (tgtval-20261009-source-roohani2025: Table 1, sections 2 and 4.1, conflicts of interest; tgtval-20261009-source-gupta2025: Tables 1 and 2, abstract, section 5; tgtval-20261009-source-debrouwer2026: Tables 2 and 3, sections 2 to 5)
• Llama-3.1-8B: description: Description Language model in Meta's Llama 3 herd, which natively supports multilinguality, coding, reasoning and tool use. (qa-descriptions-20261011-b-source-llama3: abstract ('a herd of language models that natively support multilinguality, coding, reasoning, and tool usage'))
• Qwen-2-7B: description: Description Language model in the Qwen2 series, a suite of foundational and instruction-tuned language models from 0.5 to 72 billion parameters. (qa-descriptions-20261011-b-source-qwen2: abstract)
• Relevance of tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 to "Which targets should I test to change a defined disease-relevant phenotype?": Assessed by Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes (tgtval-20261009-source-gupta2025: Sections 4.1, 5 and 6; Tables 1 and 2; appendix B.1.1)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.