| Configuration: BDA, llama-3-1-8b backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 32.4 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBDA (Llama-3.1-8B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Llama-3.1-8B backbone' block, row 'BDA', column 'Carnevale' |
|---|
| Configuration: BDA, qwen-2-7b backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 27.2 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBDA (Qwen-2-7B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Qwen-2-7B backbone' block, row 'BDA', column 'Carnevale' |
|---|
| Configuration: BDA-Rand, claude-3-5-sonnet backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 42 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBDA-Rand (Claude 3.5 Sonnet backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Claude 3.5 Sonnet backbone' block, row 'BDA-Rand', column 'Carnevale' |
|---|
| Configuration: BDA-Rand, llama-3-1-8b backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 31.6 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBDA-Rand (Llama-3.1-8B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Llama-3.1-8B backbone' block, row 'BDA-Rand', column 'Carnevale' |
|---|
| Configuration: BDA-Rand, qwen-2-7b backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 29 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBDA-Rand (Qwen-2-7B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Qwen-2-7B backbone' block, row 'BDA-Rand', column 'Carnevale' |
|---|
| Configuration: BDA (Replicated), claude-3-5-sonnet backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 43.8 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBDA (Replicated) (Claude 3.5 Sonnet backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Claude 3.5 Sonnet backbone' block, row 'BDA (Replicated)', column 'Carnevale' |
|---|
| Configuration: BDA (Reported Numbers), claude-3-5-sonnet backbone (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 39.6 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Result quoted from another source · Source checkedMethods, coverage and sourceBDA (Reported Numbers) (Claude 3.5 Sonnet backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 1, 'Claude 3.5 Sonnet backbone' block, row 'BDA (Reported Numbers)', column 'Carnevale' |
|---|
| Configuration: GP over llama-3-1-8b embeddings (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 22.2 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceGP (Llama-3.1-8B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 2, 'Llama-3.1-8B backbone' block, row 'GP', column 'Carnevale' |
|---|
| Configuration: GP over qwen-2-7b embeddings (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 22.2 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceGP (Qwen-2-7B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 2, 'Qwen-2-7B backbone' block, row 'GP', column 'Carnevale' |
|---|
| Configuration: Linear UCB over llama-3-1-8b embeddings (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 38 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceLinear UCB (Llama-3.1-8B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 2, 'Llama-3.1-8B backbone' block, row 'Linear UCB', column 'Carnevale' |
|---|
| Configuration: Linear UCB over qwen-2-7b embeddings (Gupta et al. 2025) | Protocol: Independent replication of 1-gene perturbation design: cumulative hits after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 31 true-positive-count count · higher Uncertainty: Not reported by the source Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceLinear UCB (Qwen-2-7B backbone) on Carnevale (Gupta et al. 2025) tgtval-20261009-protocol-gupta2025-cumulative-hits-round5 Aggregation: Not reported LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? · Table 2, 'Qwen-2-7B backbone' block, row 'Linear UCB', column 'Carnevale' |
|---|
| Configuration: Badge acquisition function on an MLP surrogate (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.044 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBadge on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Badge', column 'All' under 'Carnev.' |
|---|
| Configuration: Badge acquisition function on an MLP surrogate (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.036 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceBadge on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Badge', column 'N/E' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3.5 Sonnet (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.042 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3.5 Sonnet on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3.5 Sonnet', column 'All' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3.5 Sonnet (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.044 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3.5 Sonnet on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3.5 Sonnet', column 'N/E' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3 Haiku (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.032 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3 Haiku on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Haiku', column 'All' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3 Haiku (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.034 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3 Haiku on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Haiku', column 'N/E' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3 Opus (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.043 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3 Opus on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Opus', column 'All' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3 Opus (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.043 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3 Opus on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Opus', column 'N/E' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3 Sonnet (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.041 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3 Sonnet on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Sonnet', column 'All' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude 3 Sonnet (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.042 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude 3 Sonnet on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude 3 Sonnet', column 'N/E' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude v1 (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.038 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude v1 on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude v1', column 'All' under 'Carnev.' |
|---|
| Configuration: BioDiscoveryAgent (No-Tools), Claude v1 (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.045 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude v1 on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Claude v1', column 'N/E' under 'Carnev.' |
|---|
| Configuration: Coreset acquisition function on an MLP surrogate (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.047 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceCoreset on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Coreset', column 'All' under 'Carnev.' |
|---|
| Configuration: Coreset acquisition function on an MLP surrogate (Roohani et al. 2025) | Protocol: BioDiscoveryAgent 1-gene perturbation design: hit ratio after 5 rounds of 128 genes Dataset: Carnevale et al. 2022 screen, T-cell resistance to tumour-microenvironment inhibitory signals | 0.038 recall fraction · higher Uncertainty: Not yet extracted: Appendix Table 7 prints one standard deviation over 10 runs for these values; not extracted in this pass. Coverage: Not reported scored / Not reported eligible | Independent external evaluation · Source checkedMethods, coverage and sourceCoreset on Carnev. (Roohani et al. 2025) tgtval-20261009-protocol-roohani2025-hitratio-round5 Aggregation: Not reported BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments · Table 1, row 'Coreset', column 'N/E' under 'Carnev.' |
|---|