BenchBench-Cloning protocol-development set
The 50 BenchBench-Cloning tasks in which an agent is given a desired construct and an inventory of starting materials and must submit a cloning protocol that a simulator executes to one isolated product.
Overview
An explanatory profile has not yet been reviewed for this record.
Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.
56 recorded evaluations, 56 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
56 evaluations · 56 results. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 13% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings) percent · higher Uncertainty: 95% CI 7.6 to 19.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Mythos 5.1 at high effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'mid' effort 'high', column 'Protocol dev. % [95% CI]' |
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, low reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 9% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings) percent · higher Uncertainty: 95% CI 5.0 to 13.8. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Mythos 5.1 at low effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' |
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 15.6% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 9.6 to 22.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 78 passed; Table E2 prints 500 recorded. The printed pass rate equals 78/500.) | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Mythos 5.1 at max effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Protocol dev. %' |
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 60% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 46.0 to 74.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Mythos 5.1 on BenchBench-Cloning protocol development, pass@10 benchbench-cloning-20261106-protocol-protocol-development-pass-at-10 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'pass@10 % [95% CI]' |
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 42.5% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 32.1 to 52.9. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Mythos 5.1 on BenchBench-Cloning protocol development, pass@5 benchbench-cloning-20261106-protocol-protocol-development-pass-at-5 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'pass@5 % [95% CI]' |
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 2 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings) count · higher Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts. Coverage: 50/50 tasks scored | Author-reported evaluation · Source checkedMethods, coverage and sourcebenchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'Solved all' |
| Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 30 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings) count · higher Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts. Coverage: 50/50 tasks scored | Author-reported evaluation · Source checkedMethods, coverage and sourcebenchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'Solved >=1' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 15.6% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings) percent · higher Uncertainty: 95% CI 10.4 to 21.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Opus 5.5 at high effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'mid' effort 'high', column 'Protocol dev. % [95% CI]' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, low reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 7.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings) percent · higher Uncertainty: 95% CI 3.8 to 11.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Opus 5.5 at low effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 19% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 12.0 to 27.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 95 passed; Table E2 prints 500 recorded. The printed pass rate equals 95/500.) | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Opus 5.5 at max effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'top' effort 'max', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Claude Opus 5.5', column 'Protocol dev. %' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 58% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 44.0 to 72.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Opus 5.5 on BenchBench-Cloning protocol development, pass@10 benchbench-cloning-20261106-protocol-protocol-development-pass-at-10 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'pass@10 % [95% CI]' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 44.6% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 33.4 to 56.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceClaude Opus 5.5 on BenchBench-Cloning protocol development, pass@5 benchbench-cloning-20261106-protocol-protocol-development-pass-at-5 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 3 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings) count · higher Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts. Coverage: 50/50 tasks scored | Author-reported evaluation · Source checkedMethods, coverage and sourcebenchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'Solved all' |
| Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 29 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings) count · higher Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts. Coverage: 50/50 tasks scored | Author-reported evaluation · Source checkedMethods, coverage and sourcebenchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'Solved >=1' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 4% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 1.0 to 8.4. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: 496/500 attempts scored (Table E1 prints 496 graded attempts and 20 passed; Table E2 prints 500 recorded and 4 unscored. Appendix B's budget is 500 attempts. The printed rate does not say which denominator was used: 20/500 and 20/496 = 4.032 both round to the printed value.) | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 3.8 Flash at high effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'top' effort 'high', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Gemini 3.8 Flash', column 'Protocol dev. %' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, low reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 7.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings) percent · higher Uncertainty: 95% CI 3.2 to 12.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 3.8 Flash at low effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, medium reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 5.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings) percent · higher Uncertainty: 95% CI 2.0 to 9.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 3.8 Flash at medium effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'mid' effort 'medium', column 'Protocol dev. % [95% CI]' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 16% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 6.0 to 26.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 3.8 Flash on BenchBench-Cloning protocol development, pass@10 benchbench-cloning-20261106-protocol-protocol-development-pass-at-10 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'pass@10 % [95% CI]' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 10.5% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 4.0 to 18.1. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 3.8 Flash on BenchBench-Cloning protocol development, pass@5 benchbench-cloning-20261106-protocol-protocol-development-pass-at-5 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'pass@5 % [95% CI]' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 0 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings) count · higher Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts. Coverage: 50/50 tasks scored | Author-reported evaluation · Source checkedMethods, coverage and sourcebenchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'Solved all' |
| Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 8 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings) count · higher Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts. Coverage: 50/50 tasks scored | Author-reported evaluation · Source checkedMethods, coverage and sourcebenchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'Solved >=1' |
| Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 15.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 8.4 to 23.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 76 passed; Table E2 prints 500 recorded. The printed pass rate equals 76/500.) | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 4 Argon at high effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'top' effort 'high', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Gemini 4 Argon', column 'Protocol dev. %' |
| Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, low reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 11% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings) percent · higher Uncertainty: 95% CI 7.4 to 15.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 4 Argon at low effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' |
| Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, medium reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 14.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings) percent · higher Uncertainty: 95% CI 9.0 to 20.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 4 Argon at medium effort on BenchBench-Cloning protocol-development pass rate benchbench-cloning-20261106-protocol-protocol-development-pass-rate Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'mid' effort 'medium', column 'Protocol dev. % [95% CI]' |
| Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, high reasoning effort | Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) | 42% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings) percent · higher Uncertainty: 95% CI 28.0 to 56.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4). Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceGemini 4 Argon on BenchBench-Cloning protocol development, pass@10 benchbench-cloning-20261106-protocol-protocol-development-pass-at-10 Aggregation: Not reported BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 4 Argon', column 'pass@10 % [95% CI]' |
Source checking is not independent reproduction. Release 2026-11-21-adc15377fab2.
Methods and evaluation design
Procedure, tasks and evaluated configurations
Recorded evaluations
Each evaluation records what was tested and under which conditions.
- Claude Mythos 5.1 at high effort on BenchBench-Cloning protocol-development pass rate
- Claude Mythos 5.1 at low effort on BenchBench-Cloning protocol-development pass rate
- Claude Mythos 5.1 at max effort on BenchBench-Cloning protocol-development pass rate
- Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@10
- Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@5
- Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- Claude Opus 5.5 at high effort on BenchBench-Cloning protocol-development pass rate
- Claude Opus 5.5 at low effort on BenchBench-Cloning protocol-development pass rate
- Claude Opus 5.5 at max effort on BenchBench-Cloning protocol-development pass rate
- Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@10
- Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@5
Baseline coverage
Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.
0 of 10 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.
Baseline status by linked protocol
- BenchBench-Cloning protocol development: pass@10 over 50 tasks · 0/2 roles measured
- BenchBench-Cloning protocol development: pass@5 over 50 tasks · 0/2 roles measured
- BenchBench-Cloning protocol development: single-attempt pass rate over 50 tasks · 0/2 roles measured
- BenchBench-Cloning protocol development: tasks solved in every one of the ten recorded attempts, out of 50 · 0/2 roles measured
- BenchBench-Cloning protocol development: tasks solved at least once in the ten recorded attempts, out of 50 · 0/2 roles measured
Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)
Coverage is derived from release 2026-11-21-adc15377fab2. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.
Run instructions
No runnable recipe has been reviewed for this benchmark. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
0 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|
No evidence rows match these filters. Choose another scope or clear the search.
Sources and history
Release 2026-11-21-adc15377fab2 · Record review: source checked
1 source record and release history
- BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Original source · arXiv v1, dated 23 October 2026
Technical metadata and record identifiers
Technical metadata for this record is tracked under stable ID benchbench-cloning-20261106-benchmark-protocol-development.
Source locator: Section 3.1; Table 1
Related records
- part of: BenchBench-Cloning
- uses data: BenchBench-Cloning protocol-development tasks (50)
- subject: BenchBench-Cloning: saturation
- subject: BenchBench-Cloning: data release
- subject: BenchBench-Cloning: task count
- used in: BenchBench-Cloning protocol-development tasks: Backbone end chemistry (6)
- used in: BenchBench-Cloning protocol-development tasks: Counterselection (4)
- used in: BenchBench-Cloning protocol-development tasks: Expression vectors and (2)
- used in: BenchBench-Cloning protocol-development tasks: GC-rich templates (4)
- used in: BenchBench-Cloning protocol-development tasks: Golden Gate / MoClo (5)
- used in: BenchBench-Cloning protocol-development tasks: Homology-directed and (7)
- used in: BenchBench-Cloning protocol-development tasks: Methylation state (11)
- used in: BenchBench-Cloning protocol-development tasks: Recombination, CRISPR (4)
- used in: BenchBench-Cloning protocol-development tasks: Repeats and tandem (7)
- used in: BenchBench-Cloning protocol-development tasks (50)
- assessment: Claude Mythos 5.1 at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: Claude Mythos 5.1 at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: Claude Mythos 5.1 at max effort on BenchBench-Cloning protocol-development pass rate
- assessment: Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@10
- assessment: Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@5
- assessment: Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: Claude Opus 5.5 at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: Claude Opus 5.5 at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: Claude Opus 5.5 at max effort on BenchBench-Cloning protocol-development pass rate
- assessment: Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@10
- assessment: Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@5
- assessment: Claude Opus 5.5 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: Claude Opus 5.5 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: Gemini 3.8 Flash at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: Gemini 3.8 Flash at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: Gemini 3.8 Flash at medium effort on BenchBench-Cloning protocol-development pass rate
- assessment: Gemini 3.8 Flash on BenchBench-Cloning protocol development, pass@10
- assessment: Gemini 3.8 Flash on BenchBench-Cloning protocol development, pass@5
- assessment: Gemini 3.8 Flash on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: Gemini 3.8 Flash on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: Gemini 4 Argon at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: Gemini 4 Argon at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: Gemini 4 Argon at medium effort on BenchBench-Cloning protocol-development pass rate
- assessment: Gemini 4 Argon on BenchBench-Cloning protocol development, pass@10
- assessment: Gemini 4 Argon on BenchBench-Cloning protocol development, pass@5
- assessment: Gemini 4 Argon on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: Gemini 4 Argon on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: GPT-6.1 Sol at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: GPT-6.1 Sol at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: GPT-6.1 Sol at max effort on BenchBench-Cloning protocol-development pass rate
- assessment: GPT-6.1 Sol on BenchBench-Cloning protocol development, pass@10
- assessment: GPT-6.1 Sol on BenchBench-Cloning protocol development, pass@5
- assessment: GPT-6.1 Sol on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: GPT-6.1 Sol on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: GPT-6 Astra at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: GPT-6 Astra at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: GPT-6 Astra at max effort on BenchBench-Cloning protocol-development pass rate
- assessment: GPT-6 Astra on BenchBench-Cloning protocol development, pass@10
- assessment: GPT-6 Astra on BenchBench-Cloning protocol development, pass@5
- assessment: GPT-6 Astra on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: GPT-6 Astra on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: Grok 4.7 at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: Grok 4.7 at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: Grok 4.7 on BenchBench-Cloning protocol development, pass@10
- assessment: Grok 4.7 on BenchBench-Cloning protocol development, pass@5
- assessment: Grok 4.7 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: Grok 4.7 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- assessment: Grok 4.7 at xhigh effort on BenchBench-Cloning protocol-development pass rate
- assessment: Kimi K3 at high effort on BenchBench-Cloning protocol-development pass rate
- assessment: Kimi K3 at low effort on BenchBench-Cloning protocol-development pass rate
- assessment: Kimi K3 at medium effort on BenchBench-Cloning protocol-development pass rate
- assessment: Kimi K3 on BenchBench-Cloning protocol development, pass@10
- assessment: Kimi K3 on BenchBench-Cloning protocol development, pass@5
- assessment: Kimi K3 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
- assessment: Kimi K3 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
- part of: BenchBench-Cloning cloning simulator
- part of: BenchBench-Cloning protocol development: pass@10 over 50 tasks
- part of: BenchBench-Cloning protocol development: pass@5 over 50 tasks
- part of: BenchBench-Cloning protocol development: single-attempt pass rate over 50 tasks
- part of: BenchBench-Cloning protocol development: tasks solved in every one of the ten recorded attempts, out of 50
- part of: BenchBench-Cloning protocol development: tasks solved at least once in the ten recorded attempts, out of 50