rewirebio.iobenchmarks
Benchmark

BenchBench-Cloning protocol-development set

The 50 BenchBench-Cloning tasks in which an agent is given a desired construct and an inventory of starting materials and must submit a cloning protocol that a simulator executes to one isolated product.

56 evaluations · 56 results

Overview

An explanatory profile has not yet been reviewed for this record.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

56 recorded evaluations, 56 metric rows. A comparison chart has not yet been validated for these results. The table retains the individual findings and their sources.

View coverage and remaining gaps across all benchmarks

Results

Results are available, but no reviewed comparison panel is linked in this release.

All evaluations

56 evaluations · 56 results. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
13% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings)
percent · higher

Uncertainty: 95% CI 7.6 to 19.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 at high effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'mid' effort 'high', column 'Protocol dev. % [95% CI]'
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, low reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
9% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)
percent · higher

Uncertainty: 95% CI 5.0 to 13.8. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 at low effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]'
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
15.6% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 9.6 to 22.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 78 passed; Table E2 prints 500 recorded. The printed pass rate equals 78/500.)

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 at max effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Protocol dev. %'
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
60% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 46.0 to 74.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@10

benchbench-cloning-20261106-protocol-protocol-development-pass-at-10

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'pass@10 % [95% CI]'
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
42.5% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 32.1 to 52.9. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@5

benchbench-cloning-20261106-protocol-protocol-development-pass-at-5

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'pass@5 % [95% CI]'
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
2 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)
count · higher

Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.

Coverage: 50/50 tasks scored

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts

benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'Solved all'
Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
30 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)
count · higher

Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.

Coverage: 50/50 tasks scored

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts

benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'Solved >=1'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
15.6% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings)
percent · higher

Uncertainty: 95% CI 10.4 to 21.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 at high effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'mid' effort 'high', column 'Protocol dev. % [95% CI]'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, low reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
7.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)
percent · higher

Uncertainty: 95% CI 3.8 to 11.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 at low effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
19% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 12.0 to 27.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 95 passed; Table E2 prints 500 recorded. The printed pass rate equals 95/500.)

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 at max effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'top' effort 'max', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Claude Opus 5.5', column 'Protocol dev. %'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
58% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 44.0 to 72.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@10

benchbench-cloning-20261106-protocol-protocol-development-pass-at-10

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'pass@10 % [95% CI]'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
44.6% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 33.4 to 56.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@5

benchbench-cloning-20261106-protocol-protocol-development-pass-at-5

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
3 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)
count · higher

Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.

Coverage: 50/50 tasks scored

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts

benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'Solved all'
Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
29 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)
count · higher

Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.

Coverage: 50/50 tasks scored

Author-reported evaluation · Source checked
Methods, coverage and source

Claude Opus 5.5 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts

benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'Solved >=1'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
4% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 1.0 to 8.4. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: 496/500 attempts scored (Table E1 prints 496 graded attempts and 20 passed; Table E2 prints 500 recorded and 4 unscored. Appendix B's budget is 500 attempts. The printed rate does not say which denominator was used: 20/500 and 20/496 = 4.032 both round to the printed value.)

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash at high effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'top' effort 'high', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Gemini 3.8 Flash', column 'Protocol dev. %'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, low reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
7.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)
percent · higher

Uncertainty: 95% CI 3.2 to 12.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash at low effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, medium reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
5.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings)
percent · higher

Uncertainty: 95% CI 2.0 to 9.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash at medium effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'mid' effort 'medium', column 'Protocol dev. % [95% CI]'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
16% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 6.0 to 26.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash on BenchBench-Cloning protocol development, pass@10

benchbench-cloning-20261106-protocol-protocol-development-pass-at-10

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'pass@10 % [95% CI]'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
10.5% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 4.0 to 18.1. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash on BenchBench-Cloning protocol development, pass@5

benchbench-cloning-20261106-protocol-protocol-development-pass-at-5

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'pass@5 % [95% CI]'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
0 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)
count · higher

Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.

Coverage: 50/50 tasks scored

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts

benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'Solved all'
Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
8 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)
count · higher

Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.

Coverage: 50/50 tasks scored

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 3.8 Flash on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts

benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 3.8 Flash', column 'Solved >=1'
Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
15.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 8.4 to 23.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 76 passed; Table E2 prints 500 recorded. The printed pass rate equals 76/500.)

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 4 Argon at high effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'top' effort 'high', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Gemini 4 Argon', column 'Protocol dev. %'
Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, low reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
11% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)
percent · higher

Uncertainty: 95% CI 7.4 to 15.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 4 Argon at low effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]'
Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, medium reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
14.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings)
percent · higher

Uncertainty: 95% CI 9.0 to 20.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 4 Argon at medium effort on BenchBench-Cloning protocol-development pass rate

benchbench-cloning-20261106-protocol-protocol-development-pass-rate

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'mid' effort 'medium', column 'Protocol dev. % [95% CI]'
Configuration: Gemini 4 Argon in Gemini CLI 0.63.0, high reasoning effortBenchmark: BenchBench-Cloning protocol-development set
Dataset subset: BenchBench-Cloning protocol-development tasks (50)
42% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)
percent · higher

Uncertainty: 95% CI 28.0 to 56.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Gemini 4 Argon on BenchBench-Cloning protocol development, pass@10

benchbench-cloning-20261106-protocol-protocol-development-pass-at-10

Aggregation: Not reported

BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Gemini 4 Argon', column 'pass@10 % [95% CI]'

Source checking is not independent reproduction. Release 2026-11-21-adc15377fab2.

Methods and evaluation design

Procedure, tasks and evaluated configurations

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Baseline coverage

Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.

0 of 10 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.

Baseline status by linked protocol

Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)

Coverage is derived from release 2026-11-21-adc15377fab2. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.

Run instructions

No runnable recipe has been reviewed for this benchmark. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

0 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-adc15377fab2
Property and statementOriginal source and locationReview and provenance

No evidence rows match these filters. Choose another scope or clear the search.

Sources and history

Release 2026-11-21-adc15377fab2 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and record identifiers

Technical metadata for this record is tracked under stable ID benchbench-cloning-20261106-benchmark-protocol-development.

Source locator: Section 3.1; Table 1

Related records

Suggest a correction