Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, high reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 15.5% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; mid reasoning tier, the middle of this system's three settings)percent · higher
Uncertainty: 95% CI 9.8 to 21.8. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 at high effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'mid' effort 'high', column 'Aggregate % [95% CI]' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, high reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 13% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings)percent · higher
Uncertainty: 95% CI 7.6 to 19.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 at high effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'mid' effort 'high', column 'Protocol dev. % [95% CI]' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, low reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 10.6% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; low reasoning tier, the lowest of this system's three settings)percent · higher
Uncertainty: 95% CI 6.3 to 15.4. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 at low effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'low' effort 'low', column 'Aggregate % [95% CI]' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, low reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 9% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)percent · higher
Uncertainty: 95% CI 5.0 to 13.8. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 at low effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 16% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 10.5 to 22.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 at max effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 15.6% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 9.6 to 22.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 78 passed; Table E2 prints 500 recorded. The printed pass rate equals 78/500.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 at max effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Protocol dev. %' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 60% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 46.0 to 74.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@10
benchbench-cloning-20261106-protocol-protocol-development-pass-at-10
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'pass@10 % [95% CI]' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 42.5% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 32.1 to 52.9. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 on BenchBench-Cloning protocol development, pass@5
benchbench-cloning-20261106-protocol-protocol-development-pass-at-5
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'pass@5 % [95% CI]' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 2 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)count · higher
Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.
Coverage: 50/50 tasks scored
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'Solved all' Configuration: Claude Mythos 5.1 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 30 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)count · higher
Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.
Coverage: 50/50 tasks scored
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Mythos 5.1 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Mythos 5.1', column 'Solved >=1' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, high reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 20% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; mid reasoning tier, the middle of this system's three settings)percent · higher
Uncertainty: 95% CI 14.2 to 26.3. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 at high effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'mid' effort 'high', column 'Aggregate % [95% CI]' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, high reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 15.6% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; mid reasoning tier, the middle of this system's three settings)percent · higher
Uncertainty: 95% CI 10.4 to 21.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 at high effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'mid' effort 'high', column 'Protocol dev. % [95% CI]' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, low reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 12.3% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; low reasoning tier, the lowest of this system's three settings)percent · higher
Uncertainty: 95% CI 7.7 to 17.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 at low effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'low' effort 'low', column 'Aggregate % [95% CI]' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, low reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 7.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)percent · higher
Uncertainty: 95% CI 3.8 to 11.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 at low effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 20.5% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 13.7 to 27.8. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 at max effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Opus 5.5', column 'Aggregate %' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 19% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 12.0 to 27.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 500/500 attempts scored (Table E1 prints 500 graded attempts and 95 passed; Table E2 prints 500 recorded. The printed pass rate equals 95/500.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 at max effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Claude Opus 5.5' tier 'top' effort 'max', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Claude Opus 5.5', column 'Protocol dev. %' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 58% Success rate (protocol-development pass@10: probability that at least one of 10 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 44.0 to 72.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@10
benchbench-cloning-20261106-protocol-protocol-development-pass-at-10
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'pass@10 % [95% CI]' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 44.6% Success rate (protocol-development pass@5: probability that at least one of 5 attempts passes; best-of-k, never comparable with single-attempt pass rate; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 33.4 to 56.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 on BenchBench-Cloning protocol development, pass@5
benchbench-cloning-20261106-protocol-protocol-development-pass-at-5
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 3 Count (protocol-development tasks solved in every one of ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)count · higher
Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.
Coverage: 50/50 tasks scored
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 on BenchBench-Cloning protocol development, tasks solved in every one of ten attempts
benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-every-attempt
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'Solved all' Configuration: Claude Opus 5.5 in Claude Code 2.1.289, max reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 29 Count (protocol-development tasks solved at least once in ten attempts, out of 50; a coverage count under repeated sampling, not a single-attempt performance measure; top reasoning tier, the highest of this system's three settings)count · higher
Uncertainty: Not reported by the source: Table G1 prints no interval for the solved-task counts.
Coverage: 50/50 tasks scored
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Claude Opus 5.5 on BenchBench-Cloning protocol development, tasks solved at least once in ten attempts
benchbench-cloning-20261106-protocol-protocol-development-tasks-solved-once
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table G1, row 'Claude Opus 5.5', column 'Solved >=1' Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 8.6% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 4.0 to 14.0. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Gemini 3.8 Flash at high effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'top' effort 'high', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Gemini 3.8 Flash', column 'Aggregate %' Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, high reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 4% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; top reasoning tier, the highest of this system's three settings)percent · higher
Uncertainty: 95% CI 1.0 to 8.4. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 496/500 attempts scored (Table E1 prints 496 graded attempts and 20 passed; Table E2 prints 500 recorded and 4 unscored. Appendix B's budget is 500 attempts. The printed rate does not say which denominator was used: 20/500 and 20/496 = 4.032 both round to the printed value.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Gemini 3.8 Flash at high effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'top' effort 'high', column 'Protocol dev. % [95% CI]'; the same cell is printed in Table 2, row 'Gemini 3.8 Flash', column 'Protocol dev. %' Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, low reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 10.8% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; low reasoning tier, the lowest of this system's three settings)percent · higher
Uncertainty: 95% CI 5.7 to 16.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Gemini 3.8 Flash at low effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'low' effort 'low', column 'Aggregate % [95% CI]' Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, low reasoning effort Benchmark: BenchBench-Cloning protocol-development set Dataset subset: BenchBench-Cloning protocol-development tasks (50) 7.2% Success rate (protocol-development pass rate at a single attempt (pass@1), simulator-graded and binary per attempt; low reasoning tier, the lowest of this system's three settings)percent · higher
Uncertainty: 95% CI 3.2 to 12.2. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: Not reported scored / Not reported eligible
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Gemini 3.8 Flash at low effort on BenchBench-Cloning protocol-development pass rate
benchbench-cloning-20261106-protocol-protocol-development-pass-rate
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'low' effort 'low', column 'Protocol dev. % [95% CI]' Configuration: Gemini 3.8 Flash in Gemini CLI 0.63.0, medium reasoning effort Benchmark: BenchBench-Cloning Dataset subset: BenchBench-Cloning protocol-development tasks (50) 9.2% Success rate (aggregate single-attempt score over all 65 tasks, task-weighted 50/65 protocol development and 15/65 construct design; mid reasoning tier, the middle of this system's three settings)percent · higher
Uncertainty: 95% CI 4.5 to 14.6. 95% confidence intervals from 20,000 bootstrap resamples of the tasks, retaining all attempts of each sampled task (Section 3.4).
Coverage: 575/575 attempts scored (500 protocol-development attempts plus 75 construct-design attempts; the weights are over tasks, 50/65 and 15/65.)
Author-reported evaluation · Source checked Evidence concern: excluded from comparisons Table D2's construct-design rate for Gemini 4 Argon at low effort, 31.9%, cannot be a mean of binary outcomes over the stated 15 tasks times 5 attempts: no integer numerator over 75 falls in its rounding window [23.8875, 23.9625], while all 23 other printed construct rates do. Its printed 95% interval [10.7, 52.0] is also byte-identical to the same system's top-effort interval. The two signals together indicate a transcription fault in the paper rather than a measured value, and the paper prints nothing that resolves it: the lane's aggregate is 15.8 under either 31.9 or 32.0, and Table G2's per-task solves are top-effort only. The printed value is kept as published and this entry withholds it from automatic comparison; the lane's other results are not covered.BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 4 Argon' tier 'low' effort 'low', column 'Construct design % [95% CI]'
Methods, coverage and source Gemini 3.8 Flash at medium effort on BenchBench-Cloning aggregate score
benchbench-cloning-20261106-protocol-aggregate-score
Aggregation: Not reported
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) · Table D2, row 'Gemini 3.8 Flash' tier 'mid' effort 'medium', column 'Aggregate % [95% CI]'