| Comparison: adaptation Single-turn agent attempt: the agent inspects the supplied files, uses the available tools, revises within the attempt and writes its submission to a designated answer file; no clarifying questions and no feedback between attempts Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.adaptation Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: aggregation pass@5 per task, then unweighted mean over the 50 tasks Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.aggregation Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: budget Modal sandbox, 2 vCPU and 6 GiB memory, network restricted to the model provider, two hours per attempt Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.budget Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: dataset version BenchBench-Cloning task corpus as published in arXiv:2610.33355v1 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.dataset_version Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: inputs Scientist's request, inventory of up to eight starting plasmids as GenBank or FASTA with annotations and material-property notes, documentation for the 42-operation protocol language, and analysis tools (Python, Biopython, Primer3-py, plotting libraries) Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.inputs Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: metric implementation pass@5: probability that at least one of 5 attempts passes the simulator Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.metric_implementation Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: population 50 protocol-development tasks, ten recorded attempts each Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.population Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: protocol id benchbench-cloning-20261106-protocol-protocol-development-pass-at-5 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.protocol_id Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: split Evaluation only; all 50 tasks scored Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.split Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Coverage: eligible 500 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table G1, row 'Claude Opus 5.5', column 'pass@5 % [95% CI]' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.coverage.eligible Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|