| Comparison: adaptation Single-turn agent attempt: the agent inspects the supplied files, uses the available tools, revises within the attempt and writes its submission to a designated answer file; no clarifying questions and no feedback between attempts Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.adaptation Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: aggregation Task-weighted mean of the protocol-development and construct-design pass rates, weights 50/65 and 15/65 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.aggregation Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: budget Modal sandbox, 2 vCPU and 6 GiB memory, network restricted to the model provider, two hours per attempt Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.budget Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: dataset version BenchBench-Cloning task corpus as published in arXiv:2610.33355v1 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.dataset_version Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: inputs Scientist's request, inventory of up to eight starting plasmids as GenBank or FASTA with annotations and material-property notes, documentation for the 42-operation protocol language, and analysis tools (Python, Biopython, Primer3-py, plotting libraries). Scientist's objective, sequence records and supporting materials; for six tasks also experimental records from the source study Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.inputs Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: metric implementation Weighted mean of the two task-set pass rates, weights 50/65 and 15/65 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.metric_implementation Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: population All 65 tasks: 50 protocol-development with ten attempts each and 15 construct-design with five Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.population Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: protocol id benchbench-cloning-20261106-protocol-aggregate-score Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.protocol_id Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Comparison: split Evaluation only; all 65 tasks scored Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.comparison.split Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|
| Coverage: eligible 575 Context-only references | BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development (arXiv:2610.33355v1) Original source ↗ Table D2, row 'Claude Mythos 5.1' tier 'top' effort 'max', column 'Aggregate % [95% CI]'; the same cell is printed in Table 2, row 'Claude Mythos 5.1', column 'Aggregate %' Version: arXiv v1, dated 23 October 2026 Retrieved: 2026-11-06T11:51:23Z | not individually reviewed No individual claim review recorded author reported Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: attributes.coverage.eligible Source artifact SHA-256: 6f5044b1bbbd81ad86b23cb536e2cae6428e0d9126f11546041355518febdc39 Hash scope: Hash scope not separately documented; inspect source record Archive member: artifacts/arxiv-2610.33355v1.layout.txt.gz |
|---|