Result

Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted) of AIDE v6.3.3, GPT-4.1 backend (BioML-bench) on BioML-bench v0.1-alpha single cell omics tasks (5 tasks)

60.67 ± 11.16% Leaderboard percentile (mean over tasks in the domain; non-penalized: runs without a valid submission omitted)

Methods

Tested configuration: AIDE v6.3.3, GPT-4.1 backend (BioML-bench)
Protocol: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks
Dataset subset: BioML-bench v0.1-alpha single cell omics tasks (5 tasks)
Procedure: BioML-bench single cell omics domain: agent leaderboard percentile over 5 tasks
Evaluation: AIDE on BioML-bench single cell omics (non-penalized)
Coverage: Not reported scored / Not reported eligible
Uncertainty: ± 11.16 (type not stated). Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM.
Evidence: Independent external evaluation · source checkedBioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'AIDE', column 'Leaderboard Percentile'

A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.

Reproduction

Split: Task-specific splits defined by BioML-bench's own preparation scripts
Adaptation: Autonomous agent builds and trains its own ML pipeline; LLM backend: GPT-4.1
Scoring implementation: Task's original metric mapped to the task's public leaderboard percentile

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-11-02-a7f03c9e55a5
Property and statement	Original source and location	Review and provenance
Reported result 60.67 ± 11.16 Individual claims	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'AIDE', column 'Leaderboard Percentile' Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	source checked ["source-hash-verification","pdf-text-parse","independent-cell-check","scope-review"] · 2026-10-13 independent paper Audit details Extraction: parsed from the pdftotext -layout text of the pinned PDF by extract/extract_biomlbench.py. Independent review 2026-10-13: matched. The spread stays unresolved_spread: the Table 4 caption does not name it, although every cell is arithmetically consistent with the SEM across the domain's tasks (see docs/reviews/2026-10-13-biomlbench-agents.md). Field: `attributes.printed_value` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact Extraction artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Extraction artifact

How to read the evidence

Sources and history

Release 2026-11-02-a7f03c9e55a5 · Record review: source checked

1 source record and release history

BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Original source · bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07)

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: lit-agents-20261013-biomlbench-result-single-cell-omics-non-penalized-aide-leaderboard-percentile

metric: leaderboard-percentile
metric qualifier: mean over tasks in the domain; non-penalized: runs without a valid submission omitted
metric direction: higher
unit: percent
printed value: 60.67 ± 11.16
numeric value: 60.67
source locator: Table 4 (page 12), block 'Non-penalized', domain 'Single Cell Omics', row 'AIDE', column 'Leaderboard Percentile'
unit detail: percentile of public leaderboard entries, 0 to 100
uncertainty: type: unresolved_spread; value: 11.16; printed: ± 11.16; note: Table 4 caption does not name the spread. Figures 2 and 4 show per-domain leaderboard percentiles as mean ± SEM across tasks; the Table 2 caption refers to the reported SEM.
review: method: source-hash-verification; pdf-text-parse; independent-cell-check; scope-review; method note: Pinned v3 PDF re-checked: the extractor's file hashes to the pinned SHA-256, and a fresh download differs only in bioRxiv's per-download stamps (43 bytes: XMP dates and IDs, Info ModDate, trailer /ID) with an identical pdftotext -layout text layer. An independent parser of Table 4 (rows located by domain and agent labels, 5 cells per row asserted) matched printed_value, numeric_value, spread, locator, metric, unit, direction and evaluation link for all 132 results. Section 3.4 definitions, Table 2 and the bioRxiv v2 Table 4 were read for context.; reviewer: claude; reviewer note: Claude (Opus 5.5) research agent, independent of the extractor; no human review claimed; date: 2026-10-13; artifact sha256: 875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35; retrieval url: https://www.biorxiv.org/content/10.1101/2025.09.01.673319v3.full.pdf; note: Extraction: parsed from the pdftotext -layout text of the pinned PDF by extract/extract_biomlbench.py. Independent review 2026-10-13: matched. The spread stays unresolved_spread: the Table 4 caption does not name it, although every cell is arithmetically consistent with the SEM across the domain's tasks (see docs/reviews/2026-10-13-biomlbench-agents.md).

Related records

evaluation: AIDE on BioML-bench single cell omics (non-penalized)

Suggest a correction