Evidence claim

benchmark-biomlbench: limitations

Agent scores are ranked against human leaderboards which are largely expert-populated; public leaderboards inevitably reflect a mix of participant expertise, effort, and compute budgets.

Descriptive fact transcribed from the pinned source.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

6 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-02-a7f03c9e55a5
Property and statement	Original source and location	Review and provenance
Claimed field attributes.limitations Context-only references	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Figure 1 caption; Section 5.1, second paragraph Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.field` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact
Source location Figure 1 caption; Section 5.1, second paragraph Context-only references	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Figure 1 caption; Section 5.1, second paragraph Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.source_locator` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact
Claimed value Agent scores are ranked against human leaderboards which are largely expert-populated; public leaderboards inevitably reflect a mix of participant expertise, effort, and compute budgets. Context-only references	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Figure 1 caption; Section 5.1, second paragraph Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	not individually reviewed No individual claim review recorded Audit details Field: `attributes.value` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact
Description Descriptive fact transcribed from the pinned source. Context-only references	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Figure 1 caption; Section 5.1, second paragraph Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	not individually reviewed No individual claim review recorded Audit details Field: `description` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact
Relationship: subject lit-agents-20261013-biomlbench-benchmark-biomlbench Context-only references	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Figure 1 caption; Section 5.1, second paragraph Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	not individually reviewed No individual claim review recorded Audit details Field: `links:subject:lit-agents-20261013-biomlbench-benchmark-biomlbench` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact
Name benchmark-biomlbench: attributes.limitations Context-only references	BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML Original source ↗ Figure 1 caption; Section 5.1, second paragraph Version: bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07) Retrieved: 2026-10-13T12:38:42Z	not individually reviewed No individual claim review recorded Audit details Field: `name` Source artifact SHA-256: `875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35` Hash scope: SHA-256 of the PDF bytes as downloaded 2026-10-13T12:38:42Z. bioRxiv rewrites the XMP dates and document IDs, the Info ModDate and the trailer /ID on download, so a re-download does not reproduce the hash; the review found a fresh copy differing only in those 43 bytes, with an identical text layer. Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-02-a7f03c9e55a5 · Record review: source checked

1 source record and release history

BioML-bench: Evaluation of AI Agents for End-to-End Biomedical ML · Original source · bioRxiv 2025.09.01.673319 version 3, posted 2025-09-28 (versions 1 and 2 posted 2025-09-04 and 2025-09-07)

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: lit-agents-20261013-biomlbench-claim-benchmark-human-baselines

areas: molecular-interactions; protein-function; single-cell
contexts: research
field: attributes.limitations
value: Agent scores are ranked against human leaderboards which are largely expert-populated; public leaderboards inevitably reflect a mix of participant expertise, effort, and compute budgets.
source locator: Figure 1 caption; Section 5.1, second paragraph
review: method: source-hash-verification; pdf-text-parse; independent-cell-check; method note: Searched the pdftotext -layout text of the pinned v3 PDF for the quoted wording at the locator.; reviewer: claude; reviewer note: Claude (Opus 5.5) research agent, independent of the extractor; no human review claimed; date: 2026-10-13; artifact sha256: 875cb3519dac9b0ec51c35224e1fc90b107b0676a4cc7c22cecb56eb8ac38b35; retrieval url: https://www.biorxiv.org/content/10.1101/2025.09.01.673319v3.full.pdf; note: Independent review 2026-10-13: wording found at the locator.

Related records

subject: BioML-bench

Suggest a correction