rewirebio.iobenchmarks
Evidence claim

BacBench changed the essential gene and PPI splits on 2026-05-13

Protocol change of BacBench

On 13 May 2026 BacBench updated the essential gene and protein-protein interaction tasks to a phylogeny-aware split by genus, added BacLM as a benchmarked model and deprecated the strain clustering task

Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

8 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Claimed field
protocol_change
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
1 values
  • The README does not publish a split manifest, so whether version 3 of the preprint used exactly the current split files cannot be checked from these sources; its Methods do describe genus-level 60/20/20 splits for both tasks.
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled.
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Source location
BacBench README 'News', entry dated 13.05.2026
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_locator

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed value
On 13 May 2026 BacBench updated the essential gene and protein-protein interaction tasks to a phylogeny-aware split by genus, added BacLM as a benchmarked model and deprecated the strain clustering task
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.value

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Description
Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled.
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: description

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: subject
bacformer-20261115-benchmark-bacbench
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: links:subject:bacformer-20261115-benchmark-bacbench

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Name
BacBench changed the essential gene and PPI splits on 2026-05-13
Context-only references
BacBench repository README at commit 64b7d45

Original source ↗

BacBench README 'News', entry dated 13.05.2026

Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Retrieved: 2026-11-15T23:09:36Z

not individually reviewed

No individual claim review recorded

Audit details

Field: name

Source artifact SHA-256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: bacformer-20261115-claim-bacbench-split-change

areas
microbes-communities
contexts
research
field
protocol_change
limitations
The README does not publish a split manifest, so whether version 3 of the preprint used exactly the current split files cannot be checked from these sources; its Methods do describe genus-level 60/20/20 splits for both tasks.
rationale
Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled.
review
artifact sha256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e; method: transcription; source-hash-verification; ai-assisted-source-review; method note: Independent review pass. Each source artifact was retrieved again at review time and its SHA-256 compared with the source record; the quoted statement was then read at the recorded locator and compared with the recorded value. Access states were rechecked rather than taken from notes.md.; note: Statement re-read against the pinned artifact by a reviewer who did not extract this batch. Every artifact except the two preprint PDFs was downloaded again at review time and hashed to exactly the value its source record records, and is byte-identical to the copy kept in artifacts/.; retrieval url: https://raw.githubusercontent.com/macwiatrak/BacBench/64b7d45823b2ff2ef88458e3aa235caa461a12aa/README.md; reviewed at: 2026-11-15T23:55:00Z; reviewer: claude; reviewer note: Claude (Opus 5) review agent, independent of the agent that extracted this batch; no human review claimed. Transcription was checked against the pinned sources. Nothing was reproduced and no value was recomputed.
source locator
BacBench README 'News', entry dated 13.05.2026
value
On 13 May 2026 BacBench updated the essential gene and protein-protein interaction tasks to a phylogeny-aware split by genus, added BacLM as a benchmarked model and deprecated the strain clustering task
Related records

Suggest a correction