BacBench changed the essential gene and PPI splits on 2026-05-13
Protocol change of BacBench
On 13 May 2026 BacBench updated the essential gene and protein-protein interaction tasks to a phylogeny-aware split by genus, added BacLM as a benchmarked model and deprecated the strain clustering task
Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
8 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Claimed field protocol_change Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations1 values
| BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Rationale Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled. Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Source location BacBench README 'News', entry dated 13.05.2026 Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed value On 13 May 2026 BacBench updated the essential gene and protein-protein interaction tasks to a phylogeny-aware split by genus, added BacLM as a benchmarked model and deprecated the strain clustering task Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled. Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Relationship: subject bacformer-20261115-benchmark-bacbench Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Name BacBench changed the essential gene and PPI splits on 2026-05-13 Context-only references | BacBench repository README at commit 64b7d45 BacBench README 'News', entry dated 13.05.2026 Version: Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-21-c8bea25e5abb · Record review: source checked
1 source record and release history
- BacBench repository README at commit 64b7d45 · Original source · Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
Technical metadata and extraction receipts
Stable ID: bacformer-20261115-claim-bacbench-split-change
- areas
- microbes-communities
- contexts
- research
- field
- protocol_change
- limitations
- The README does not publish a split manifest, so whether version 3 of the preprint used exactly the current split files cannot be checked from these sources; its Methods do describe genus-level 60/20/20 splits for both tasks.
- rationale
- Results produced before that change are on a different split from results produced after it, so they are not the same protocol and must not be pooled.
- review
- artifact sha256: c9d28efef5b4cc30adbcc9de224c350100f5aa2a15a025beaf10dcdeb86cd93e; method: transcription; source-hash-verification; ai-assisted-source-review; method note: Independent review pass. Each source artifact was retrieved again at review time and its SHA-256 compared with the source record; the quoted statement was then read at the recorded locator and compared with the recorded value. Access states were rechecked rather than taken from notes.md.; note: Statement re-read against the pinned artifact by a reviewer who did not extract this batch. Every artifact except the two preprint PDFs was downloaded again at review time and hashed to exactly the value its source record records, and is byte-identical to the copy kept in artifacts/.; retrieval url: https://raw.githubusercontent.com/macwiatrak/BacBench/64b7d45823b2ff2ef88458e3aa235caa461a12aa/README.md; reviewed at: 2026-11-15T23:55:00Z; reviewer: claude; reviewer note: Claude (Opus 5) review agent, independent of the agent that extracted this batch; no human review claimed. Transcription was checked against the pinned sources. Nothing was reproduced and no value was recomputed.
- source locator
- BacBench README 'News', entry dated 13.05.2026
- value
- On 13 May 2026 BacBench updated the essential gene and protein-protein interaction tasks to a phylogeny-aware split by genus, added BacLM as a benchmarked model and deprecated the strain clustering task
Related records
- subject: BacBench