BacBench
BacBench is a multi-task benchmark for machine-learning models of bacterial genomics, which embeds whole genomes with a chosen model and scores the embeddings on five active tasks across the bacterial tree of life.
Results are recorded on the linked protocols, tasks or configurations listed below. Their settings and evaluations are kept separate.
Overview
An explanatory profile has not yet been reviewed for this record.
Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.
Results on linked records
This benchmark has no results of its own in this release. 17 linked records have results. They are not assigned to the underlying benchmark: each record keeps its own settings and evaluations, and its scores are on its own page.
- BacBench antibiotic resistance, binary linear probe AUPRC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench antibiotic resistance, binary linear probe AUROC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench antibiotic resistance, MIC regression Pearson correlation Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench antibiotic resistance, MIC regression R squared Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench essential gene prediction, linear probe AUPRC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench essential gene prediction, linear probe AUROC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench operon identification, zero-shot AUPRC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench operon identification, zero-shot AUROC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench phenotypic trait prediction, linear probe AUPRC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench phenotypic trait prediction, linear probe AUROC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench protein-protein interaction, linear probe AUPRC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench protein-protein interaction, linear probe AUROC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench protein-protein interaction, zero-shot AUPRC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench protein-protein interaction, zero-shot AUROC Protocol · part of this benchmark · 12 evaluations · 12 results
- BacBench strain clustering, adjusted Rand index Protocol · part of this benchmark · 6 evaluations · 6 results
- BacBench strain clustering, average silhouette width Protocol · part of this benchmark · 6 evaluations · 6 results
- BacBench strain clustering, normalised mutual information Protocol · part of this benchmark · 6 evaluations · 6 results
Methods and evaluation design
Procedure, tasks and evaluated configurations
Additional methodological details, tasks and evaluated configurations have not yet been extracted into a structured profile for this benchmark. Consult the linked sources and evaluation records below.
Baseline coverage
Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.
0 of 34 active baseline roles have published Rewire measurements in this release. Measurements on a selected protocol do not establish coverage of an entire suite.
Baseline status by linked protocol
- BacBench antibiotic resistance, binary linear probe AUPRC · 0/2 roles measured
- BacBench antibiotic resistance, binary linear probe AUROC · 0/2 roles measured
- BacBench antibiotic resistance, MIC regression Pearson correlation · 0/2 roles measured
- BacBench antibiotic resistance, MIC regression R squared · 0/2 roles measured
- BacBench essential gene prediction, linear probe AUPRC · 0/2 roles measured
- BacBench essential gene prediction, linear probe AUROC · 0/2 roles measured
- BacBench operon identification, zero-shot AUPRC · 0/2 roles measured
- BacBench operon identification, zero-shot AUROC · 0/2 roles measured
- BacBench phenotypic trait prediction, linear probe AUPRC · 0/2 roles measured
- BacBench phenotypic trait prediction, linear probe AUROC · 0/2 roles measured
- BacBench protein-protein interaction, linear probe AUPRC · 0/2 roles measured
- BacBench protein-protein interaction, linear probe AUROC · 0/2 roles measured
- BacBench protein-protein interaction, zero-shot AUPRC · 0/2 roles measured
- BacBench protein-protein interaction, zero-shot AUROC · 0/2 roles measured
- BacBench strain clustering, adjusted Rand index · 0/2 roles measured
- BacBench strain clustering, average silhouette width · 0/2 roles measured
- BacBench strain clustering, normalised mutual information · 0/2 roles measured
Protocol coverage CSV (gzip) · Model evaluation matrix (gzip) · Source table (gzip) · Release and checksums (gzip)
Coverage is derived from release 2026-11-21-adc15377fab2. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.
Run instructions
No runnable recipe has been reviewed for this benchmark. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
0 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|
No evidence rows match these filters. Choose another scope or clear the search.
Sources and history
Release 2026-11-21-adc15377fab2 · Record review: source checked
2 source records and release history
- BacBench repository README at commit 64b7d45 · Original source · Commit 64b7d45823b2ff2ef88458e3aa235caa461a12aa
- Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Original source · Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19
Technical metadata and record identifiers
Technical metadata for this record is tracked under stable ID bacformer-20261115-benchmark-bacbench.
Source locator: BacBench README opening paragraph, 'News', 'Benchmarked models', 'Task overview' and 'To-do-list'
Related records
- subject: BacBench has five active tasks and one deprecated task
- subject: Apache-2.0 covers the BacBench code
- subject: BacBench publishes no numeric results
- subject: Where each BacBench task's printed results are
- subject: BacBench changed the essential gene and PPI splits on 2026-05-13
- subject: Bacformer's authors also built and ran the benchmark
- subject: Model table differences between the BacBench README and the preprint
- part of: BacBench antibiotic resistance, binary linear probe AUPRC
- part of: BacBench antibiotic resistance, binary linear probe AUROC
- part of: BacBench antibiotic resistance, MIC regression Pearson correlation
- part of: BacBench antibiotic resistance, MIC regression R squared
- part of: BacBench essential gene prediction, linear probe AUPRC
- part of: BacBench essential gene prediction, linear probe AUROC
- part of: BacBench operon identification, zero-shot AUPRC
- part of: BacBench operon identification, zero-shot AUROC
- part of: BacBench phenotypic trait prediction, linear probe AUPRC
- part of: BacBench phenotypic trait prediction, linear probe AUROC
- part of: BacBench protein-protein interaction, linear probe AUPRC
- part of: BacBench protein-protein interaction, linear probe AUROC
- part of: BacBench protein-protein interaction, zero-shot AUPRC
- part of: BacBench protein-protein interaction, zero-shot AUROC
- part of: BacBench strain clustering, adjusted Rand index
- part of: BacBench strain clustering, average silhouette width
- part of: BacBench strain clustering, normalised mutual information
- part of: BacBench Antibiotic resistance prediction
- part of: BacBench Essential gene prediction
- part of: BacBench Operon identification from long-read RNA sequencing
- part of: BacBench Phenotypic trait prediction
- part of: BacBench Protein-protein interaction prediction
- part of: BacBench Strain clustering