| Comparison: adaptation No training; embeddings scored directly Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.adaptation Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: aggregation Mean and standard deviation across held-out test genomes Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.aggregation Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: budget About 90 minutes for the zero-shot evaluation of all models and 91 hours for the finetuning including the learning-rate sweep, on one NVIDIA H100 with 96 GB of memory and 72 CPU cores, excluding the cost of embedding the genomes Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.budget Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: dataset version As described in version 3 of the preprint and published on Hugging Face Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.dataset_version Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: inputs Single protein sequence. Each protein embedded on its own, last-layer hidden states mean-pooled over amino acids Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.inputs Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: metric implementation zero-shot pair scoring by embedding cosine similarity, genus-level split, mean over held-out test genomes Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.metric_implementation Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: population 261 genomes across 215 genera, randomly sampled from the bacterial organisms in STRING DB Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.population Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: protocol id bacformer-20261115-protocol-ppi-zero-shot-auprc Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.protocol_id Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
| Comparison: split Phylogeny-aware split by genus, 60/20/20 train, validation and test. Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.comparison.split Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|
Limitations2 values- Reported by the authors of Bacformer, who also built BacBench and ran every benchmarked model on it.
- Not reproduced by Rewire; the exact splits, seeds and BacBench commit behind the runs are not published.
Context-only references | Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 Original source ↗ Table S13, row 'ProtBERT', column 'Zero-shot AUPRC' Version: Version 3, posted 17 August 2026 according to the page banner; the bioRxiv API lists version 3 with date 2026-08-19 Retrieved: 2026-11-15T22:55:00Z | not individually reviewed No individual claim review recorded author reported Audit detailsField: attributes.limitations Source artifact SHA-256: d86de3315063e038ec8d5bed60b7b1a916c137ee60973e55d9d5c82f078d8dfd Hash scope: The hash is of one download. bioRxiv regenerates this PDF on request, so the container bytes differ between downloads: a second download minutes later gave 08fcc83af256845d942345fc6b63c64911878733acd028edab5e9307d63319db. The pdftotext -layout text layer of both downloads is byte-identical (SHA-256 86636e89b167d7304b3a83ee7f159cad0a9bdbe97019588ce836087b8fe3b0b2), and that text layer, archived gzipped in artifacts/ with SHA-256 0afdefd78a410d036d1c4f17e33bdf481e8d76241ee02f4c84c7c4be91ee72aa, is what every extracted value was read from. |
|---|