| Configuration: Bacformer (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 67.97 ± 20.73 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 20.73. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceBacformer on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Bacformer', column 'Binary AUPRC' |
|---|
| Configuration: Bacformer (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 87.61 ± 5.68 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 5.68. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceBacformer on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Bacformer', column 'Binary AUROC' |
|---|
| Configuration: Bacformer Large (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 76.14 ± 19.59 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 19.59. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceBacformer Large on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Bacformer Large', column 'Binary AUPRC' |
|---|
| Configuration: Bacformer Large (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 92.15 ± 4.20 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 4.20. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceBacformer Large on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Bacformer Large', column 'Binary AUROC' |
|---|
| Configuration: DNABERT-2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 52.35 ± 23.55 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.55. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceDNABERT-2 on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'DNABERT-2', column 'Binary AUPRC' |
|---|
| Configuration: DNABERT-2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 79.88 ± 7.26 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 7.26. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceDNABERT-2 on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'DNABERT-2', column 'Binary AUROC' |
|---|
| Configuration: ESM-2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 62.04 ± 23.64 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.64. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ESM-2', column 'Binary AUPRC' |
|---|
| Configuration: ESM-2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 85.05 ± 6.63 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 6.63. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ESM-2', column 'Binary AUROC' |
|---|
| Configuration: ESM-C (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 63.41 ± 23.43 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.43. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-C on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ESM-C', column 'Binary AUPRC' |
|---|
| Configuration: ESM-C (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 85.68 ± 6.81 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 6.81. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-C on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ESM-C', column 'Binary AUROC' |
|---|
| Configuration: Evo-2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 38.35 ± 23.08 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.08. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceEvo-2 on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Evo-2', column 'Binary AUPRC' |
|---|
| Configuration: Evo-2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 66.27 ± 10.19 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 10.19. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceEvo-2 on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Evo-2', column 'Binary AUROC' |
|---|
| Configuration: Evo (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 44.06 ± 23.28 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.28. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceEvo on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Evo', column 'Binary AUPRC' |
|---|
| Configuration: Evo (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 68.22 ± 9.32 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 9.32. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceEvo on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Evo', column 'Binary AUROC' |
|---|
| Configuration: gLM2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 55.80 ± 24.46 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 24.46. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourcegLM2 on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'gLM2', column 'Binary AUPRC' |
|---|
| Configuration: gLM2 (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 82.16 ± 6.75 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 6.75. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourcegLM2 on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'gLM2', column 'Binary AUROC' |
|---|
| Configuration: Mistral-DNA (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 49.86 ± 22.73 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 22.73. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceMistral-DNA on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Mistral-DNA', column 'Binary AUPRC' |
|---|
| Configuration: Mistral-DNA (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 76.05 ± 9.14 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 9.14. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceMistral-DNA on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Mistral-DNA', column 'Binary AUROC' |
|---|
| Configuration: Nucleotide Transformer (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 59.70 ± 22.90 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 22.90. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceNucleotide Transformer on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Nucleotide Transformer', column 'Binary AUPRC' |
|---|
| Configuration: Nucleotide Transformer (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 84.22 ± 6.47 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 6.47. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceNucleotide Transformer on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'Nucleotide Transformer', column 'Binary AUROC' |
|---|
| Configuration: ProkBERT (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 58.90 ± 23.67 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.67. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceProkBERT on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ProkBERT', column 'Binary AUPRC' |
|---|
| Configuration: ProkBERT (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 83.94 ± 6.39 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 6.39. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceProkBERT on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ProkBERT', column 'Binary AUROC' |
|---|
| Configuration: ProtBERT (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUPRC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 61.79 ± 23.19 AUPRC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 23.19. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceProtBERT on BacBench antibiotic resistance, binary linear probe AUPRC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ProtBERT', column 'Binary AUPRC' |
|---|
| Configuration: ProtBERT (frozen embeddings with a trained linear head) | Protocol: BacBench antibiotic resistance, binary linear probe AUROC Dataset subset: Antibiotic resistance binary prediction subset, 37 antibiotics | 84.51 ± 6.31 AUROC (linear head per drug on frozen genome embeddings, random 70/10/20 split, mean over 37 antibiotics) unitless · higher Uncertainty: SD 6.31. The caption says the spread is across antibiotics; the Methods state 37 antibiotics in the binary setting and 56 in the regression setting. Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceProtBERT on BacBench antibiotic resistance, binary linear probe AUROC One linear model per drug on mean-pooled whole-genome embeddings: a single layer preceded by dropout of 0.1 and layer normalisation, binary cross-entropy for the resistant or susceptible setting and mean squared error for the log(1+MIC) regression, at most 100 epochs, early stopping patience 10 on validation AUROC or validation R squared, a learning-rate sweep per model across seeds and drugs, Adam with weight decay 0.01, and three random splits. Aggregation: Not reported Bacformer preprint, bioRxiv 2025.07.20.665723 version 3 · Table S18, row 'ProtBERT', column 'Binary AUROC' |
|---|