Boltz-2: Towards Accurate and Efficient Binding Affinity Prediction, supplementary appendix

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 1 model
• 12 methods
• 16 configurations
• 2 benchmarks
• 3 protocols
• 3 datasets
• 26 evaluations
• 244 results
• 11 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• ABFE protocol optimised on the data it is scored on: Limitation The supplement states that 'the results of the ABFE protocol [Wu et al., 2025] on the 4 target subset may also represent an optimistic estimate of its prospective performance given that the protocol was optimized based this dataset'. (Appendix D.2.2, 'Baselines')
• Evidence summary: ranking analogues of a hit by affinity: Summary Three sources carry the extracted evidence. Every number below is a mean over congeneric series: each error and correlation is computed inside one series first, so these are averages of within-series values and not correlations pooled across unrelated targets. No source prints a value for an individual series, or the spread across series. Ross et al. 2023, written by the developers of the FEP+ workflow, sets both the free energy baseline and the assay noise floor on the same page. On a curated collection of 1,237 compounds across more than 50 proteins, FEP+ reached a compound-weighted pairwise RMSE of 1.25 kcal/mol [1.17, 1.33], a pairwise mean unsigned error of 0.98 [0.91, 1.05], a coefficient of determination of 0.56 [0.51, 0.60] and a Kendall tau of 0.51 [0.48, 0.55]; its edgewise errors, which depend on the perturbation graph's topology, were 1.17 [1.08, 1.25] and 0.91 [0.84, 0.98]. The same table's survey of 1,867 compounds whose affinities were measured in at least two independent assays gives the noise floor: pairwise RMSE 0.91 kcal/mol [0.83, 1.11], mean unsigned error 0.67 [0.61, 0.83], R2 0.79 [0.75, 0.82] and Kendall tau 0.71 [0.65, 0.74]. So the error of the free energy calculation is close to the error between two assays, while its rank correlation sits well below. The FEP+ protocol was adjusted by hand per system after the error against experiment had been seen, so this is a maximum-attainable rather than a prospective figure. The Boltz-2 supplement, written by the developers of Boltz-2, is the only source here that puts a machine-learning predictor beside free energy calculation on the same compounds. On 876 protein-ligand complexes from the same collection, Boltz-2 reached an assay-weighted Pearson R of 0.62 and Kendall tau of 0.46, against 0.63 and 0.47 for OpenFE under a fixed automated protocol and 0.72 and 0.53 for FEP+; the sequence-only baselines reached 0.29 and 0.19 (BACPI) and 0.28 and 0.20 for its ligand-only ablation, and Boltz-2's own interface confidence score reached -0.07 and -0.05. On a four-target subset of 87 neutral compounds the supplement prints compute cost beside accuracy: Boltz-2 0.66 and 0.48 at 20 GPU seconds, OpenFE 0.66 and 0.51 at 6-12 GPU hours, ABFE 0.75 and 0.54 at more than 20 GPU hours, FEP+ 0.78 and 0.63 with its time cell printed as '-', FMO 0.55 and 0.38 at 2-10 CPU minutes, Chemgauss4 0.26 and 0.17 at 20-30 CPU seconds and MM/PBSA 0.18 and 0.16 at 10-15 CPU minutes. No measured FEP+ cost appears in the table. On the CASP16 blind challenge, where the mmseqs 90% identity filtering used for the other benchmarks was not applied because the data postdates the training cutoff, the supplement reports Boltz-2 at 0.65 and 0.45 against the highest-ranked participant at 0.54 and 0.42. Tables 11, 12 and 14 print one value per cell with no uncertainty. The independent CASP16 assessment sets those challenge figures against a ceiling. Its abstract gives the maximum Kendall tau across all participants as 0.42, against an estimated best achievable value of 0.73 derived by resampling the measurements under an assumed threefold IC50 error (0.54 with a standard deviation of 0.11 for chymase, 0.76 with 0.02 for autotaxin); under an assumed 30-fold error the expected best tau for autotaxin falls to 0.43. Its body text gives the maximum achieved value as 0.39 for one figure and 'about 0.4' elsewhere without reconciling the three numbers, and it scores submissions under its own weighting and skip conventions, so its 0.42 and the 0.42 printed for group LG016 in the Boltz-2 supplement are not established to be the same quantity. Supplying experimental structures in stage 2 produced no shift toward greater accuracy, and the assessors note that the best CASP16 stage-1 results are no better than the typical best stage 1 D3R result, which averaged a Kendall tau of 0.48 with a minimum of 0.21 and a maximum of 0.71. Two comparisons set a machine-learning predictor against free energy calculation on the same series, and both are reported by the developers of the machine-learning method: the Boltz-2 supplement's two FEP+ benchmark subsets. Both take their FEP+ arm from a benchmark whose protocol was tuned per system after seeing the experimental error, which raises the baseline being compared against. And because every value is a mean over series, these numbers say how well each method ranked analogues on average across these series, not how well it would rank any one series. (Ross et al. 2023 Tables 1-3; Boltz-2 supplement Tables 11, 12 and 14; Gilson et al. 2026 Abstract and Section 3.3.1)
• FEP+ protocol tuned per system after seeing the error against experiment: Limitation The Boltz-2 supplement states that 'The results of FEP+ are intended to show the maximum attainable accuracy of current (commercial) FEP simulations by manually adjusting the protocol (input preparation, perturbation map, force field) to the system at hand after observing the error with respect to the experiment [Ross et al., 2023]'. Ross et al. 2023 describes the same adjustments in its own words: extra rotamer states, protonation states, atom mappings and scaled torsions were added 'from visual inspection and manual intervention' when poor sampling was observed in prior simulations. (Boltz-2 supplement appendix D.2.2, 'Baselines'; Ross et al. 2023 Results, 'Using FEP to resolve ambiguities in the structural inputs')
• Leakage control applied to the FEP+ benchmark subsets: Leakage control Training proteins with at least 90% sequence similarity to a validation or test protein were removed, by clustering all affinity-dataset protein sequences with 'mmseqs easy-cluster ... --min-seq-id 0.9 --cov-mode 0 -c 0.01' and dropping any training protein sharing a cluster with a validation or test protein (appendix D.2.1, 'Data leakage control'). Compound-level dependence was checked in Figure 10 and no strong dependence was observed; that figure prints no values. (Appendix D.2.1, 'Data leakage control'; Figure 10)
• Leakage control applied to the four-target subset: Leakage control Training proteins with at least 90% sequence similarity to a validation or test protein were removed, by clustering all affinity-dataset protein sequences with 'mmseqs easy-cluster ... --min-seq-id 0.9 --cov-mode 0 -c 0.01' and dropping any training protein sharing a cluster with a validation or test protein (appendix D.2.1, 'Data leakage control'). Compound-level dependence was checked in Figure 10 and no strong dependence was observed; that figure prints no values. (Appendix D.2.1, 'Data leakage control'; Figure 10)
• No sequence filtering applied to the CASP16 affinity benchmark: Leakage control The sequence-level filtering described in appendix D.2.1 was not applied to CASP16, because the CASP16 data was released after the training-data cutoff. The maximum Tanimoto similarity of test compounds to the affinity training set is stated as 0.41 for L1000 and 0.59 for L3000. (Appendix D.2.1, 'Data leakage control')
• Per-series affinity-ranking values are not printed: Limitation Every value in Boltz-2 supplement Tables 11, 12 and 14 and in Ross et al. 2023 Table 3 is an average over series: the Boltz-2 metrics are computed per assay and averaged across assays weighted by compound count, and the Ross values are weighted averages over FEP graphs. Neither source prints a per-series value, and neither prints the spread across series, so these tables support statements about the mean over series and not about any one series. (Boltz-2 supplement appendix D.2.2 'Evaluation metrics' and Tables 11, 12, 14; Ross et al. 2023 Table 3 legend)
• Relevance of CASP16 affinity challenge, target-averaged to ranking analogues of a hit: Assessed by CASP16 affinity challenge, Boltz-2 against the top-ranked participants (Appendix D.2.1 'Data leakage control'; D.2.2 'Evaluation metrics' and 'Baselines'; Tables 11, 12 and 14)
• Relevance of FEP+ benchmark, 876-complex OpenFE subset to ranking analogues of a hit: Assessed by FEP+ benchmark OpenFE subset, affinity ranking (Appendix D.2.1 'Data leakage control'; D.2.2 'Evaluation metrics' and 'Baselines'; Tables 11, 12 and 14)
• Relevance of FEP+ benchmark, four-target subset to ranking analogues of a hit: Assessed by FEP+ benchmark four-target subset, affinity ranking and cost (Appendix D.2.1 'Data leakage control'; D.2.2 'Evaluation metrics' and 'Baselines'; Tables 11, 12 and 14)
• The speed claim against FEP is not backed by a measured FEP cost: Limitation Table 12 is the only table that prints compute cost. Its FEP+ time cell is printed as '-' and its OpenFE and ABFE cells are ranges ('6-12 GPU hours', '>20 GPU hours'), so no measured FEP+ cost appears anywhere in the extracted tables. (Table 12, column 'time')

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.