Evidence summary: ranking analogues of a hit by affinity

Summary of Rank close analogues of a hit by binding affinity

Three sources carry the extracted evidence. Every number below is a mean over congeneric series: each error and correlation is computed inside one series first, so these are averages of within-series values and not correlations pooled across unrelated targets. No source prints a value for an individual series, or the spread across series. Ross et al. 2023, written by the developers of the FEP+ workflow, sets both the free energy baseline and the assay noise floor on the same page. On a curated collection of 1,237 compounds across more than 50 proteins, FEP+ reached a compound-weighted pairwise RMSE of 1.25 kcal/mol [1.17, 1.33], a pairwise mean unsigned error of 0.98 [0.91, 1.05], a coefficient of determination of 0.56 [0.51, 0.60] and a Kendall tau of 0.51 [0.48, 0.55]; its edgewise errors, which depend on the perturbation graph's topology, were 1.17 [1.08, 1.25] and 0.91 [0.84, 0.98]. The same table's survey of 1,867 compounds whose affinities were measured in at least two independent assays gives the noise floor: pairwise RMSE 0.91 kcal/mol [0.83, 1.11], mean unsigned error 0.67 [0.61, 0.83], R2 0.79 [0.75, 0.82] and Kendall tau 0.71 [0.65, 0.74]. So the error of the free energy calculation is close to the error between two assays, while its rank correlation sits well below. The FEP+ protocol was adjusted by hand per system after the error against experiment had been seen, so this is a maximum-attainable rather than a prospective figure. The Boltz-2 supplement, written by the developers of Boltz-2, is the only source here that puts a machine-learning predictor beside free energy calculation on the same compounds. On 876 protein-ligand complexes from the same collection, Boltz-2 reached an assay-weighted Pearson R of 0.62 and Kendall tau of 0.46, against 0.63 and 0.47 for OpenFE under a fixed automated protocol and 0.72 and 0.53 for FEP+; the sequence-only baselines reached 0.29 and 0.19 (BACPI) and 0.28 and 0.20 for its ligand-only ablation, and Boltz-2's own interface confidence score reached -0.07 and -0.05. On a four-target subset of 87 neutral compounds the supplement prints compute cost beside accuracy: Boltz-2 0.66 and 0.48 at 20 GPU seconds, OpenFE 0.66 and 0.51 at 6-12 GPU hours, ABFE 0.75 and 0.54 at more than 20 GPU hours, FEP+ 0.78 and 0.63 with its time cell printed as '-', FMO 0.55 and 0.38 at 2-10 CPU minutes, Chemgauss4 0.26 and 0.17 at 20-30 CPU seconds and MM/PBSA 0.18 and 0.16 at 10-15 CPU minutes. No measured FEP+ cost appears in the table. On the CASP16 blind challenge, where the mmseqs 90% identity filtering used for the other benchmarks was not applied because the data postdates the training cutoff, the supplement reports Boltz-2 at 0.65 and 0.45 against the highest-ranked participant at 0.54 and 0.42. Tables 11, 12 and 14 print one value per cell with no uncertainty. The independent CASP16 assessment sets those challenge figures against a ceiling. Its abstract gives the maximum Kendall tau across all participants as 0.42, against an estimated best achievable value of 0.73 derived by resampling the measurements under an assumed threefold IC50 error (0.54 with a standard deviation of 0.11 for chymase, 0.76 with 0.02 for autotaxin); under an assumed 30-fold error the expected best tau for autotaxin falls to 0.43. Its body text gives the maximum achieved value as 0.39 for one figure and 'about 0.4' elsewhere without reconciling the three numbers, and it scores submissions under its own weighting and skip conventions, so its 0.42 and the 0.42 printed for group LG016 in the Boltz-2 supplement are not established to be the same quantity. Supplying experimental structures in stage 2 produced no shift toward greater accuracy, and the assessors note that the best CASP16 stage-1 results are no better than the typical best stage 1 D3R result, which averaged a Kendall tau of 0.48 with a minimum of 0.21 and a maximum of 0.71. Two comparisons set a machine-learning predictor against free energy calculation on the same series, and both are reported by the developers of the machine-learning method: the Boltz-2 supplement's two FEP+ benchmark subsets. Both take their FEP+ arm from a benchmark whose protocol was tuned per system after seeing the experimental error, which raises the baseline being compared against. And because every value is a mean over series, these numbers say how well each method ranked analogues on average across these series, not how well it would rank any one series.

Plain-language summary of what the extracted evidence shows. Draft until reviewed.