CASP16 affinity challenge, Boltz-2 against the top-ranked participants

Stable ID: uc-leadopt-20261022-protocol-boltz2-casp16

areas

molecular-interactions

contexts

research

protocol

Pearson R, Kendall tau, pairwise MAE, MAE (non-centered and centered) and the fraction of predictions within 1 and 2 kcal/mol, as printed in Table 14

version

Boltz-2 supplement (bioRxiv v1) Table 14

aggregation

Calculated per assay and averaged across assays with weight proportional to the number of compounds in each assay (appendix D.2.2, 'Evaluation metrics'). The Pearson R, Kendall tau and PMAE columns are printed under the label 'target avg.'; assays and targets are not the same unit, and the source does not reconcile the printed label with the stated computation.

metric definition

Appendix D.2.2 'Evaluation metrics': Pearson R, Kendall tau, pairwise MAE (PMAE), MAE and the fraction of predictions within 1 and 2 kcal/mol, each computed per assay and averaged across assays weighted by compound count, with predictions converted to kcal/mol first; the centered variants translate each assay's predictions to the ground-truth mean

control

The sequence-level filtering described in appendix D.2.1 was not applied to CASP16, because the CASP16 data was released after the training-data cutoff. The maximum Tanimoto similarity of test compounds to the affinity training set is stated as 0.41 for L1000 and 0.59 for L3000.

heldout test

Blind challenge data released after the training-data cutoff; no sequence filtering was applied

source locator

Table 14, 'CASP16 competition'

limitations

One value per cell with no uncertainty.; The column headers read 'target avg.' while appendix D.2.2 defines the aggregation as a compound-weighted mean across assays; the source does not reconcile the two labels.; No per-series value is printed, so the table cannot say how either target was ranked on its own.; The participant rows are anonymous CASP16 group identifiers and the supplement says nothing about what methods they used.; The supplement does not state how the participants' submissions were obtained or re-scored.; The mmseqs 90% identity filtering applied to the other affinity benchmarks was not applied here.; The units of the pairwise MAE and MAE columns are not printed in the table header; appendix D.2.2 states that all predictions are converted to kcal/mol before the metrics are computed.; The 'Perc. within 1' and 'Perc. within 2' headers say percentage, but the printed values lie on a 0 to 1 scale.; One target can carry several assays, so a compound-weighted mean across assays is not a mean over targets even where the two happen to coincide; Ross et al. 2023 Table 1 surveys this collection's own proteins measured in more than one assay.

missing metadata

uncertainty reporting: reason: unreported; note: Table 14 prints no uncertainty

review

method: source-hash-verification; pdf-text-parse; independent-cell-check; rendered-source-visual-check; ai-assisted-source-review; method note: Re-retrieved all three sources and matched each pinned SHA-256. Re-rendered the Boltz-2 supplement with pdftotext -layout (poppler 26.08.0) and matched the text layer's recorded SHA-256. Re-read all 256 cells with a parser written for this review that does not import the extraction script, matching each result to its cell through its own source_locator, and separately read appendix tables 11, 12 and 14 from PNG renderings of PDF pages 29 and 30 at 160 dpi, and Ross Table 3 from the pinned XML markup. Checked printed value, numeric value, locator, metric concept, qualifier, direction, unit, uncertainty and the linked configuration, protocol and dataset, and that every N/A or '-' cell is recorded as missing with a reason.; reviewer: claude; reviewer note: Claude review agent, independent of the extraction agent; no human review claimed; reviewed at: 2026-10-22T13:40:00Z; artifact sha256: be867a5d92add8b223d47d882a4213ad489741f3e4d5081a0901a54370342569; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12262699/supplementaryFiles

• part of: CASP16 binding affinity challenge
• uses data: CASP16 affinity targets L1000 and L3000
• evaluates task: Small-molecule affinity
• subject: No sequence filtering applied to the CASP16 affinity benchmark
• assessment: BACPI on the CASP16 affinity challenge
• assessment: Boltz-2 on the CASP16 affinity challenge
• assessment: Boltz-2 iptm on the CASP16 affinity challenge
• assessment: GAT on the CASP16 affinity challenge
• assessment: LG008 on the CASP16 affinity challenge
• assessment: LG016 on the CASP16 affinity challenge
• assessment: LG055 on the CASP16 affinity challenge
• assessment: LG082 on the CASP16 affinity challenge
• assessment: LG204 on the CASP16 affinity challenge
• assessment: LG207 on the CASP16 affinity challenge
• assessed by: Rank close analogues of a hit by binding affinity