Evaluating deep learning based structure prediction methods on antibody-antigen complexes

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 9 configurations
• 1 protocol
• 1 dataset
• 9 evaluations
• 27 results
• 3 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• Evidence summary: structural hypotheses for experiments: Summary Two new comparisons join the existing docking and FoldBench evidence, both on antibody complexes, and neither ties a prediction to an experiment's outcome. Fromm et al. 2026 generated 200 AlphaFold3 models for each of 110 antibody-antigen complexes deposited after the training cutoff, then asked which model each confidence score picks. Every score a user can compute picked models of about the same quality: mean DockQ 0.375 for ranking confidence, 0.370 for ipTM, 0.369 for pTM, 0.353 for ipSAE and 0.352 for pDockQ2, while the best of the 200 models averaged 0.544. The per-target correlation between score and accuracy stayed at or below 0.247, so within one target the confidence ranking carried little information. Smorodina et al. 2026, a bioRxiv preprint that has not been peer reviewed, ran AlphaFold3, Boltz-2, Boltz-1 and Chai-1 on 106 nanobody-antigen complexes. Sampling raised the best available model for every tool, with median best DockQ rising from 0.24 to 0.68 for AlphaFold3 and from 0.57 to 0.80 for Boltz-2 between 1 and 100 samples, though picking that best sample needs the experimental structure. Confidence tracked accuracy unevenly: the correlation between ipTM and best-sample DockQ was 0.736 for AlphaFold3, 0.665 for Boltz-2 and 0.612 for Chai-1, with 18% of Boltz-2 predictions confident but wrong and 24% of Chai-1 predictions accurate but unconfident. Pairing every nanobody with every antigen showed that confidence does not indicate that an interaction exists at all: average precision for ranking the real pairings above shuffled ones was 0.187 for AlphaFold3, 0.067 for Chai-1 and 0.026 for Boltz-2, against a baseline of about 0.011. The preprint's nanobody set mixes systems inside and outside each tool's training data, and its non-cognate pairings are assumed not to bind rather than tested. A high confidence score is a statement about a predicted structure, not evidence that two molecules interact. (structural-20261009-source-fromm2026: Table 1, sections 2.1 and 2.2; structural-20261009-source-smorodina2026: Results P8, P11, P19 and P36)
• headline_finding: structural-20261009-protocol-fromm2026-abag-model-selection: Headline finding For AlphaFold3 the mean DockQ of the top-ranked model rises only from 0.29 to 0.37 as sampling grows, while the best generated model reaches 0.52. For Boltz-1 and Chai-1 the top-ranked model barely improves with more samples (from 0.12 to 0.14). Scores computed from predicted aligned errors give a mean DockQ of about 0.35 for the selected model against 0.54 for the best model; the same scores computed from the true aligned errors close most of that gap. (Section 3.3 paragraph 1; section 3.6 paragraph 4)
• Relevance of structural-20261009-protocol-fromm2026-abag-model-selection to "Which predicted interfaces or structures are reliable enough to guide my next experiment?": Assessed by Picking the top AlphaFold3 model per antibody-antigen target with a confidence score (structural-20261009-source-fromm2026: Section 2.1; sections 3.3 and 3.6; Table 1)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.