Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 4 configurations
• 3 protocols
• 2 datasets
• 10 evaluations
• 23 results
• 6 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• baseline: structural-20261009-protocol-smorodina2026-vhh-cognate-vs-shuffled: Baseline Random baseline precision is about 0.011: about 89 to 91 positives out of about 8041 to 8281 evaluated pairs, depending on the tool after excluding missing values. (Results P11)
• Evidence summary: structural hypotheses for experiments: Summary Two new comparisons join the existing docking and FoldBench evidence, both on antibody complexes, and neither ties a prediction to an experiment's outcome. Fromm et al. 2026 generated 200 AlphaFold3 models for each of 110 antibody-antigen complexes deposited after the training cutoff, then asked which model each confidence score picks. Every score a user can compute picked models of about the same quality: mean DockQ 0.375 for ranking confidence, 0.370 for ipTM, 0.369 for pTM, 0.353 for ipSAE and 0.352 for pDockQ2, while the best of the 200 models averaged 0.544. The per-target correlation between score and accuracy stayed at or below 0.247, so within one target the confidence ranking carried little information. Smorodina et al. 2026, a bioRxiv preprint that has not been peer reviewed, ran AlphaFold3, Boltz-2, Boltz-1 and Chai-1 on 106 nanobody-antigen complexes. Sampling raised the best available model for every tool, with median best DockQ rising from 0.24 to 0.68 for AlphaFold3 and from 0.57 to 0.80 for Boltz-2 between 1 and 100 samples, though picking that best sample needs the experimental structure. Confidence tracked accuracy unevenly: the correlation between ipTM and best-sample DockQ was 0.736 for AlphaFold3, 0.665 for Boltz-2 and 0.612 for Chai-1, with 18% of Boltz-2 predictions confident but wrong and 24% of Chai-1 predictions accurate but unconfident. Pairing every nanobody with every antigen showed that confidence does not indicate that an interaction exists at all: average precision for ranking the real pairings above shuffled ones was 0.187 for AlphaFold3, 0.067 for Chai-1 and 0.026 for Boltz-2, against a baseline of about 0.011. The preprint's nanobody set mixes systems inside and outside each tool's training data, and its non-cognate pairings are assumed not to bind rather than tested. A high confidence score is a statement about a predicted structure, not evidence that two molecules interact. (structural-20261009-source-fromm2026: Table 1, sections 2.1 and 2.2; structural-20261009-source-smorodina2026: Results P8, P11, P19 and P36)
• negative_definition: structural-20261009-protocol-smorodina2026-vhh-cognate-vs-shuffled: Negative definition The source states: 'we assume that VHHs and antigens of “shuffled complexes” are non-binders'. Negatives are therefore pairings assumed not to bind, not pairings shown not to bind. (Results P9)
• Relevance of structural-20261009-protocol-smorodina2026-vhh-cognate-vs-shuffled to "Which predicted interfaces or structures are reliable enough to guide my next experiment?": Assessed by Telling cognate from shuffled nanobody-antigen pairs by ipTM (structural-20261009-source-smorodina2026: Results P9 to P11; Figure 2)
• Relevance of structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling to "Which predicted interfaces or structures are reliable enough to guide my next experiment?": Assessed by Best DockQ against sampling depth on cognate nanobody-antigen complexes (structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B)
• Relevance of structural-20261009-protocol-smorodina2026-vhh-iptm-dockq-calibration to "Which predicted interfaces or structures are reliable enough to guide my next experiment?": Assessed by How well ipTM tracks DockQ on cognate nanobody-antigen complexes (structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.