rewirebio.iobenchmarks
Contested result

Boltz-2 and AlphaFold 3: Shen et al. 2025, Chemical Science (Europe PMC full text) and Kim, Correy, Hall et al. 2026, eLife 15:RP110475 (Europe PMC full text) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • Same task, different protocols

Shen et al. 2025, Chemical Science (Europe PMC full text) ranks AlphaFold 3 above Boltz-2 in 9 of 9 comparisons; Kim, Correy, Hall et al. 2026, eLife 15:RP110475 (Europe PMC full text) ranks Boltz-2 above AlphaFold 3 in 2 of 2 comparisons. No other study gives a verdict, so neither is the outlier.

An automatic lead, not a finding of error. The results below are shown as published, everywhere on the site, until a reviewer decides otherwise.

Which studies agree

3 papers rank Boltz-2 above AlphaFold 3 in 4 of their 19 comparisons and AlphaFold 3 higher in 14; 1 is a tie. Of those papers, Shen et al. 2025, Structural Plausibility Without Binding… rank AlphaFold 3 higher on their own (9 of 10, 5 of 7).

Papers grouped by which system they rank higher: Boltz-2 or AlphaFold 3Rank Boltz-2 higher: Kim, Correy, Hall et al. 2026. Rank AlphaFold 3 higher: Shen et al. 2025, Structural Plausibility Without Binding…. Mixed: none.Rank Boltz-2 higherMixedRank AlphaFold 3 higherKim, Correy, Hall et al.2026Kim, Correy, Hall et al. 2026, eLife 15:RP110475 (Europe PMC full text)Shen et al. 2025Shen et al. 2025, Chemical Science (Europe PMC full text)Structural PlausibilityWithout Binding…Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores
  • Rank Boltz-2 higher (1): Kim, Correy, Hall et al. 2026
  • Rank AlphaFold 3 higher (2): Shen et al. 2025, Structural Plausibility Without Binding…
  • Mixed (0): none

Each paper sits where at least two thirds of its comparisons point; Mixed means neither model reaches two thirds. The larger, outlined dot is the paper that disagrees.

Comparisons in which each paper scores Boltz-2 or AlphaFold 3 higherShen et al. 2025: Boltz-2 higher in 0, AlphaFold 3 higher in 9, 1 tie. Structural Plausibility Without Binding…: Boltz-2 higher in 2, AlphaFold 3 higher in 5, 0 ties. Kim, Correy, Hall et al. 2026: Boltz-2 higher in 2, AlphaFold 3 higher in 0, 0 ties.Shen et al. 2025Shen et al. 2025, Chemical Science (Europe PMC full text)1 tie19 AlphaFold 3 higher9Structural Plausibility Without Binding…Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores2 Boltz-2 higher25 AlphaFold 3 higher5Kim, Correy, Hall et al. 2026Kim, Correy, Hall et al. 2026, eLife 15:RP110475 (Europe PMC full text)2 Boltz-2 higher2
Boltz-2 higherTieAlphaFold 3 higherNumbers are counts of comparisons; the outlined bar is the paper that disagrees.
Comparisons in Shen et al. 2025 (10)
ProtocolDatasetMetricBoltz-2AlphaFold 3Higher
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025AUROC (AUROC, mean over the 79 DEKOIS 2.0 targets), unitless0.860.892AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025AUROC (AUROC, median over the 79 DEKOIS 2.0 targets), unitless0.8940.933AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Bedroc (BEDROC with alpha = 80.5, mean over the 79 DEKOIS 2.0 targets), unitless0.560.635AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Bedroc (BEDROC with alpha = 80.5, median over the 79 DEKOIS 2.0 targets), unitless0.6180.659AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Enrichment factor (EF at the top 0.5% of the ranked library, mean over the 79 DEKOIS 2.0 targets), unitless19.220.5AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Enrichment factor (EF at the top 0.5% of the ranked library, median over the 79 DEKOIS 2.0 targets), unitless22.122.1Tie
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Enrichment factor (EF at the top 1% of the ranked library, mean over the 79 DEKOIS 2.0 targets), unitless1819.3AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Enrichment factor (EF at the top 1% of the ranked library, median over the 79 DEKOIS 2.0 targets), unitless19.121.5AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Enrichment factor (EF at the top 5% of the ranked library, mean over the 79 DEKOIS 2.0 targets), unitless10.312.6AlphaFold 3
DEKOIS 2.0 screening with co-folding confidence scores against docking tools (Shen et al. 2025 Table 1)DEKOIS 2.0, 79 targets as scored by Shen et al. 2025Enrichment factor (EF at the top 5% of the ranked library, median over the 79 DEKOIS 2.0 targets), unitless1113.5AlphaFold 3
Comparisons in Structural Plausibility Without Binding… (7)
ProtocolDatasetMetricBoltz-2AlphaFold 3Higher
Telling cognate from shuffled nanobody-antigen pairs by ipTMSmorodina et al. 2026 all-against-all VHH-antigen pairing matrix (91 unique PDB entries)Average precision (cognate against shuffled pairs ranked by ipTM), unitless0.0260.187AlphaFold 3
Best DockQ against sampling depth on cognate nanobody-antigen complexesSmorodina et al. 2026 nanobody-antigen benchmark: 106 cognate VHH-antigen complexesDockQ (median over complexes of the best DockQ among N = 1 samples), unitless0.570.24Boltz-2
Best DockQ against sampling depth on cognate nanobody-antigen complexesSmorodina et al. 2026 nanobody-antigen benchmark: 106 cognate VHH-antigen complexesDockQ (median over complexes of the best DockQ among N = 100 samples), unitless0.80.68Boltz-2
How well ipTM tracks DockQ on cognate nanobody-antigen complexesSmorodina et al. 2026 nanobody-antigen benchmark: 106 cognate VHH-antigen complexesPearson (change in ipTM against change in DockQ under saturation sampling), unitless-0.04-0.027AlphaFold 3
How well ipTM tracks DockQ on cognate nanobody-antigen complexesSmorodina et al. 2026 nanobody-antigen benchmark: 106 cognate VHH-antigen complexesPearson (ipTM against DockQ for the best-DockQ sample of each complex), unitless0.6650.736AlphaFold 3
How well ipTM tracks DockQ on cognate nanobody-antigen complexesSmorodina et al. 2026 nanobody-antigen benchmark: 106 cognate VHH-antigen complexesPearson (ipTM against DockQ for the first sample (sample0) of each complex), unitless0.4930.888AlphaFold 3
How well ipTM tracks DockQ on cognate nanobody-antigen complexesSmorodina et al. 2026 nanobody-antigen benchmark: 106 cognate VHH-antigen complexesProportion (share of predictions with ipTM at least 0.5 and DockQ below 0.23 (confident failures, Q2)), percent182AlphaFold 3
Comparisons in Kim, Correy, Hall et al. 2026 (2)
ProtocolDatasetMetricBoltz-2AlphaFold 3Higher
Re-ranking the AmpC beta-lactamase docking hit list by co-folding and docking scores (Kim, Correy, Hall et al. 2026)AmpC beta-lactamase docking hit list, 1293 tested compoundsAUROC (Discrimination of measured binders from high-ranking docking false positives in the tested AmpC beta-lactamase hit list), unitless68.160.5Boltz-2
Re-ranking the Sigma-2 receptor docking hit list by co-folding and docking scores (Kim, Correy, Hall et al. 2026)Sigma-2 receptor docking hit list, 506 tested compoundsAUROC (Discrimination of measured binders from high-ranking docking false positives in the tested sigma-2 receptor hit list), unitless83.856.1Boltz-2

What disagrees

studies
2
Compared
Boltz-2, AlphaFold 3
Study that disagrees
Other studies
Rule
Two-study conflict; same task, different protocols
Model identity
Matched across studies partly through links not yet reviewed

Follow-up

Current state: Not yet reviewed.

No one has reviewed this finding yet.

How to help resolve this

If you know why these studies disagree, for example a different population, metric or preprocessing step, or you have a corrected value, tell us through the contribution form. A reviewer checks every submission against its source before anything changes, and the step is added to the follow-up above.

Finding discrepancy-965ec4c8efac9e07 · All contested results · How findings are made and followed up