rewirebio.iobenchmarks
Evidence and investigation

Contested results

Where one study disagrees with what other studies report for the same models, and what we are doing to resolve each case.

These are automatic leads, not findings of error. Each release, a check compares how every study ranks the models it shares with other studies. It flags a study that reverses the order the others agree on, or whose gap to a shared reference model lies well outside theirs. Different tasks, populations or metrics often explain a flag.

A flag never hides a result. Flagged results stay in every table, page and download, marked Contested. A result is withheld only when a reviewer marks it disputed.

How the check works, and how each follow-up is recorded: discrepancies between studies.

In this release

  • 192open
  • 6settled
  • 7resolved
  • 3medium severity
  • 202low severity

Release 2026-11-21-c8bea25e5abb · 205 findings in all · rules version 5

Findings

Status: All · Open · Settled · Resolved

Severity: All · Medium · Low

State: All · Not yet reviewed · Under investigation · Explained · Confirmed error · Not a real disagreement

Showing findings that compare Boltz-2. All models

2 findings match, most severe first.

Boltz-2 and Protenix: Shen et al. 2025, Chemical Science (Europe PMC full text) and POISK Explorer project site disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • different-task

Shen et al. 2025, Chemical Science (Europe PMC full text) ranks Boltz-2 above Protenix in 26 of 28 comparisons; POISK Explorer project site ranks Protenix above Boltz-2 in 10 of 10 comparisons. No other study gives a verdict, so neither is the outlier.

2 papers rank Boltz-2 above Protenix in 26 of their 42 comparisons and Protenix higher in 12; 4 are ties. Of those papers, POISK Explorer project site ranks Protenix higher on its own (10 of 10).

Comparisons in which each paper scores Boltz-2 or Protenix higher2 other papers: Boltz-2 higher in 26, Protenix higher in 12, 4 ties.2 other papers26 Boltz-2 higher264 tie412 Protenix higher12
Boltz-2 higherTieProtenix higher
Compared
Boltz-2, Protenix

Not reviewed yet.

Boltz-2 and AlphaFold 3: Shen et al. 2025, Chemical Science (Europe PMC full text) and Kim, Correy, Hall et al. 2026, eLife 15:RP110475 (Europe PMC full text) disagree

  • Low severity
  • Not yet reviewed
  • Two-study conflict
  • Same task, different protocols

Shen et al. 2025, Chemical Science (Europe PMC full text) ranks AlphaFold 3 above Boltz-2 in 9 of 9 comparisons; Kim, Correy, Hall et al. 2026, eLife 15:RP110475 (Europe PMC full text) ranks Boltz-2 above AlphaFold 3 in 2 of 2 comparisons. No other study gives a verdict, so neither is the outlier.

3 papers rank Boltz-2 above AlphaFold 3 in 4 of their 19 comparisons and AlphaFold 3 higher in 14; 1 is a tie. Of those papers, Shen et al. 2025, Structural Plausibility Without Binding… rank AlphaFold 3 higher on their own (9 of 10, 5 of 7).

Comparisons in which each paper scores Boltz-2 or AlphaFold 3 higher3 other papers: Boltz-2 higher in 4, AlphaFold 3 higher in 14, 1 tie.3 other papers4 Boltz-2 higher41 tie14 AlphaFold 3 higher14
Boltz-2 higherTieAlphaFold 3 higher
Compared
Boltz-2, AlphaFold 3

Not reviewed yet.