rewirebio.iobenchmarks
Evaluation

Boltz-2 on POISK pairwise binder-versus-decoy accuracy at 25% coverage

Author-reported measurement transcribed from the pinned preprint. Not independently reproduced.

Evaluation results

1 evaluation · 1 result. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: Boltz-2 with full-Fv ipSAE rankingProtocol: POISK pairwise binder-versus-decoy accuracy at 25% coverage
Dataset subset: POISK co-folding binder-decoy benchmark set (22,494 complexes)
0.622 Paired accuracy (binder scored above its matched putative decoy by full-Fv ipSAE, top 25% of the 11,247 pairs ranked by within-pair maximum ipSAE, exact ties at half credit)
fraction · higher

Uncertainty: Not reported by the source: Table 7 prints no spread; the paper says the pairwise benchmarks operate on the full set of 11,247 pairs without random sampling, so error bars do not apply.

Coverage: Not reported scored / Not reported eligible

Author-reported evaluation · Source checked
Methods, coverage and source

Boltz-2 on POISK pairwise binder-versus-decoy accuracy at 25% coverage

poisk-20261115-protocol-pairwise-coverage-25

Aggregation: Not reported

Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF) · Table 7, row "Boltz-2", column "25%"

Source checking is not independent reproduction. Release 2026-11-21-c8bea25e5abb.

Research readiness

0 of 4 readiness checks met. These checks assess whether the evidence supports a reproducible investigation; a source-checked score alone does not meet them.

Readiness checks, gaps and artifacts

Release 2026-11-21-c8bea25e5abb · Evidence verified: Not verified

Evidence incomplete

Replay metrics

Exact outcomes, predictions, identifiers and evaluator are connected.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • Metrics are recomputed from the saved predictions: not yet verified

Verified: Not verified

Evidence incomplete

Investigate discrepancies

Replay evidence includes annotations and an assessment of dependence. Unknown independence permits descriptive analysis only.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • Metrics are recomputed from the saved predictions: not yet verified
  • Sample annotations are recorded: not yet verified
  • Dependence between samples is assessed: not yet verified

Verified: Not verified

Evidence incomplete

Run locally

A pinned recipe describes the inputs, environment and resource requirements.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • A pinned run recipe exists: not yet verified
  • Compute requirements are estimated: not yet verified

Verified: Not verified

Evidence incomplete

Validate independently

Separate data and exposure records support an independent test.

Missing or unresolved evidence

  • No verified artifact manifest is linked to this exact record.
  • File checksums match the recorded files: not yet verified
  • Predictions are matched to the right samples: not yet verified
  • Score meaning and direction are confirmed: not yet verified
  • Independent validation data exist: not yet verified
  • Overlap with training data is checked: not yet verified

Verified: Not verified

Readiness describes the evidence in this release. Availability on your computer is checked separately when an investigation runs. Existing data exposure can prevent independent validation even when files are available.

Artifacts and reproduction

No verified artifact manifest is connected to this record yet. The gaps above identify what is needed before analysis can begin.

Evaluation procedure

POISK pairwise binder-versus-decoy accuracy at 25% coverage

Configuration
Boltz-2 with full-Fv ipSAE ranking
Protocol
POISK pairwise binder-versus-decoy accuracy at 25% coverage
Dataset subset
POISK co-folding binder-decoy benchmark set (22,494 complexes)
origin
Author-reported evaluation
configuration
bioRxiv preprint v1 posted 26 October 2026, retrieved 2026-11-15
adaptation
Zero-shot: no model was trained or fine-tuned on POISK
aggregation
As printed by the source for this endpoint
budget
Folding all 22,494 complexes with all six models took approximately 9,000 NVIDIA B200 GPU-hours
dataset version
POISK dataset version 1.0.1; the folding predictions of the six models were run against that version
inputs
Antibody heavy-chain and light-chain amino-acid sequences and the target amino-acid sequence, with ColabFold multiple sequence alignments and no templates
metric implementation
Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.
population
binder scored above its matched putative decoy by full-Fv ipSAE, top 25% of the 11,247 pairs ranked by within-pair maximum ipSAE, exact ties at half credit
protocol id
poisk-20261115-protocol-pairwise-coverage-25
split
Evaluation only; the benchmark is scored in full and nothing is held out for training

Metadata review: source checked. Unreported conditions prevent automatic comparisons.

Reproduction

Split
Evaluation only; the benchmark is scored in full and nothing is held out for training
Adaptation
Zero-shot: no model was trained or fine-tuned on POISK
Scoring implementation
Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.

No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.

Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

21 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Comparison: adaptation
Zero-shot: no model was trained or fine-tuned on POISK
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.adaptation

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: aggregation
As printed by the source for this endpoint
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.aggregation

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: budget
Folding all 22,494 complexes with all six models took approximately 9,000 NVIDIA B200 GPU-hours
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.budget

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: dataset version
POISK dataset version 1.0.1; the folding predictions of the six models were run against that version
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.dataset_version

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: inputs
Antibody heavy-chain and light-chain amino-acid sequences and the target amino-acid sequence, with ColabFold multiple sequence alignments and no templates
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.inputs

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: metric implementation
Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.metric_implementation

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: population
binder scored above its matched putative decoy by full-Fv ipSAE, top 25% of the 11,247 pairs ranked by within-pair maximum ipSAE, exact ties at half credit
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.population

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: protocol id
poisk-20261115-protocol-pairwise-coverage-25
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.protocol_id

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Comparison: split
Evaluation only; the benchmark is scored in full and nothing is held out for training
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.comparison.split

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Denominator
11247
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Table 7, row "Boltz-2", column "25%"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

author reported

Audit details

Field: attributes.denominator

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

How to read the evidence

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: poisk-20261115-eval-boltz-2-pairwise-coverage-25

areas
molecular-interactions; proteins-complexes
contexts
research
comparison
adaptation: Zero-shot: no model was trained or fine-tuned on POISK; aggregation: As printed by the source for this endpoint; budget: Folding all 22,494 complexes with all six models took approximately 9,000 NVIDIA B200 GPU-hours; dataset version: POISK dataset version 1.0.1; the folding predictions of the six models were run against that version; inputs: Antibody heavy-chain and light-chain amino-acid sequences and the target amino-acid sequence, with ColabFold multiple sequence alignments and no templates; metric implementation: Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.; population: binder scored above its matched putative decoy by full-Fv ipSAE, top 25% of the 11,247 pairs ranked by within-pair maximum ipSAE, exact ties at half credit; protocol id: poisk-20261115-protocol-pairwise-coverage-25; split: Evaluation only; the benchmark is scored in full and nothing is held out for training
denominator
11247
limitations
Reported by the authors of the benchmark, who built the dataset and ran every model on it. Not reproduced by Rewire.; The scores being ranked are structural confidence scores, not measured binding or affinity.; The negatives are putative: constructed from repertoire antibodies or by germline reversion, not shown experimentally not to bind.
origin
author_reported
protocol
poisk-20261115-protocol-pairwise-coverage-25
retrospective
true
review
artifact sha256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924; date: 2026-11-15; method: deterministic-table-parse; pdf-text-parse; source-hash-verification; method note: Extraction pass. Every stored table value was parsed from the pdftotext -layout text layer of the pinned bioRxiv v1 PDF by extract/build_batch.py, which asserts the sha256 of the text layer and then every table caption, row label and column header before reading a cell. Values the paper prints only in prose were transcribed by hand and the script asserts the sentence they come from. No value was read from a figure.; note: Transcribed from the pinned preprint; not rerun.; retrieval url: https://www.biorxiv.org/content/10.64898/2026.10.24.761513v1.full.pdf; reviewer: claude; reviewer note: Claude (Opus 5) extraction agent; no human review claimed. Extraction only: no independent review is claimed and a different agent reviews this batch. Transcription from the pinned source is not reproduction.
source locator
Table 7, row "Boltz-2", column "25%"
version
bioRxiv preprint v1 posted 26 October 2026, retrieved 2026-11-15
Related records

Suggest a correction