Top-1 accuracy (retrieval; one patent-supported binder ranked against 49 putative decoys for the same antigen by full-Fv ipSAE, mean over 27 antigen constructs, 20 trials per construct per seed, 25 seeds) of OpenFold3 with full-Fv ipSAE ranking on POISK 1-in-50 screening targets (27 antigen constructs)
11.6% Top-1 accuracy (retrieval; one patent-supported binder ranked against 49 putative decoys for the same antigen by full-Fv ipSAE, mean over 27 antigen constructs, 20 trials per construct per seed, 25 seeds)
Methods
- Tested configuration
- OpenFold3 with full-Fv ipSAE ranking
- Protocol
- POISK 1-in-50 screening simulation, top-1 retrieval
- Dataset subset
- POISK 1-in-50 screening targets (27 antigen constructs)
- Procedure
- POISK 1-in-50 screening simulation, top-1 retrieval
- Evaluation
- OpenFold3 on POISK 1-in-50 screening simulation, top-1 retrieval
- Coverage
- 540/540 discovery trials per seed scored, 25 repeats
- Uncertainty
- SD 1.6. Table 8 also prints a standard deviation across targets for this engine (11.4 percentage points), which measures between-target spread rather than run-to-run spread and is kept as a separate claim record.
- Evidence
- Author-reported evaluation · source checkedTupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF) · Table 8, row "Mean over targets", column "OF3"
A source-checked result verifies the numerical transcription, not every model or protocol detail. Evaluation metadata: source checked. Source checked does not mean independently reproduced.
Reproduction
- Split
- Evaluation only; the benchmark is scored in full and nothing is held out for training
- Adaptation
- Zero-shot: no model was trained or fine-tuned on POISK
- Scoring implementation
- Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.
No execution recipe has been verified for this exact configuration and evaluation. A benchmark's general instructions may use different inputs, splits or model settings.
Reproducing this published result requires matching its model configuration, data, split and scorer. Source checking or a successful smoke test does not establish score reproduction.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Reported result 11.6 Individual claims | Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF) Table 8, row "Mean over targets", column "OF3" Version: v1, posted 26 October 2026 | source checked ["independent-cell-check","deterministic-table-parse","pdf-text-parse","source-hash-verification"] · 2026-11-16T00:00:00Z author reported Audit detailsColumn assignment re-checked from the character geometry of the table's own header line. Checked against the archived pdftotext -layout text layer of the pinned v1 PDF, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696. The PDF container itself is not byte-stable at bioRxiv, so the source record's artifact_sha256 (526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924) does not reproduce; the text layer does, from three independent downloads. The value was also confirmed against the version 2 PDF posted 2026-11-14, whose tables print the same numbers. Field: Source artifact SHA-256: Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from. Format: application/pdf, 16 pages Extraction artifact SHA-256: |
Sources and history
Release 2026-11-21-c8bea25e5abb · Record review: source checked
1 source record and release history
- Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF) · Original source · v1, posted 26 October 2026
Technical metadata and extraction receipts
Stable ID: poisk-20261115-result-openfold3-screen-1in50-top1-t8
- coverage
- eligible: 540; repeats: 25; scored: 540; unit: discovery trials per seed
- denominator
- 540
- metric
- top-1-accuracy
- metric direction
- higher
- metric qualifier
- retrieval; one patent-supported binder ranked against 49 putative decoys for the same antigen by full-Fv ipSAE, mean over 27 antigen constructs, 20 trials per construct per seed, 25 seeds
- numeric value
- 11.6
- printed value
- 11.6
- replicates or seeds
- 25 scenario-generation seeds, 20 trials per target per seed
- review
- artifact sha256: e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696; method: independent-cell-check; deterministic-table-parse; pdf-text-parse; source-hash-verification; method note: Independent review pass. All five pinned artifacts were re-retrieved. The two Zenodo API responses, the explorer page and the bioRxiv v1 JATS XML all hashed to the values on their source records. The v1 PDF did not: three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once, none equal to the recorded artifact_sha256, because bioRxiv regenerates the container on request. A pdftotext -layout run (poppler 26.08.0) on all three downloads gave the same text layer, byte for byte the contents of the archived artifacts/biorxiv-v1-layout.txt.gz, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, so the archive is provably derived from the pinned article and is the artifact this review verified values against. npm run batch -- check-values reproduced the extractor's figures exactly (36 in_cell, 36 in_row, 6 locator_not_parsed, no wrong_column and no overclaimed_in_row) and wrote an identical value-check.tsv. All 78 results were then re-read by review/check_values_independent.py, written for this pass and importing nothing from extract/build_batch.py: it takes each table's column spans from the character offsets of the label text in that table's own printed header lines, assigns every numeric token on a body row to the column whose centre is nearest and rejects the row unless the token count matches the column count, the nearest column is the one in the token's own left-to-right position, and the runner-up column loses by at least three characters; it also checks that Table 1's sub-columns sit under their group labels and that Table 7's Coverage group does not reach the Ties column. Records are matched to cells through each result's source_locator text, not its ID. 72 results were checked in cell, including all 36 the repository parser could only place at row level, and the 6 prose values were checked against the sentence each locator names. Two negative controls confirmed the checker fails on a swapped pair of values and on a value shifted into a neighbouring column of the same row, which a row-level check passes. Independently of the script, all 42 printed cells of Tables 2 and 7 were transcribed by hand into review/manual-transcription.tsv and compared with the records, covering column assignment for every one of the 36 cells left unverified at extraction. Version 2 of the preprint, posted 2026-11-14, was retrieved (PDF SHA-256 aeaec593d7aa2d42382a49bd6fadbbccabaa6c084843ea1c7f0c91097dedb0c2, text layer 82a99bc2e061b7247cf65f7bc9450924b7af5be7ab3ef283d2284ba787ea629c): every numeric token in it is identical to v1's apart from an added author affiliation and the two Zenodo record identifiers, so no stored value changes and the batch stays pinned to v1. The 40 descriptive claims were read against the pinned PDF text layer, both Zenodo API responses and the explorer page.; note: Column assignment re-checked from the character geometry of the table's own header line. Checked against the archived pdftotext -layout text layer of the pinned v1 PDF, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696. The PDF container itself is not byte-stable at bioRxiv, so the source record's artifact_sha256 (526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924) does not reproduce; the text layer does, from three independent downloads. The value was also confirmed against the version 2 PDF posted 2026-11-14, whose tables print the same numbers.; retrieval url: https://www.biorxiv.org/content/10.64898/2026.10.24.761513v1.full.pdf; reviewed at: 2026-11-16T00:00:00Z; reviewer: claude; reviewer note: Claude review agent (independent of the extractor); no human review claimed. Transcription was checked against the pinned source; nothing was reproduced and no value was recomputed.
- scope note
- Mean over the 27 antigen constructs. Table 8's per-construct values range from 2.2% to 70.8% for Protenix-v2, so this aggregate hides a wide spread of screening difficulty.
- source locator
- Table 8, row "Mean over targets", column "OF3"
- uncertainty
- n: 25; note: Table 8 also prints a standard deviation across targets for this engine (11.4 percentage points), which measures between-target spread rather than run-to-run spread and is kept as a separate claim record.; printed: 1.6; scope: sample standard deviation of the per-seed means across the 25 scenario-generation seeds, in percentage points; Figure 5 plots plus or minus twice this quantity; source column: Table 8 summary row "SD across seeds (pp)"; type: standard_deviation; unit: percentage points; value: 1.6
- unit
- percent