rewirebio.iobenchmarks
Evidence claim

POISK pair coverage under cross-model structural consensus: metric_implementation

Metric implementation of POISK pair coverage under cross-model structural consensus

Cross-model pose agreement: DockQ scores are computed for the Fv-antigen interface of each model's predicted complex, and consensus is the stated number of models predicting the same pose above the stated DockQ threshold. The paper states the decision rule as "For any given pair, a prediction was considered successful if the binder demonstrated structural agreement amongst more models than its decoy". The pair is decided by which member has agreement among more models, not by an interface confidence score; ipSAE does not enter this endpoint.

Correction made by the independent review of the POISK batch. The previous value is kept here.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

7 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-c8bea25e5abb
Property and statementOriginal source and locationReview and provenance
Claimed field
metric_implementation
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Previous value
Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.previous_value

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Source location
Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.source_locator

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Claimed value
Cross-model pose agreement: DockQ scores are computed for the Fv-antigen interface of each model's predicted complex, and consensus is the stated number of models predicting the same pose above the stated DockQ threshold. The paper states the decision rule as "For any given pair, a prediction was considered successful if the binder demonstrated structural agreement amongst more models than its decoy". The pair is decided by which member has agreement among more models, not by an interface confidence score; ipSAE does not enter this endpoint.
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.value

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Description
Correction made by the independent review of the POISK batch. The previous value is kept here.
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: description

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Relationship: subject
poisk-20261115-protocol-consensus-pair-coverage
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: links:subject:poisk-20261115-protocol-consensus-pair-coverage

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Name
POISK pair coverage under cross-model structural consensus: metric_implementation
Context-only references
Tupikina et al. 2026, Can Folding Models Tell Binders from Bluffers? Evidence from POISK (bioRxiv v1 PDF)

Original source ↗

Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"

Version: v1, posted 26 October 2026
Retrieved: 2026-11-15T23:32:24Z

not individually reviewed

No individual claim review recorded

Audit details

Field: name

Source artifact SHA-256: 526def150fb5ae8f9ac4533efcf16c86e547e636cdb9bfba9a574cd1e2a2f924

Hash scope: One download of 2026-11-15. The PDF container is not byte-stable: bioRxiv regenerates it on request, so artifact_sha256 is that one download's hash and no refetch reproduces it. Three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once. All three yield the same pdftotext -layout text layer, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, which is the article's stable identity, is archived gzipped in artifacts/biorxiv-v1-layout.txt.gz under archive_sha256, is what extract/build_batch.py asserts, and is what every stored value was read from.

Format: application/pdf, 16 pages

Sources and history

Release 2026-11-21-c8bea25e5abb · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: metadata-correction-fb8648245db333adbca9

areas
molecular-interactions; proteins-complexes
field
metric_implementation
previous value
Full-Fv ipSAE: the interaction prediction Score from Aligned Errors computed asymmetrically with the paired antibody VH and VL concatenated as one chain against the target (VH+VL to Target), with a PAE cutoff of 10 angstrom and a distance cutoff of 10 angstrom, using a modified implementation of the ipSAE tool. It is a structural confidence score used here to rank candidates; it is not a measured binding or affinity value.
review
artifact sha256: e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696; method: independent-cell-check; deterministic-table-parse; pdf-text-parse; source-hash-verification; method note: Independent review pass. All five pinned artifacts were re-retrieved. The two Zenodo API responses, the explorer page and the bioRxiv v1 JATS XML all hashed to the values on their source records. The v1 PDF did not: three downloads on 2026-11-16 gave 3389c1bf6d3654c5b2151a07fcfcefc67cf6f36e5f0facfc03c33521d8b84491 twice and d699fd680e3b0097ca80130c288e8d872773dc1e523044f4605e06fc4f89d9bc once, none equal to the recorded artifact_sha256, because bioRxiv regenerates the container on request. A pdftotext -layout run (poppler 26.08.0) on all three downloads gave the same text layer, byte for byte the contents of the archived artifacts/biorxiv-v1-layout.txt.gz, SHA-256 e75c18130ca43770d2f0ed05ae96528759666c0e4fa08f7725f5a32b21187696, so the archive is provably derived from the pinned article and is the artifact this review verified values against. npm run batch -- check-values reproduced the extractor's figures exactly (36 in_cell, 36 in_row, 6 locator_not_parsed, no wrong_column and no overclaimed_in_row) and wrote an identical value-check.tsv. All 78 results were then re-read by review/check_values_independent.py, written for this pass and importing nothing from extract/build_batch.py: it takes each table's column spans from the character offsets of the label text in that table's own printed header lines, assigns every numeric token on a body row to the column whose centre is nearest and rejects the row unless the token count matches the column count, the nearest column is the one in the token's own left-to-right position, and the runner-up column loses by at least three characters; it also checks that Table 1's sub-columns sit under their group labels and that Table 7's Coverage group does not reach the Ties column. Records are matched to cells through each result's source_locator text, not its ID. 72 results were checked in cell, including all 36 the repository parser could only place at row level, and the 6 prose values were checked against the sentence each locator names. Two negative controls confirmed the checker fails on a swapped pair of values and on a value shifted into a neighbouring column of the same row, which a row-level check passes. Independently of the script, all 42 printed cells of Tables 2 and 7 were transcribed by hand into review/manual-transcription.tsv and compared with the records, covering column assignment for every one of the 36 cells left unverified at extraction. Version 2 of the preprint, posted 2026-11-14, was retrieved (PDF SHA-256 aeaec593d7aa2d42382a49bd6fadbbccabaa6c084843ea1c7f0c91097dedb0c2, text layer 82a99bc2e061b7247cf65f7bc9450924b7af5be7ab3ef283d2284ba787ea629c): every numeric token in it is identical to v1's apart from an added author affiliation and the two Zenodo record identifiers, so no stored value changes and the batch stays pinned to v1. The 40 descriptive claims were read against the pinned PDF text layer, both Zenodo API responses and the explorer page.; note: The protocol carried the full-Fv ipSAE implementation text, which describes a scoring step this endpoint does not use. Corrected on the protocol and, with the same text, in comparison.metric_implementation on the evaluations that assess it: poisk-20261115-eval-consensus-k3-dockq-0-23-coverage, poisk-20261115-eval-consensus-k3-dockq-0-80-coverage.; retrieval url: https://www.biorxiv.org/content/10.64898/2026.10.24.761513v1.full.pdf; reviewed at: 2026-11-16T00:00:00Z; reviewer: claude; reviewer note: Claude review agent (independent of the extractor); no human review claimed. Transcription was checked against the pinned source; nothing was reproduced and no value was recomputed.
source locator
Section 3.3.1, sentences beginning "To test this, we calculated DockQ scores" and "For any given pair, a prediction was considered successful"
value
Cross-model pose agreement: DockQ scores are computed for the Fv-antigen interface of each model's predicted complex, and consensus is the stated number of models predicting the same pose above the stated DockQ threshold. The paper states the decision rule as "For any given pair, a prediction was considered successful if the binder demonstrated structural agreement amongst more models than its decoy". The pair is decided by which member has agreement among more models, not by an interface confidence score; ipSAE does not enter this endpoint.
Related records

Suggest a correction