rewirebio.iobenchmarks
← Browse all evidence claims
Evidence claim

Relevance of structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling to "Which predicted interfaces or structures are reliable enough to guide my next experiment?"

Use case
Choose structural hypotheses to guide experiments
Assessed by
Best DockQ against sampling depth on cognate nanobody-antigen complexes
Relevance
Proxy evidence
Endpoint
Median best DockQ among 1 and among 100 samples for AlphaFold3, Boltz-2, Boltz-1 and Chai-1 on 106 nanobody-antigen complexes

Shows how much sampling raises the best available nanobody-antigen model for four tools. Proxy evidence: the best sample is picked with the experimental structure, the set mixes training and held-out systems, and the source is a preprint.

4 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

20 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-02-a7f03c9e55a5
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    structural-20261009-source-smorodina2026
    locator
    Results P33 to P36; Methods P83 and P91; Figure 5B
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Comparison group
nanobody-antigen-interfaces
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Comparison title
Nanobody-antigen interface predictions (preprint)
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Constraints
4 values
  • Inspect every linked evaluation's source locator and preserved limitations before citing a result.
  • Do not combine this mapping's evaluations with any other protocol's results; complex sets, sampling and metrics differ.
  • Results for one complex class do not establish performance in another.
  • Structural accuracy and confidence are not evidence that an interaction exists, how strong it is or that an experiment will work.
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Endpoint
Median best DockQ among 1 and among 100 samples for AlphaFold3, Boltz-2, Boltz-1 and Chai-1 on 106 nanobody-antigen complexes
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Claimed field
links:assessed_by:structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Headline metric
dockq
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Limitations
10 values
  • Best-of-N DockQ needs the experimental structure to pick the best sample, so it is an upper bound on what a user could select.
  • Each sampling depth is a separate run with its own seed (seeds 1 to 5 for N = 1, 10, 25, 50 and 100), unlike the single 50-sample runs of the other protocols.
  • For Chai-1 the smallest depth is 5 samples (5 trunk samples x 1 diffusion sample), so its N = 1 value is the best of 5.
  • Boltz-1 appears only in this analysis.
  • Nanobody (VHH)-antigen complexes only; results do not transfer to conventional antibodies or other complex classes.
  • The set mixes systems inside and outside each tool's training data (AF3 30 of 106 in training, Chai-1 25, Boltz-2 64); the printed values are over all systems, so they are not post-cutoff results, and Boltz-2's are the most exposed.
  • Preprint, not peer reviewed. Supplementary tables were not read.
  • No uncertainty is printed for these values.
  • The dataset paragraph (P62) says post-October 2021 depositions were kept, 'corresponding to the earliest training cutoff among the evaluated tools (Boltz-2)'. The methods (P76) give Chai-1 about 12 January 2021, AF3 about 30 September 2021 and Boltz-2 about 1 June 2023, so Chai-1 is the earliest and Boltz-2 the latest, and the October 2021 date matches AF3. The per-tool train and test counts follow the methods, and no printed value depends on the mis-stated sentence.
  • Preprint, not peer reviewed.
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Pinned records
15 values
  • record id
    structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-af3
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    d54f6bff06ec0bed3da27817c838489e63f5fadf13a18e418f24226b1589f122
  • record id
    structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-boltz1
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    fe64cec44a87893752b3d9ece48b2ac833172d9062d622e4fdbe80729d8be2c0
  • record id
    structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-boltz2
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    068591522daeb20f9d93c2cbc2311bd9f8d14dab5642b9a728c2b5ecf79795be
  • record id
    structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-chai1
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    c05db467bf9cb89f59d41b2ef2d04217b4f9a95ce37725eb02d54accde448e6c
  • record id
    structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    1daccdfa9f94ed30f52d7ce68352d0bc288ccfec15c699bf3a48a8c2b938434e
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-af3-n1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    4d394c7bb40a2f7862a8bd77e2a596b9f466bc2f702e78a50ab8b876606f3639
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-af3-n100
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    315074c0641cfae3faf55dcc5905789d962d824d43ec4ac7adf9a1e6f0a54be9
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz1-n1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    08e670ccf069271c8e8b7e5d045a73c717eca141a6ffaa50b730b52880e6a473
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz1-n100
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    d248e1d291622d31c2398fcaf5c4fca86e9312d699bfe746454f78e976b31f62
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz2-n1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    38153765a422b3f17e288afecc96048fc1792f69d19b6dbe02bab8a2b7937177
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz2-n100
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    36e92e43497c6cb8bfafd5a2580b9cca92682ebdce9fbf3f2a6c08cce8491448
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-chai1-n1
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    2083bb1a19b3eea228254f635af09941c2d50e0305f9cbec6a587542b20244e1
  • record id
    structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-chai1-n100
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    12905b5bbf93a606d897be256cd3153d91eb6e25cbfeab85c61872d579a1ffc6
  • record id
    structural-20261009-source-smorodina2026
    fields
    • status
    • attributes.artifact_sha256
    sha256
    36153b6f7f2004b2450c64593497cffb16a6fe6676875e5b986458cef5c96d93
  • record id
    use-case-structural-hypotheses-experiments
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    323c287cae7885e983feb5d58f24115afe49ecfbb868a0238c75b1247ffcf228
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

Rationale
Shows how much sampling raises the best available nanobody-antigen model for four tools. Proxy evidence: the best sample is picked with the experimental structure, the set mixes training and held-out systems, and the source is a not yet peer reviewed.
Context-only references		Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores

Original source ↗

structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B

Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Retrieved: 2026-10-09T21:25:48Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: 0ad24054d98fd1888fa8bf3120f76e22f7bf6b1965261b16f045986d72c50732

Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-02-a7f03c9e55a5 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-structural-20261009-smorodina2026-sampling

field
links:assessed_by:structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling
value
structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling
relevance
proxy
endpoint
Median best DockQ among 1 and among 100 samples for AlphaFold3, Boltz-2, Boltz-1 and Chai-1 on 106 nanobody-antigen complexes
rationale
Shows how much sampling raises the best available nanobody-antigen model for four tools. Proxy evidence: the best sample is picked with the experimental structure, the set mixes training and held-out systems, and the source is a not yet peer reviewed.
constraints
Inspect every linked evaluation's source locator and preserved limitations before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; complex sets, sampling and metrics differ.; Results for one complex class do not establish performance in another.; Structural accuracy and confidence are not evidence that an interaction exists, how strong it is or that an experiment will work.
limitations
Best-of-N DockQ needs the experimental structure to pick the best sample, so it is an upper bound on what a user could select.; Each sampling depth is a separate run with its own seed (seeds 1 to 5 for N = 1, 10, 25, 50 and 100), unlike the single 50-sample runs of the other protocols.; For Chai-1 the smallest depth is 5 samples (5 trunk samples x 1 diffusion sample), so its N = 1 value is the best of 5.; Boltz-1 appears only in this analysis.; Nanobody (VHH)-antigen complexes only; results do not transfer to conventional antibodies or other complex classes.; The set mixes systems inside and outside each tool's training data (AF3 30 of 106 in training, Chai-1 25, Boltz-2 64); the printed values are over all systems, so they are not post-cutoff results, and Boltz-2's are the most exposed.; Preprint, not peer reviewed. Supplementary tables were not read.; No uncertainty is printed for these values.; The dataset paragraph (P62) says post-October 2021 depositions were kept, 'corresponding to the earliest training cutoff among the evaluated tools (Boltz-2)'. The methods (P76) give Chai-1 about 12 January 2021, AF3 about 30 September 2021 and Boltz-2 about 1 June 2023, so Chai-1 is the earliest and Boltz-2 the latest, and the October 2021 date matches AF3. The per-tool train and test counts follow the methods, and no printed value depends on the mis-stated sentence.; Preprint, not peer reviewed.
citation locators
source id: structural-20261009-source-smorodina2026; locator: Results P33 to P36; Methods P83 and P91; Figure 5B
source locator
structural-20261009-source-smorodina2026: Results P33 to P36; Methods P83 and P91; Figure 5B
revision
1
reason
Add comparisons of structure-prediction reliability by complex class from the structural hypotheses use-case pass 2026-10-09.
comparison group
nanobody-antigen-interfaces
comparison title
Nanobody-antigen interface predictions (preprint)
headline metric
dockq
stratum label
Best of N samples
stratum order
1
reviewed evaluations
structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-af3; structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-boltz1; structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-boltz2; structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-chai1
review
method: automated-source-review; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-10T00:30:00Z; note: Independent review of endpoint fit, relevance grade, limitations, exclusions and grouping against the use case and the preprint; every value of the listed evaluations was checked against the paragraph its locator names. Preprint; no execution, reproduction, human scientific review or clinical validation.
pins
record id: structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-af3; fields: status; links; attributes.origin; attributes.comparison; sha256: d54f6bff06ec0bed3da27817c838489e63f5fadf13a18e418f24226b1589f122; record id: structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-boltz1; fields: status; links; attributes.origin; attributes.comparison; sha256: fe64cec44a87893752b3d9ece48b2ac833172d9062d622e4fdbe80729d8be2c0; record id: structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-boltz2; fields: status; links; attributes.origin; attributes.comparison; sha256: 068591522daeb20f9d93c2cbc2311bd9f8d14dab5642b9a728c2b5ecf79795be; record id: structural-20261009-eval-smorodina2026-vhh-dockq-vs-sampling-chai1; fields: status; links; attributes.origin; attributes.comparison; sha256: c05db467bf9cb89f59d41b2ef2d04217b4f9a95ce37725eb02d54accde448e6c; record id: structural-20261009-protocol-smorodina2026-vhh-dockq-vs-sampling; fields: status; name; description; facets; source_ids; links; attributes; sha256: 1daccdfa9f94ed30f52d7ce68352d0bc288ccfec15c699bf3a48a8c2b938434e; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-af3-n1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 4d394c7bb40a2f7862a8bd77e2a596b9f466bc2f702e78a50ab8b876606f3639; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-af3-n100; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 315074c0641cfae3faf55dcc5905789d962d824d43ec4ac7adf9a1e6f0a54be9; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz1-n1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 08e670ccf069271c8e8b7e5d045a73c717eca141a6ffaa50b730b52880e6a473; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz1-n100; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: d248e1d291622d31c2398fcaf5c4fca86e9312d699bfe746454f78e976b31f62; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz2-n1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 38153765a422b3f17e288afecc96048fc1792f69d19b6dbe02bab8a2b7937177; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-boltz2-n100; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 36e92e43497c6cb8bfafd5a2580b9cca92682ebdce9fbf3f2a6c08cce8491448; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-chai1-n1; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2083bb1a19b3eea228254f635af09941c2d50e0305f9cbec6a587542b20244e1; record id: structural-20261009-result-smorodina2026-vhh-dockq-vs-sampling-chai1-n100; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 12905b5bbf93a606d897be256cd3153d91eb6e25cbfeab85c61872d579a1ffc6; record id: structural-20261009-source-smorodina2026; fields: status; attributes.artifact_sha256; sha256: 36153b6f7f2004b2450c64593497cffb16a6fe6676875e5b986458cef5c96d93; record id: use-case-structural-hypotheses-experiments; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 323c287cae7885e983feb5d58f24115afe49ecfbb868a0238c75b1247ffcf228
Related records

Suggest a correction