Relevance of structural-20261009-protocol-smorodina2026-vhh-iptm-dockq-calibration to "Which predicted interfaces or structures are reliable enough to guide my next experiment?"
- Use case
- Choose structural hypotheses to guide experiments
- Assessed by
- How well ipTM tracks DockQ on cognate nanobody-antigen complexes
- Relevance
- Proxy evidence
- Endpoint
- Pearson correlation of ipTM with DockQ, share of confident failures and unconfident successes, and correlation of ipTM change with DockQ change under sampling, for AlphaFold3, Boltz-2 and Chai-1 on 106 nanobody-antigen complexes
Measures whether each tool's confidence tracks its structural accuracy on nanobody complexes, which decides whether a confident model can be trusted. Proxy evidence: cognate complexes only, mixed training exposure, preprint.
3 reviewed evaluations
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
20 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
| Comparison group nanobody-antigen-interfaces Context-only references | Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
| Comparison title Nanobody-antigen interface predictions (preprint) Context-only references | Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
Constraints4 values
| Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
| Endpoint Pearson correlation of ipTM with DockQ, share of confident failures and unconfident successes, and correlation of ipTM change with DockQ change under sampling, for AlphaFold3, Boltz-2 and Chai-1 on 106 nanobody-antigen complexes Context-only references | Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
| Claimed field links:assessed_by:structural-20261009-protocol-smorodina2026-vhh-iptm-dockq-calibration Context-only references | Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
| Headline metric pearson-correlation Context-only references | Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
Limitations9 values
| Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
Pinned records18 values
| Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
| Rationale Measures whether each tool's confidence tracks its structural accuracy on nanobody complexes, which decides whether a confident model can be trusted. Proxy evidence: cognate complexes only, mixed training exposure, not yet peer reviewed. Context-only references | Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3 Version: bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: JATS XML parse (xml.etree), by extract/extract_structural.py |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: source checked
1 source record and release history
- Structural Plausibility Without Binding Specificity: Limits of AI-Based Antibody-Antigen Structure Prediction Confidence Scores · Original source · bioRxiv version 1, posted 2026-03-03; Europe PMC preprint full text PPR1221387 (manuscript EMS215481); not peer reviewed
Technical metadata and extraction receipts
Stable ID: use-case-mapping-structural-20261009-smorodina2026-calibration
- field
- links:assessed_by:structural-20261009-protocol-smorodina2026-vhh-iptm-dockq-calibration
- value
- structural-20261009-protocol-smorodina2026-vhh-iptm-dockq-calibration
- relevance
- proxy
- endpoint
- Pearson correlation of ipTM with DockQ, share of confident failures and unconfident successes, and correlation of ipTM change with DockQ change under sampling, for AlphaFold3, Boltz-2 and Chai-1 on 106 nanobody-antigen complexes
- rationale
- Measures whether each tool's confidence tracks its structural accuracy on nanobody complexes, which decides whether a confident model can be trusted. Proxy evidence: cognate complexes only, mixed training exposure, not yet peer reviewed.
- constraints
- Inspect every linked evaluation's source locator and preserved limitations before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; complex sets, sampling and metrics differ.; Results for one complex class do not establish performance in another.; Structural accuracy and confidence are not evidence that an interaction exists, how strong it is or that an experiment will work.
- limitations
- Calibration is computed on cognate complexes only; it says nothing about whether ipTM separates binders from non-binders.; Quadrant thresholds (DockQ 0.23, ipTM 0.5) are the source's choice, and the text does not say which sample (best, first or worst) the quadrant shares refer to.; Not every tool has every value printed in the text.; Nanobody (VHH)-antigen complexes only; results do not transfer to conventional antibodies or other complex classes.; The set mixes systems inside and outside each tool's training data (AF3 30 of 106 in training, Chai-1 25, Boltz-2 64); the printed values are over all systems, so they are not post-cutoff results, and Boltz-2's are the most exposed.; Preprint, not peer reviewed. Supplementary tables were not read.; No uncertainty is printed for these values.; The dataset paragraph (P62) says post-October 2021 depositions were kept, 'corresponding to the earliest training cutoff among the evaluated tools (Boltz-2)'. The methods (P76) give Chai-1 about 12 January 2021, AF3 about 30 September 2021 and Boltz-2 about 1 June 2023, so Chai-1 is the earliest and Boltz-2 the latest, and the October 2021 date matches AF3. The per-tool train and test counts follow the methods, and no printed value depends on the mis-stated sentence.; Preprint, not peer reviewed.
- citation locators
- source id: structural-20261009-source-smorodina2026; locator: Results P18 to P22; Figure 3
- source locator
- structural-20261009-source-smorodina2026: Results P18 to P22; Figure 3
- revision
- 1
- reason
- Add comparisons of structure-prediction reliability by complex class from the structural hypotheses use-case pass 2026-10-09.
- comparison group
- nanobody-antigen-interfaces
- comparison title
- Nanobody-antigen interface predictions (preprint)
- headline metric
- pearson-correlation
- stratum label
- Confidence against accuracy
- stratum order
- 2
- reviewed evaluations
- structural-20261009-eval-smorodina2026-vhh-iptm-dockq-calibration-af3; structural-20261009-eval-smorodina2026-vhh-iptm-dockq-calibration-boltz2; structural-20261009-eval-smorodina2026-vhh-iptm-dockq-calibration-chai1
- review
- method: automated-source-review; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-10T00:30:00Z; note: Independent review of endpoint fit, relevance grade, limitations, exclusions and grouping against the use case and the preprint; every value of the listed evaluations was checked against the paragraph its locator names. Preprint; no execution, reproduction, human scientific review or clinical validation.
- pins
- record id: structural-20261009-eval-smorodina2026-vhh-iptm-dockq-calibration-af3; fields: status; links; attributes.origin; attributes.comparison; sha256: 7194b00f635cefaa5079a335a69fddb7a6a15e21c65f2031453d4be3d5132f7d; record id: structural-20261009-eval-smorodina2026-vhh-iptm-dockq-calibration-boltz2; fields: status; links; attributes.origin; attributes.comparison; sha256: e2c9c9fea3cfd3c1ddd8866a198d7fb9bb7ab53b0290b59a866a0a697b8e053a; record id: structural-20261009-eval-smorodina2026-vhh-iptm-dockq-calibration-chai1; fields: status; links; attributes.origin; attributes.comparison; sha256: 26967baa94304ae8eec217f6cf1b3bda23514bbddbbafec4386da8aea6dc9d22; record id: structural-20261009-protocol-smorodina2026-vhh-iptm-dockq-calibration; fields: status; name; description; facets; source_ids; links; attributes; sha256: a7ae56aeda598940bc7686fb0bbb850acaee5a7a8776f189515ffa97c2fe2438; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-af3-best; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 19ca403b0fe3a0b2f770a6bcfd70c713e349f19a894f55dddc42adfc4837f0f1; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-af3-change; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f3641fb4e1235828e0a8730c07d2de799ff797e6a00dc117b18a35018a025a28; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-af3-q2; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 15ad32b0f06b197b7701661d9e9a389e40e4db02c8d6951568cbd9012e00bafc; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-af3-q4; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3070951ea21973b19b1db6d4c8cea3a0e24a2e76b9e3611996ec7a53e1c11daa; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-af3-sample0; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c4a3d0888e2e142ca836b4c5f6776db0d2527d4c449494e4dceefd71f025e196; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-boltz2-best; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7bcb9c6c8ad2f70118757c452ce7a2cc6fe9055a5d1571ac47235f66facbcb03; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-boltz2-change; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 155e0da0109c23f5c80087aae4ae9ed777e2e03c5d6fcbbd4c6285d95408bd51; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-boltz2-q2; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c76d3a4232c2daa63c06fb1be0ed344958c63cd76d0ff055cb1448ca953af8f0; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-boltz2-sample0; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 61c0ab7b6030475e093026b0c055521103f1d208a4e23aa742bd69aff4a18a1c; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-chai1-best; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ed16f370a28c95be9ded48311d0aa18eca5ad6cdcfebc57580cd41e986cdfd3c; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-chai1-change; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 893056f6e779e7aedc81f2dd6b4a5086f7df5512bc64e38fbd25d949807831a0; record id: structural-20261009-result-smorodina2026-vhh-iptm-dockq-calibration-chai1-q4; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: c9c9e74cce2c4fbbd6b0d4c952407c57b40187d67bb6a63acf5757144a2c2b5a; record id: structural-20261009-source-smorodina2026; fields: status; attributes.artifact_sha256; sha256: 36153b6f7f2004b2450c64593497cffb16a6fe6676875e5b986458cef5c96d93; record id: use-case-structural-hypotheses-experiments; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 323c287cae7885e983feb5d58f24115afe49ecfbb868a0238c75b1247ffcf228