Only 9% of the ordered Bennett VHH designs have AlphaFold3 ipTM above 0.6
Filter power of Fine-tuned RFdiffusion VHH design with fine-tuned RoseTTAFold2 filtering at hotspot proportion 1
Results 'Improved oracles increase success rate', verbatim: 'We found that the AlphaFold3 interface predicted template modelling (ipTM) score, a measure of model confidence over the interface, is predictive of binding success (area under the curve = 0.86; Extended Data Fig. 10a,b). Overall, only 9% of our ordered VHH designs have an ipTM > 0.6, suggesting that success rate will be improved by incorporation of an ipTM filter.' And for the scFv libraries: 'Only 4% of the initial design library has ipTM > 0.85, whereas 5 out of the 6 experimentally confirmed designs pass this threshold.' The paper also states that the fine-tuned RoseTTAFold2 filter's 'filtering power ... is limited (at least with the settings used; providing 100% of interface hotspots)' and that this 'probably accounts for the low experimental success rates'.
The paper's retrospective analysis finds that AlphaFold3 interface pTM separates binders from non-binders with an area under the curve of 0.86, and that only 9% of the ordered VHH designs have ipTM above 0.6, so the tested set was chosen by a filter the authors now consider weak.
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
6 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Claimed field filter_power Context-only references | Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) Results 'Improved oracles increase success rate' Version: Nature, version of record; Europe PMC JATS full text for PMC12727541 | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Source location Results 'Improved oracles increase success rate' Context-only references | Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) Results 'Improved oracles increase success rate' Version: Nature, version of record; Europe PMC JATS full text for PMC12727541 | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed value Results 'Improved oracles increase success rate', verbatim: 'We found that the AlphaFold3 interface predicted template modelling (ipTM) score, a measure of model confidence over the interface, is predictive of binding success (area under the curve = 0.86; Extended Data Fig. 10a,b). Overall, only 9% of our ordered VHH designs have an ipTM > 0.6, suggesting that success rate will be improved by incorporation of an ipTM filter.' And for the scFv libraries: 'Only 4% of the initial design library has ipTM > 0.85, whereas 5 out of the 6 experimentally confirmed designs pass this threshold.' The paper also states that the fine-tuned RoseTTAFold2 filter's 'filtering power ... is limited (at least with the settings used; providing 100% of interface hotspots)' and that this 'probably accounts for the low experimental success rates'. Context-only references | Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) Results 'Improved oracles increase success rate' Version: Nature, version of record; Europe PMC JATS full text for PMC12727541 | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description The paper's retrospective analysis finds that AlphaFold3 interface pTM separates binders from non-binders with an area under the curve of 0.86, and that only 9% of the ordered VHH designs have ipTM above 0.6, so the tested set was chosen by a filter the authors now consider weak. Context-only references | Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) Results 'Improved oracles increase success rate' Version: Nature, version of record; Europe PMC JATS full text for PMC12727541 | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Relationship: subject uc-binder-20261022-config-bennett-rfdiffusion-vhh Context-only references | Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) Results 'Improved oracles increase success rate' Version: Nature, version of record; Europe PMC JATS full text for PMC12727541 | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Name Only 9% of the ordered Bennett VHH designs have AlphaFold3 ipTM above 0.6 Context-only references | Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) Results 'Improved oracles increase success rate' Version: Nature, version of record; Europe PMC JATS full text for PMC12727541 | not individually reviewed No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-10-22-1b7cf027e4f6 · Record review: source checked
1 source record and release history
- Atomically accurate de novo design of antibodies with RFdiffusion (Bennett et al., Nature 2025) · Original source · Nature, version of record; Europe PMC JATS full text for PMC12727541
Technical metadata and extraction receipts
Stable ID: uc-binder-20261022-claim-bennett-iptm-filter-retrospective
- field
- filter_power
- review
- artifact sha256: d7876bc73c47f2cb3fcb0fa1ef917c89c9b67750a22f6402bf515f978cd1564a; method: source-hash-verification; independent-cell-check; ai-assisted-source-review; method note: Re-downloaded from Europe PMC (HTTP 200, 169,840 bytes) and the SHA-256 recomputed and matched. Own flattening of the JATS XML. The main text's '95 designs per target' was confirmed verbatim and the string '96 designs' does not occur, so the conflict with Supplementary Methods Table 6 is real.; note: Reviewed in docs/reviews/use-cases/binder-2026-10-22.md. Automated AI review of the source transcription only; it is not human scientific review and nothing here was reproduced experimentally.; retrieval url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12727541/fullTextXML; reviewed at: 2026-10-22T15:00:00Z; reviewer: claude; reviewer note: Claude review agent, independent of the extraction agent; no human review claimed
- source locator
- Results 'Improved oracles increase success rate'
- value
- Results 'Improved oracles increase success rate', verbatim: 'We found that the AlphaFold3 interface predicted template modelling (ipTM) score, a measure of model confidence over the interface, is predictive of binding success (area under the curve = 0.86; Extended Data Fig. 10a,b). Overall, only 9% of our ordered VHH designs have an ipTM > 0.6, suggesting that success rate will be improved by incorporation of an ipTM filter.' And for the scFv libraries: 'Only 4% of the initial design library has ipTM > 0.85, whereas 5 out of the 6 experimentally confirmed designs pass this threshold.' The paper also states that the fine-tuned RoseTTAFold2 filter's 'filtering power ... is limited (at least with the settings used; providing 100% of interface hotspots)' and that this 'probably accounts for the low experimental success rates'.