Crowdsourced Protein Design: Lessons From the Adaptyv EGFR Binder Competition (Cotet et al., bioRxiv 2025)

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 2 methods
• 6 configurations
• 1 pipeline
• 3 protocols
• 3 datasets
• 12 evaluations
• 18 results
• 9 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• Baseline for the de novo binder design use case: Baseline Directed evolution, display selection or grafting (Adaptyv preprint Sections 2.3 and 3.2 (EGF and cetuximab as positive controls; cetuximab at 9.94 nM); Bennett 2025 Discussion (contrast with yeast display selection from random libraries); AlphaProteo Table 1 caption ('Binders derived from selection-based methods, such as antibodies and nanobodies, are not considered here'))
• Cetuximab, obtained by conventional means, measured 9.94 nM in the same assay: Same assay reference measurement Section 2.3: 'Additionally, EGF and Cetuximab were run as positive controls.' Section 3.2: 'Cetuximab was also used as a positive control in the competition (9.94 nM), allowing for the calculation that this represents an 8.2 x improvement.' Section 4.3: 'Two R2 binders have KD values on par or better than Cetuximab.' Cetuximab is an approved therapeutic antibody obtained by conventional antibody discovery, so its 9.94 nM in this assay is a reference point for what a conventional binder achieves on EGFR, measured on the same instrument as the designs. It is not evidence about the hit rate of conventional methods, because no library selection campaign was run and no candidate set was tested. (Sections 2.3, 3.2 and 4.3)
• Per-method hit rates cannot be read from the Adaptyv round 2 data: Per method aggregation not stated results/result_summary.csv at commit fc91b91ddc367830b755b215dbc69669675ad6a0 has 402 data rows and the columns username, name, kd, binding_strength, binding, expression, sequence, dna, pae_interaction, esm_pll, iptm, plddt, similarity_check and design_models. The paper states 400 designs were tested in round 2, so the row count does not match. design_models is the entrant's own free-text declaration (for example '["Custom PLM"]'); Section 4.2 states the organisers 'manually curated and categorized' entrants' workflow summaries into five design choices, so method identity is self-declared and then curated. Neither the README's Methods section nor the preprint states how to aggregate these rows into a per-method or per-category rate, and Section 4.1 reports Figure 8's binder counts 'across the 3 binding thresholds' without defining them. Under docs/collection.md a value may be computed from a source's own rows only where the source states the aggregation, so the per-method rates in Figures 3, 8 and 9 and the 60 affinities in Figure 10 remain unextracted. (results/result_summary.csv header and row count; README.md 'Methods' and 'Data analysis'; preprint Sections 4.1 and 4.2)
• Relevance of uc-binder-20261022-protocol-adaptyv-bli-affinity to "Which computational design workflow produces binders that work when they are made and tested?": Assessed by Adaptyv EGFR competition BLI affinity (uc-binder-20261022-source-adaptyv-preprint: Section 2.3; Sections 3.1, 3.2 and 4.3; results/result_summary.csv; uc-binder-20261022-source-adaptyv-repo-round2: Section 2.3; Sections 3.1, 3.2 and 4.3; results/result_summary.csv)
• Relevance of uc-binder-20261022-protocol-adaptyv-bli-binding to "Which computational design workflow produces binders that work when they are made and tested?": Assessed by Adaptyv EGFR competition BLI binding screen (uc-binder-20261022-source-adaptyv-preprint: Sections 2.3 and 2.4; round 2 repository README, 'Experimental Workflow'; uc-binder-20261022-source-adaptyv-repo-round2: Sections 2.3 and 2.4; round 2 repository README, 'Experimental Workflow')
• Relevance of uc-binder-20261022-protocol-adaptyv-expression to "Which computational design workflow produces binders that work when they are made and tested?": Assessed by Adaptyv EGFR competition cell-free expression rate (uc-binder-20261022-source-adaptyv-preprint: Sections 2.4 and 4.1; round 2 repository README, 'Protein Synthesis')
• Seven of the eight de novo binders in Adaptyv round 2 were designed with BindCraft: Share of de novo binders Section 3.1, verbatim: 'Overall, 7 out of 8 de novo binders in the second round were designed using BindCraft, despite most de novo designs in the competition originating from the RFDiffusion pipeline.' The paper does not print how many RFdiffusion designs were tested, so no RFdiffusion hit rate can be read from this source. Section 3.1 also states that 'the best R2 BindCraft design does not have any significantly similar structural hits in either the Protein Data Bank (PDB) or AlphaFold Protein Structure Database (AFDB)'. (Section 3.1)
• Summary of the de novo binder design evidence: Summary Four studies, pinned here as seven artifacts, carry measured outcomes for de novo binder design, and none of them answers which workflow to use, because no paper runs two design methods and a conventional comparator on the same targets with one assay and one hit definition. The literature pass behind this evidence found nine candidate sources with measured outcomes; the five that were not retrieved are recorded as gaps rather than as evidence of what they would show. What the measured evidence shows is that the target matters more than the method. AlphaProteo's yeast-display success rate across eight targets runs from 0% over 54 designs on TNF-alpha to 88% over 94 designs on BHRF1, at a binding-signal threshold of 0.2 (1.3 on IL-17A). BindCraft reports an average experimental success rate of 46.3% over twelve targets with a range of 10% to 100%, built from per-target counts as small as 2 binders from 7 designs on Bet v1 and as large as 6 from 6 on the SpCas9 REC1 domain. The same family of methods applied to antibody variable domains is two orders of magnitude worse: fine-tuned RFdiffusion gives 2% over 96 designs on TcdB, 0% over 96 on IL-7 receptor alpha, and 0.01% to 0.2% over 9,000 designs per target by yeast display. On influenza H1 stem the same design campaign gives 1% over 96 designs by SPR and 0.2% over 9,000 by display, so the screening method moves the rate as much as the target does. Nearly all of this is author-reported by the group that built the method, which also chose the targets, ran the assay and chose which designs to test. The exception is the Adaptyv EGFR competition, where a third-party laboratory expressed and measured designs from many entrants on one target and one assay: 2.5% of 201 round-1 designs bound (printed elsewhere in the same paper as 3%), and 53 of 400 round-2 designs bound, printed as 13%. That source cannot be turned into a per-method rate: design methods are the entrants' own declarations, the organisers hand-picked 101 of 201 round-1 and 300 of 400 round-2 designs for diversity, and the three binding thresholds behind the figure counts are never defined. Affinities reached are better than the hit rates suggest. AlphaProteo's best measured binder was 0.082 nM on IL-7 receptor alpha by HTRF; BindCraft's best ranged from 5 nM on CbAgo to 5.7 micromolar on CrSAS-6; the best de novo design in the Adaptyv competition reached 82 nM and the winning optimised cetuximab fragment 1.21 nM, against 9.94 nM for the cetuximab positive control measured on the same instrument. Three things stop these numbers from ranking methods. The hit definitions are different endpoints, not different thresholds on one endpoint. The denominators are small and chosen, and the designs tested are the top picks of each method's own filter, so design and selection are coupled: changing only AlphaProteo's filter moved its PD-L1 rate from 9.6% over 94 designs to 16.1% over 31, and the Bennett authors attribute their low antibody rates partly to a filter they describe as weak. And the spread within one method across targets is wider than any plausible difference between methods. No conventional comparator is run anywhere in this evidence, so there is no measurement of what a designed binder is being compared against. (AlphaProteo Table 1, Table S2 and Sections S1.2.1, S2 and S4.1; BindCraft Introduction and Results prose; Bennett Supplementary Methods Table 6; Adaptyv Sections 2.3, 2.4, 3.1, 3.2, 4.1, 4.2 and 4.3)
• The share of true binders from hallucination designs fell between the Adaptyv rounds: Design category share of binders Section 4.2, verbatim: 'Interestingly, hallucinated designs, despite becoming more preferred in the second round, had a lower share of true binders (from 40% to 12%, Figure 9 C).' These are shares of the binder set by design category, not hit rates: the number of designs submitted or tested in each category is printed only in Figure 9, which was not read. The five curated categories are diversified binders, optimized binders, physics-based, de novo and hallucination. (Section 4.2; Figure 9 caption)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.