Adaptyv EGFR design competition round 2 results repository

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 1 protocol
• 1 dataset
• 1 result
• 4 evidence claims
• 1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Claims

Statements this source supports, each with where in the source it was checked.

• Per-method hit rates cannot be read from the Adaptyv round 2 data: Per method aggregation not stated results/result_summary.csv at commit fc91b91ddc367830b755b215dbc69669675ad6a0 has 402 data rows and the columns username, name, kd, binding_strength, binding, expression, sequence, dna, pae_interaction, esm_pll, iptm, plddt, similarity_check and design_models. The paper states 400 designs were tested in round 2, so the row count does not match. design_models is the entrant's own free-text declaration (for example '["Custom PLM"]'); Section 4.2 states the organisers 'manually curated and categorized' entrants' workflow summaries into five design choices, so method identity is self-declared and then curated. Neither the README's Methods section nor the preprint states how to aggregate these rows into a per-method or per-category rate, and Section 4.1 reports Figure 8's binder counts 'across the 3 binding thresholds' without defining them. Under docs/collection.md a value may be computed from a source's own rows only where the source states the aggregation, so the per-method rates in Figures 3, 8 and 9 and the 60 affinities in Figure 10 remain unextracted. (results/result_summary.csv header and row count; README.md 'Methods' and 'Data analysis'; preprint Sections 4.1 and 4.2)
• Relevance of uc-binder-20261022-protocol-adaptyv-bli-affinity to "Which computational design workflow produces binders that work when they are made and tested?": Assessed by Adaptyv EGFR competition BLI affinity (uc-binder-20261022-source-adaptyv-preprint: Section 2.3; Sections 3.1, 3.2 and 4.3; results/result_summary.csv; uc-binder-20261022-source-adaptyv-repo-round2: Section 2.3; Sections 3.1, 3.2 and 4.3; results/result_summary.csv)
• Relevance of uc-binder-20261022-protocol-adaptyv-bli-binding to "Which computational design workflow produces binders that work when they are made and tested?": Assessed by Adaptyv EGFR competition BLI binding screen (uc-binder-20261022-source-adaptyv-preprint: Sections 2.3 and 2.4; round 2 repository README, 'Experimental Workflow'; uc-binder-20261022-source-adaptyv-repo-round2: Sections 2.3 and 2.4; round 2 repository README, 'Experimental Workflow')
• Summary of the de novo binder design evidence: Summary Four studies, pinned here as seven artifacts, carry measured outcomes for de novo binder design, and none of them answers which workflow to use, because no paper runs two design methods and a conventional comparator on the same targets with one assay and one hit definition. The literature pass behind this evidence found nine candidate sources with measured outcomes; the five that were not retrieved are recorded as gaps rather than as evidence of what they would show. What the measured evidence shows is that the target matters more than the method. AlphaProteo's yeast-display success rate across eight targets runs from 0% over 54 designs on TNF-alpha to 88% over 94 designs on BHRF1, at a binding-signal threshold of 0.2 (1.3 on IL-17A). BindCraft reports an average experimental success rate of 46.3% over twelve targets with a range of 10% to 100%, built from per-target counts as small as 2 binders from 7 designs on Bet v1 and as large as 6 from 6 on the SpCas9 REC1 domain. The same family of methods applied to antibody variable domains is two orders of magnitude worse: fine-tuned RFdiffusion gives 2% over 96 designs on TcdB, 0% over 96 on IL-7 receptor alpha, and 0.01% to 0.2% over 9,000 designs per target by yeast display. On influenza H1 stem the same design campaign gives 1% over 96 designs by SPR and 0.2% over 9,000 by display, so the screening method moves the rate as much as the target does. Nearly all of this is author-reported by the group that built the method, which also chose the targets, ran the assay and chose which designs to test. The exception is the Adaptyv EGFR competition, where a third-party laboratory expressed and measured designs from many entrants on one target and one assay: 2.5% of 201 round-1 designs bound (printed elsewhere in the same paper as 3%), and 53 of 400 round-2 designs bound, printed as 13%. That source cannot be turned into a per-method rate: design methods are the entrants' own declarations, the organisers hand-picked 101 of 201 round-1 and 300 of 400 round-2 designs for diversity, and the three binding thresholds behind the figure counts are never defined. Affinities reached are better than the hit rates suggest. AlphaProteo's best measured binder was 0.082 nM on IL-7 receptor alpha by HTRF; BindCraft's best ranged from 5 nM on CbAgo to 5.7 micromolar on CrSAS-6; the best de novo design in the Adaptyv competition reached 82 nM and the winning optimised cetuximab fragment 1.21 nM, against 9.94 nM for the cetuximab positive control measured on the same instrument. Three things stop these numbers from ranking methods. The hit definitions are different endpoints, not different thresholds on one endpoint. The denominators are small and chosen, and the designs tested are the top picks of each method's own filter, so design and selection are coupled: changing only AlphaProteo's filter moved its PD-L1 rate from 9.6% over 94 designs to 16.1% over 31, and the Bennett authors attribute their low antibody rates partly to a filter they describe as weak. And the spread within one method across targets is wider than any plausible difference between methods. No conventional comparator is run anywhere in this evidence, so there is no measurement of what a designed binder is being compared against. (AlphaProteo Table 1, Table S2 and Sections S1.2.1, S2 and S4.1; BindCraft Introduction and Results prose; Bennett Supplementary Methods Table 6; Adaptyv Sections 2.3, 2.4, 3.1, 3.2, 4.1, 4.2 and 4.3)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.