proteinbench primary benchmark evidence
2409.06744v1
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- ESM-IF1: family ESM-IF1: Family ESM-IF1 (Tables 2; task methods and corresponding named row)
- ESM3 (1.4B): family ESM3: Family ESM3 (Tables 4; task methods and corresponding named row)
- ESM3: family ESM3: Family ESM3 (Tables 2 and 5; task methods and corresponding named row)
- ESMFold: family ESMFold: Family ESMFold (Tables 7; task methods and corresponding named row)
- ESMFold: family ESMFold: Family ESMFold (Appendix B.2.3 Model implementations, ESMFold; Table7 CAMEO2022)
- OpenFold: family OpenFold: Family OpenFold (Tables 7; task methods and corresponding named row)
- ProteinBench AB-ACCURACY-AAR: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Accuracy AAR ↑))
- ProteinBench AB-ACCURACY-RMSD: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Accuracy RMSD ↓))
- ProteinBench AB-ACCURACY-TM-SCORE: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Accuracy TM-score ↑))
- ProteinBench AB-FUNCTIONALITY-BINDING-ENERGY: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Functionality Binding Energy ↓))
- ProteinBench AB-RATIONALITY-CLASHES-INNER: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Rationality Clashes-inner ↓))
- ProteinBench AB-RATIONALITY-CLASHES-OUTER: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Rationality Clashes-outer ↓))
- ProteinBench AB-RATIONALITY-CN-SCORE: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Rationality CN-score ↑))
- ProteinBench AB-RATIONALITY-SCRMSD: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Rationality scRMSD ↓))
- ProteinBench AB-RATIONALITY-SEQNAT: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Rationality SeqNat ↑))
- ProteinBench AB-RATIONALITY-TOTAL-ENERGY: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Rationality Total Energy ↓))
- ProteinBench AB-SPECIFICITY-PHR: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Specificity PHR ↓))
- ProteinBench AB-SPECIFICITY-SEQSIM-INNER: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Specificity SeqSim-inner ↑))
- ProteinBench AB-SPECIFICITY-SEQSIM-OUTER: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 6, column(Specificity SeqSim-outer ↓))
- ProteinBench against ProteinGym on ESM-IF1 and ProteinMPNN: different tasks: Discrepancy review explained (The outlier and consensus results of discrepancy-d734930095432234 as published at https://benchmarks.rewirebio.io/contested/discrepancy-d734930095432234/ (release 2026-10-23-968c8720eea6))
- ProteinBench against ProteinGym on ESM-IF1 and ProteinMPNN: different tasks: Discrepancy review explained (The outlier and consensus results of discrepancy-d734930095432234 as published at https://benchmarks.rewirebio.io/contested/discrepancy-d734930095432234/ (release 2026-10-23-968c8720eea6))
- ProteinBench against ProteinGym on ESM3 and ProteinMPNN: different tasks: Discrepancy review explained (The outlier and consensus results of discrepancy-f82ed80f71c6f0bd as published at https://benchmarks.rewirebio.io/contested/discrepancy-f82ed80f71c6f0bd/ (release 2026-10-23-968c8720eea6))
- ProteinBench against ProteinGym on ESM3 and ProteinMPNN: different tasks: Discrepancy review explained (The outlier and consensus results of discrepancy-f82ed80f71c6f0bd as published at https://benchmarks.rewirebio.io/contested/discrepancy-f82ed80f71c6f0bd/ (release 2026-10-23-968c8720eea6))
- ProteinBench BB-LENGTH-50-DIVERSITY-MAX-CLUST: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 3, column(length 50, Diversity Max Clust. ↑))
- ProteinBench BB-LENGTH-50-DIVERSITY-PAIRWISE-TM: part of discovery-benchmark-proteinbench: Part of ProteinBench (Table 3, column(length 50, Diversity pairwise TM ↓))
Models, benchmarks and other records
Configurations (34)
Benchmarks (1)
Tasks (96)
- Monomer structure
- ProteinBench AB-ACCURACY-AAR: Accuracy AAR
- ProteinBench AB-ACCURACY-RMSD: Accuracy RMSD
- ProteinBench AB-ACCURACY-TM-SCORE: Accuracy TM-score
- ProteinBench AB-FUNCTIONALITY-BINDING-ENERGY: Functionality Binding Energy
- ProteinBench AB-RATIONALITY-CLASHES-INNER: Rationality Clashes-inner
- ProteinBench AB-RATIONALITY-CLASHES-OUTER: Rationality Clashes-outer
- ProteinBench AB-RATIONALITY-CN-SCORE: Rationality CN-score
- ProteinBench AB-RATIONALITY-SCRMSD: Rationality scRMSD
- ProteinBench AB-RATIONALITY-SEQNAT: Rationality SeqNat
- ProteinBench AB-RATIONALITY-TOTAL-ENERGY: Rationality Total Energy
- ProteinBench AB-SPECIFICITY-PHR: Specificity PHR
- ProteinBench AB-SPECIFICITY-SEQSIM-INNER: Specificity SeqSim-inner
- ProteinBench AB-SPECIFICITY-SEQSIM-OUTER: Specificity SeqSim-outer
- ProteinBench BB-LENGTH-50-DIVERSITY-MAX-CLUST: length 50, Diversity Max Clust.
- ProteinBench BB-LENGTH-50-DIVERSITY-PAIRWISE-TM: length 50, Diversity pairwise TM
- ProteinBench BB-LENGTH-50-NOVELTY-MAX-TM: length 50, Novelty Max TM
- ProteinBench BB-LENGTH-50-QUALITY-SCRMSD: length 50, Quality scRMSD
- ProteinBench BB-LENGTH-50-QUALITY-SCTM: length 50, Quality scTM
- ProteinBench BB-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- ProteinBench BB-LENGTH-100-DIVERSITY-PAIRWISE-TM: length 100, Diversity pairwise TM
- ProteinBench BB-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- ProteinBench BB-LENGTH-100-QUALITY-SCRMSD: length 100, Quality scRMSD
- ProteinBench BB-LENGTH-100-QUALITY-SCTM: length 100, Quality scTM
- ProteinBench BB-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
Dataset subsets (6)
- CAMEO2022, 183 proteins (ProteinBench split)
- CASP, CAMEO and de novo backbones (ProteinBench split)
- Co-generated structures and sequences at fixed lengths (ProteinBench split)
- Designed backbones at fixed lengths (ProteinBench split)
- Generated sequences at fixed lengths (ProteinBench split)
- RAbD, 55 antibody-antigen complexes (ProteinBench split)
Evaluations (556)
- AbDPO on ProteinBench AB-ACCURACY-AAR: Accuracy AAR
- AbDPO on ProteinBench AB-ACCURACY-RMSD: Accuracy RMSD
- AbDPO on ProteinBench AB-ACCURACY-TM-SCORE: Accuracy TM-score
- AbDPO on ProteinBench AB-FUNCTIONALITY-BINDING-ENERGY: Functionality Binding Energy
- AbDPO on ProteinBench AB-RATIONALITY-CLASHES-INNER: Rationality Clashes-inner
- AbDPO on ProteinBench AB-RATIONALITY-CLASHES-OUTER: Rationality Clashes-outer
- AbDPO on ProteinBench AB-RATIONALITY-CN-SCORE: Rationality CN-score
- AbDPO on ProteinBench AB-RATIONALITY-SCRMSD: Rationality scRMSD
- AbDPO on ProteinBench AB-RATIONALITY-SEQNAT: Rationality SeqNat
- AbDPO on ProteinBench AB-RATIONALITY-TOTAL-ENERGY: Rationality Total Energy
- AbDPO on ProteinBench AB-SPECIFICITY-PHR: Specificity PHR
- AbDPO on ProteinBench AB-SPECIFICITY-SEQSIM-INNER: Specificity SeqSim-inner
- AbDPO on ProteinBench AB-SPECIFICITY-SEQSIM-OUTER: Specificity SeqSim-outer
- AbDPO++ on ProteinBench AB-ACCURACY-AAR: Accuracy AAR
- AbDPO++ on ProteinBench AB-ACCURACY-RMSD: Accuracy RMSD
- AbDPO++ on ProteinBench AB-ACCURACY-TM-SCORE: Accuracy TM-score
- AbDPO++ on ProteinBench AB-FUNCTIONALITY-BINDING-ENERGY: Functionality Binding Energy
- AbDPO++ on ProteinBench AB-RATIONALITY-CLASHES-INNER: Rationality Clashes-inner
- AbDPO++ on ProteinBench AB-RATIONALITY-CLASHES-OUTER: Rationality Clashes-outer
- AbDPO++ on ProteinBench AB-RATIONALITY-CN-SCORE: Rationality CN-score
- AbDPO++ on ProteinBench AB-RATIONALITY-SCRMSD: Rationality scRMSD
- AbDPO++ on ProteinBench AB-RATIONALITY-SEQNAT: Rationality SeqNat
- AbDPO++ on ProteinBench AB-RATIONALITY-TOTAL-ENERGY: Rationality Total Energy
- AbDPO++ on ProteinBench AB-SPECIFICITY-PHR: Specificity PHR
- AbDPO++ on ProteinBench AB-SPECIFICITY-SEQSIM-INNER: Specificity SeqSim-inner
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
16 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 4334d636223ad42bfb9ae68aae03f5a255c29ba1cebe7b8f9588fb3b9b5453b2 Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Artifact url https://arxiv.org/html/2409.06744v1 Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Doi 10.48550/arXiv.2409.06744 Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Hash scope Exact retrieved primary paper artifact bytes. Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Retrieved at 2026-09-16T21:07:13.231727+00:00 Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
Review method1 values
| proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Review: scope Source identification and table transcription; no independent experimental reproduction. Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Url https://arxiv.org/html/2409.06744v1 Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Version 2409.06744v1 Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Description Primary reference; inspect retrieval status and review scope. Source metadata | proteinbench primary benchmark evidence No field-specific location recorded Version: 2409.06744v1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: source checked
Primary reference; inspect retrieval status and review scope.
Technical metadata and extraction receipts
Stable ID: expansion-p3-proteinbench
- url
- https://arxiv.org/html/2409.06744v1
- doi
- 10.48550/arXiv.2409.06744
- version
- 2409.06744v1
- retrieved at
- 2026-09-16T21:07:13.231727+00:00
- artifact sha256
- 4334d636223ad42bfb9ae68aae03f5a255c29ba1cebe7b8f9588fb3b9b5453b2
- artifact url
- https://arxiv.org/html/2409.06744v1
- hash scope
- Exact retrieved primary paper artifact bytes.
- review
- scope: Source identification and table transcription; no independent experimental reproduction.; method: automated-source-review