Datasets
Task-specific structure, sequence and complex evaluation collections.
Benchmark
ProteinBench
ProteinBench assesses multiple protein-model tasks using quality, novelty, diversity and robustness dimensions.
Sources
proteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptionsOverview
Metrics
Metric sets vary by task and include structure-predictor confidence, structural similarity and diversity measures.
Allowed inputs
Task-specific sequences, structures or complexes.
Sources
proteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptionsEvaluation procedure diagram
How it worksEvaluation procedure
Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions.
Sources
proteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptionsSource reviewed · Automated source review, 2026-09-16. All specifications and missing details
Results
Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.
ProteinBench AB-ACCURACY-AAR: Accuracy AAR
aar (percent) · Higher values are better.
ProteinBench AB-ACCURACY-AAR: Accuracy AAR · RAbD, 55 antibody-antigen complexes (ProteinBench split)
Evidence origin: Author-reported evaluation.
proteinbench primary benchmark evidence · Table 6, column(Accuracy AAR ↑)- Many of these metrics are better when lower, including scRMSD, perplexity and binding energy. The direction comes from the arrow in the paper's own header.
- Rows such as Native PDBs and RAbD are the natural reference the authors include for scale, not a competing method.
Comparison details and limitations
Every method ProteinBench reports on Accuracy AAR, scored with AAR on RAbD, 55 antibody-antigen complexes.
- Author-reported numbers, source checked but not independently reproduced.
Automated source review: 2026-09-19. Numerical source review does not establish independent reproduction.
Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.
Showing 8 of 8 matching rows.
Tested configuration
0255075100
Reported scoreMethods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
ProteinBench evaluates protein sequence and structure methods across design, folding and conformational tasks. It distinguishes quality, diversity and novelty instead of treating all protein capabilities as one accuracy score. Each task has its own reference data, sampling procedure and baseline family.
Sources
proteinbench primary benchmark evidence · Task definitions; ATLAS evaluation; antibody-design setup; result tablesEvaluation design
Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.
Tasks
- ProteinBench AB-ACCURACY-AAR: Accuracy AAR
- ProteinBench AB-ACCURACY-RMSD: Accuracy RMSD
- ProteinBench AB-ACCURACY-TM-SCORE: Accuracy TM-score
- ProteinBench AB-FUNCTIONALITY-BINDING-ENERGY: Functionality Binding Energy
- ProteinBench AB-RATIONALITY-CLASHES-INNER: Rationality Clashes-inner
- ProteinBench AB-RATIONALITY-CLASHES-OUTER: Rationality Clashes-outer
- ProteinBench AB-RATIONALITY-CN-SCORE: Rationality CN-score
- ProteinBench AB-RATIONALITY-SCRMSD: Rationality scRMSD
- ProteinBench AB-RATIONALITY-SEQNAT: Rationality SeqNat
- ProteinBench AB-RATIONALITY-TOTAL-ENERGY: Rationality Total Energy
- ProteinBench AB-SPECIFICITY-PHR: Specificity PHR
- ProteinBench AB-SPECIFICITY-SEQSIM-INNER: Specificity SeqSim-inner
- ProteinBench AB-SPECIFICITY-SEQSIM-OUTER: Specificity SeqSim-outer
- ProteinBench BB-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- ProteinBench BB-LENGTH-100-DIVERSITY-PAIRWISE-TM: length 100, Diversity pairwise TM
- ProteinBench BB-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- ProteinBench BB-LENGTH-100-QUALITY-SCRMSD: length 100, Quality scRMSD
- ProteinBench BB-LENGTH-100-QUALITY-SCTM: length 100, Quality scTM
- ProteinBench BB-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
- ProteinBench BB-LENGTH-300-DIVERSITY-PAIRWISE-TM: length 300, Diversity pairwise TM
- ProteinBench BB-LENGTH-300-NOVELTY-MAX-TM: length 300, Novelty Max TM
- ProteinBench BB-LENGTH-300-QUALITY-SCRMSD: length 300, Quality scRMSD
- ProteinBench BB-LENGTH-300-QUALITY-SCTM: length 300, Quality scTM
- ProteinBench BB-LENGTH-50-DIVERSITY-MAX-CLUST: length 50, Diversity Max Clust.
- ProteinBench BB-LENGTH-50-DIVERSITY-PAIRWISE-TM: length 50, Diversity pairwise TM
- ProteinBench BB-LENGTH-50-NOVELTY-MAX-TM: length 50, Novelty Max TM
- ProteinBench BB-LENGTH-50-QUALITY-SCRMSD: length 50, Quality scRMSD
- ProteinBench BB-LENGTH-50-QUALITY-SCTM: length 50, Quality scTM
- ProteinBench BB-LENGTH-500-DIVERSITY-MAX-CLUST: length 500, Diversity Max Clust.
- ProteinBench BB-LENGTH-500-DIVERSITY-PAIRWISE-TM: length 500, Diversity pairwise TM
- ProteinBench BB-LENGTH-500-NOVELTY-MAX-TM: length 500, Novelty Max TM
- ProteinBench BB-LENGTH-500-QUALITY-SCRMSD: length 500, Quality scRMSD
- ProteinBench BB-LENGTH-500-QUALITY-SCTM: length 500, Quality scTM
- ProteinBench CO-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- ProteinBench CO-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- ProteinBench CO-LENGTH-100-QUALITY-SCRMSD: length 100, Quality scRMSD
- ProteinBench CO-LENGTH-100-QUALITY-SCTM: length 100, Quality scTM
- ProteinBench CO-LENGTH-200-DIVERSITY-MAX-CLUST: length 200, Diversity Max Clust.
- ProteinBench CO-LENGTH-200-NOVELTY-MAX-TM: length 200, Novelty Max TM
- ProteinBench CO-LENGTH-200-QUALITY-SCRMSD: length 200, Quality scRMSD
- ProteinBench CO-LENGTH-200-QUALITY-SCTM: length 200, Quality scTM
- ProteinBench CO-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
- ProteinBench CO-LENGTH-300-NOVELTY-MAX-TM: length 300, Novelty Max TM
- ProteinBench CO-LENGTH-300-QUALITY-SCRMSD: length 300, Quality scRMSD
- ProteinBench CO-LENGTH-300-QUALITY-SCTM: length 300, Quality scTM
- ProteinBench CO-LENGTH-500-DIVERSITY-MAX-CLUST: length 500, Diversity Max Clust.
- ProteinBench CO-LENGTH-500-NOVELTY-MAX-TM: length 500, Novelty Max TM
- ProteinBench CO-LENGTH-500-QUALITY-SCRMSD: length 500, Quality scRMSD
- ProteinBench CO-LENGTH-500-QUALITY-SCTM: length 500, Quality scTM
- ProteinBench FOLD-ACCURACY-GDT-TS-MEAN: Accuracy GDT-TS (mean)
- ProteinBench FOLD-ACCURACY-GDT-TS-MEDIAN: Accuracy GDT-TS (median)
- ProteinBench FOLD-ACCURACY-LDDT-MEAN: Accuracy lDDT (mean)
- ProteinBench FOLD-ACCURACY-LDDT-MEDIAN: Accuracy lDDT (median)
- ProteinBench FOLD-ACCURACY-RMSD-MEAN: Accuracy RMSD (mean)
- ProteinBench FOLD-ACCURACY-RMSD-MEDIAN: Accuracy RMSD (median)
- ProteinBench FOLD-ACCURACY-TM-SCORE-MEAN: Accuracy TM-score (mean)
- ProteinBench FOLD-ACCURACY-TM-SCORE-MEDIAN: Accuracy TM-score (median)
- ProteinBench FOLD-QUALITY-CA-BREAK-MEAN: Quality CA break (%) (mean)
- ProteinBench FOLD-QUALITY-CA-BREAK-MEDIAN: Quality CA break (%) (median)
- ProteinBench FOLD-QUALITY-CA-CLASH-MEAN: Quality CA clash (%) (mean)
- ProteinBench FOLD-QUALITY-CA-CLASH-MEDIAN: Quality CA clash (%) (median)
- ProteinBench FOLD-QUALITY-PEPBOND-BREAK-MEAN: Quality PepBond break (%) (mean)
- ProteinBench FOLD-QUALITY-PEPBOND-BREAK-MEDIAN: Quality PepBond break (%) (median)
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-100-PLDDT: De novo backbones based sequence design, length 100 pLDDT
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-100-SCTM: De novo backbones based sequence design, length 100 scTM
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-200-PLDDT: De novo backbones based sequence design, length 200 pLDDT
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-200-SCTM: De novo backbones based sequence design, length 200 scTM
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-300-PLDDT: De novo backbones based sequence design, length 300 pLDDT
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-300-SCTM: De novo backbones based sequence design, length 300 scTM
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-400-PLDDT: De novo backbones based sequence design, length 400 pLDDT
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-400-SCTM: De novo backbones based sequence design, length 400 scTM
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-500-PLDDT: De novo backbones based sequence design, length 500 pLDDT
- ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-500-SCTM: De novo backbones based sequence design, length 500 scTM
- ProteinBench IF-FITTING-EVOLUTION-DISTRIBUTION-CAMEO-AAR: Fitting Evolution Distribution, CAMEO AAR
- ProteinBench IF-FITTING-EVOLUTION-DISTRIBUTION-CASP-AAR: Fitting Evolution Distribution, CASP AAR
- ProteinBench SEQ-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- ProteinBench SEQ-LENGTH-100-DIVERSITY-PAIRWISE-TM: length 100, Diversity pairwise TM
- ProteinBench SEQ-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- ProteinBench SEQ-LENGTH-100-QUALITY-PLDDT: length 100, Quality pLDDT
- ProteinBench SEQ-LENGTH-100-QUALITY-PPL: length 100, Quality ppl
- ProteinBench SEQ-LENGTH-200-DIVERSITY-MAX-CLUST: length 200, Diversity Max Clust.
- ProteinBench SEQ-LENGTH-200-DIVERSITY-PAIRWISE-TM: length 200, Diversity pairwise TM
- ProteinBench SEQ-LENGTH-200-NOVELTY-MAX-TM: length 200, Novelty Max TM
- ProteinBench SEQ-LENGTH-200-QUALITY-PLDDT: length 200, Quality pLDDT
- ProteinBench SEQ-LENGTH-200-QUALITY-PPL: length 200, Quality ppl
- ProteinBench SEQ-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
- ProteinBench SEQ-LENGTH-300-DIVERSITY-PAIRWISE-TM: length 300, Diversity pairwise TM
- ProteinBench SEQ-LENGTH-300-NOVELTY-MAX-TM: length 300, Novelty Max TM
- ProteinBench SEQ-LENGTH-300-QUALITY-PLDDT: length 300, Quality pLDDT
- ProteinBench SEQ-LENGTH-300-QUALITY-PPL: length 300, Quality ppl
- ProteinBench SEQ-LENGTH-500-DIVERSITY-MAX-CLUST: length 500, Diversity Max Clust.
- ProteinBench SEQ-LENGTH-500-DIVERSITY-PAIRWISE-TM: length 500, Diversity pairwise TM
- ProteinBench SEQ-LENGTH-500-NOVELTY-MAX-TM: length 500, Novelty Max TM
- ProteinBench SEQ-LENGTH-500-QUALITY-PLDDT: length 500, Quality pLDDT
- ProteinBench SEQ-LENGTH-500-QUALITY-PPL: length 500, Quality ppl
These source-backed links do not make different protocols or scores interchangeable.
Baseline coverage
Reference methods help show what a model adds beyond simple controls. We track a null control and a conventional method for each protocol.
No concrete protocols are explicitly linked to this suite. Protocol identification and baseline selection are outstanding.
Protocol coverage CSV · Model evaluation matrix · Source table · Release and checksums
Coverage is derived from release 2026-09-29-06401fd5b220. Source citations describe the original records; they do not validate an unreviewed baseline proposal. No results have been generated by this audit.
Run this benchmark
The primary project page describes multiple protein evaluation tasks and their results, but the inspected page does not establish a single reproducible run command with data and checkpoint setup. Choose a concrete task/protocol and its implementation rather than treating the summary website as an executable suite.
A maintained rewire runner has not been verified for this benchmark. Check data access, weights, licences, dependencies and hardware in the linked official documentation; requirements have not been fully extracted.
proteinbench official run documentation · Official project page: benchmark task descriptions, results and model/resource linksStrengths, limitations and unresolved questions
Strengths and limitations
Strengths supported by sources
- Separate task collections expose which capability a result measures.
Sources
proteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions
Limitations and conditions
- Architectures receive different inputs and can generate different numbers of candidates. Task-specific exclusion rules and sampling budgets must accompany any comparison.
Sources
proteinbench primary benchmark evidence · Task definitions; ATLAS evaluation; antibody-design setup; result tables
Profile review details
Primary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction.
Stable record: discovery-benchmark-proteinbenchSpecifications
Inputs, training, access and other details
Explanatory profile: source reviewed · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | Task-specific structure, sequence and complex evaluation collections.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Splits | The framework distinguishes natural in-distribution structures from generated-backbone out-of-distribution evaluations.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Metrics | Metric sets vary by task and include structure-predictor confidence, structural similarity and diversity measures.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Baselines | Comparators vary by task: structure predictors, inverse-folding/design models, Rosetta-based antibody methods and molecular-dynamics or ensemble references. The paper defines the eligible method and input information separately for each task.Sourcesproteinbench primary benchmark evidence · Task definitions; ATLAS evaluation; antibody-design setup; result tables |
| Leakage controls | The antibody setup clusters CDR-H3 sequences at 40% similarity and excludes clusters containing RAbD test complexes from training/validation. The ATLAS ensemble task also applies a held-out protocol for models trained on ATLAS; these are task-specific controls, not a universal suite split.Sourcesproteinbench primary benchmark evidence · Task definitions; ATLAS evaluation; antibody-design setup; result tables |
| Uncertainty | Some task tables report repeated-experiment averages and standard deviations; others report medians.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Entity type | Protein-model evaluation framework spanning several tasks.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Organisms | No single organism defines its sequence, structure and complex task collections. · Not applicableSourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Assays | Task-dependent protein structure and sequence/property references.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Allowed inputs | Task-specific sequences, structures or complexes.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
| Adaptation | Tasks define their own generation, prediction or adaptation regimes.Sourcesproteinbench official source · ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions |
Applicable tests and references
Applicability is distinct from a completed evaluation.
- Protein sequence recovery specialist · proposed
- ESM-IF1 · Proposed association
- ESM3 · Proposed association
- ESMFold · Proposed association
- ProteinMPNN · Proposed association
- RFdiffusion · Proposed association
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
| Paper or primary resource | Version | Reference |
|---|---|---|
| proteinbench primary benchmark evidence | 2409.06744v1 | Read source DOI: 10.48550/arXiv.2409.06744 |
Historical gaps recorded on 2026-09-17
The catalogue now holds 556 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- Mean/median slash pairs in Table 7 are not uncertainty.
- EigenFold removes unknown amino acids and lacks a peptide-bond metric; preserve applicability caveats.
- Do not transfer Table 8 bootstrap uncertainty to Table 7.
Search and extraction details
primary protocol screened
Searches
- Genomic Benchmarks collection genomic sequence classification PMC10150520
- mRNABench PMC12265608
- PFMBench 2506.14796
- ProteinBench 2409.06744
Evidence locations
- Section 3.2.1
- Table 7
- Table 8 caption
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
18 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual procedure. Task variants and protocol versions retain their separate scoring conditions. Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Evaluation procedure Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets Task-specific structure, sequence and complex evaluation collections. Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits The framework distinguishes natural in-distribution structures from generated-backbone out-of-distribution evaluations. Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation Tasks define their own generation, prediction or adaptation regimes. Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics Metric sets vary by task and include structure-predictor confidence, structural similarity and diversity measures. Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Comparators vary by task: structure predictors, inverse-folding/design models, Rosetta-based antibody methods and molecular-dynamics or ensemble references. The paper defines the eligible method and input information separately for each task. Individual claims | proteinbench primary benchmark evidence Task definitions; ATLAS evaluation; antibody-design setup; result tables Version: 2409.06744v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls The antibody setup clusters CDR-H3 sequences at 40% similarity and excludes clusters containing RAbD test complexes from training/validation. The ATLAS ensemble task also applies a held-out protocol for models trained on ATLAS; these are task-specific controls, not a universal suite split. Individual claims | proteinbench primary benchmark evidence Task definitions; ATLAS evaluation; antibody-design setup; result tables Version: 2409.06744v1 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty Some task tables report repeated-experiment averages and standard deviations; others report medians. Individual claims | proteinbench official source ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions Version: Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712 | source checked automated source review · 2026-09-16 Audit detailsPrimary paper and/or task implementation reviewed for the explicitly cited methodology claims. Scope-limited absence is recorded only after the documented source search; no model runs or independent reproduction. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: discovered
5 source records and release history
- proteinbench official source · Original source · Not reported
- proteinbench primary benchmark evidence · Original source · 2409.06744v1
- proteinbench official source · Original source · Retrieved website snapshot sha256:2e488850a6557bb57407615f2df9194351718b3dc0298a03c0c97d8e93460712
- proteinbench official run documentation · Original source · Official documentation snapshot 2026-09-17; mutable website
- proteinbench primary benchmark evidence · Original source · 2409.06744v1
Technical metadata and extraction receipts
Stable ID: discovery-benchmark-proteinbench
- areas
- protein-structure
- entity level
- suite
- scope note
- Specialist molecular or omics evaluation; protocol details require review before numerical comparison.
- task
- Protein prediction, design and dynamics evaluation
- version
- Not reported
- benchmark research
- review date: 2026-09-17; status: primary_protocol_screened; primary sources: expansion-p3-proteinbench; inspected locators: Section 3.2.1; Table 7; Table 8 caption; searched queries: Genomic Benchmarks collection genomic sequence classification PMC10150520; mRNABench PMC12265608; PFMBench 2506.14796; ProteinBench 2409.06744; gaps: Mean/median slash pairs in Table 7 are not uncertainty.; EigenFold removes unknown amino acids and lacks a peptide-bond metric; preserve applicability caveats.; Do not transfer Table 8 bootstrap uncertainty to Table 7.; claim scope: Primary-paper discovery and source inspection. Source-checked results are not independently reproduced experiments.
- historical missing metadata
- dataset release: unextracted; metric implementation: unextracted; split manifest: unextracted; version: unextracted
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- entity classification
- review date: 2026-09-17; rationale: The cited profile describes a collection of evaluation tasks or protocols; retain it as the top-level benchmark suite. Its datasets and individual protocols remain separate records.; source ids: evidence-benchmark-proteinbench-snapshot; source locator: ProteinBench official website: Abstract; sequence/structure task evaluations; metric descriptions; ambiguities: None recorded
- run documentation
- record id: discovery-benchmark-proteinbench; source ids: run-doc-proteinbench-official-20260917; status: official_documentation_linked; summary: The primary project page describes multiple protein evaluation tasks and their results, but the inspected page does not establish a single reproducible run command with data and checkpoint setup. Choose a concrete task/protocol and its implementation rather than treating the summary website as an executable suite.; source locator: Official project page: benchmark task descriptions, results and model/resource links
Related records
- applicable to: Protein sequence recovery specialist
- applicable to: ESM-IF1
- applicable to: ESM3
- applicable to: ESMFold
- applicable to: ProteinMPNN
- applicable to: RFdiffusion
- part of: ProteinBench AB-ACCURACY-AAR: Accuracy AAR
- part of: ProteinBench AB-ACCURACY-RMSD: Accuracy RMSD
- part of: ProteinBench AB-ACCURACY-TM-SCORE: Accuracy TM-score
- part of: ProteinBench AB-FUNCTIONALITY-BINDING-ENERGY: Functionality Binding Energy
- part of: ProteinBench AB-RATIONALITY-CLASHES-INNER: Rationality Clashes-inner
- part of: ProteinBench AB-RATIONALITY-CLASHES-OUTER: Rationality Clashes-outer
- part of: ProteinBench AB-RATIONALITY-CN-SCORE: Rationality CN-score
- part of: ProteinBench AB-RATIONALITY-SCRMSD: Rationality scRMSD
- part of: ProteinBench AB-RATIONALITY-SEQNAT: Rationality SeqNat
- part of: ProteinBench AB-RATIONALITY-TOTAL-ENERGY: Rationality Total Energy
- part of: ProteinBench AB-SPECIFICITY-PHR: Specificity PHR
- part of: ProteinBench AB-SPECIFICITY-SEQSIM-INNER: Specificity SeqSim-inner
- part of: ProteinBench AB-SPECIFICITY-SEQSIM-OUTER: Specificity SeqSim-outer
- part of: ProteinBench BB-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- part of: ProteinBench BB-LENGTH-100-DIVERSITY-PAIRWISE-TM: length 100, Diversity pairwise TM
- part of: ProteinBench BB-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- part of: ProteinBench BB-LENGTH-100-QUALITY-SCRMSD: length 100, Quality scRMSD
- part of: ProteinBench BB-LENGTH-100-QUALITY-SCTM: length 100, Quality scTM
- part of: ProteinBench BB-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
- part of: ProteinBench BB-LENGTH-300-DIVERSITY-PAIRWISE-TM: length 300, Diversity pairwise TM
- part of: ProteinBench BB-LENGTH-300-NOVELTY-MAX-TM: length 300, Novelty Max TM
- part of: ProteinBench BB-LENGTH-300-QUALITY-SCRMSD: length 300, Quality scRMSD
- part of: ProteinBench BB-LENGTH-300-QUALITY-SCTM: length 300, Quality scTM
- part of: ProteinBench BB-LENGTH-50-DIVERSITY-MAX-CLUST: length 50, Diversity Max Clust.
- part of: ProteinBench BB-LENGTH-50-DIVERSITY-PAIRWISE-TM: length 50, Diversity pairwise TM
- part of: ProteinBench BB-LENGTH-50-NOVELTY-MAX-TM: length 50, Novelty Max TM
- part of: ProteinBench BB-LENGTH-50-QUALITY-SCRMSD: length 50, Quality scRMSD
- part of: ProteinBench BB-LENGTH-50-QUALITY-SCTM: length 50, Quality scTM
- part of: ProteinBench BB-LENGTH-500-DIVERSITY-MAX-CLUST: length 500, Diversity Max Clust.
- part of: ProteinBench BB-LENGTH-500-DIVERSITY-PAIRWISE-TM: length 500, Diversity pairwise TM
- part of: ProteinBench BB-LENGTH-500-NOVELTY-MAX-TM: length 500, Novelty Max TM
- part of: ProteinBench BB-LENGTH-500-QUALITY-SCRMSD: length 500, Quality scRMSD
- part of: ProteinBench BB-LENGTH-500-QUALITY-SCTM: length 500, Quality scTM
- part of: ProteinBench CO-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- part of: ProteinBench CO-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- part of: ProteinBench CO-LENGTH-100-QUALITY-SCRMSD: length 100, Quality scRMSD
- part of: ProteinBench CO-LENGTH-100-QUALITY-SCTM: length 100, Quality scTM
- part of: ProteinBench CO-LENGTH-200-DIVERSITY-MAX-CLUST: length 200, Diversity Max Clust.
- part of: ProteinBench CO-LENGTH-200-NOVELTY-MAX-TM: length 200, Novelty Max TM
- part of: ProteinBench CO-LENGTH-200-QUALITY-SCRMSD: length 200, Quality scRMSD
- part of: ProteinBench CO-LENGTH-200-QUALITY-SCTM: length 200, Quality scTM
- part of: ProteinBench CO-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
- part of: ProteinBench CO-LENGTH-300-NOVELTY-MAX-TM: length 300, Novelty Max TM
- part of: ProteinBench CO-LENGTH-300-QUALITY-SCRMSD: length 300, Quality scRMSD
- part of: ProteinBench CO-LENGTH-300-QUALITY-SCTM: length 300, Quality scTM
- part of: ProteinBench CO-LENGTH-500-DIVERSITY-MAX-CLUST: length 500, Diversity Max Clust.
- part of: ProteinBench CO-LENGTH-500-NOVELTY-MAX-TM: length 500, Novelty Max TM
- part of: ProteinBench CO-LENGTH-500-QUALITY-SCRMSD: length 500, Quality scRMSD
- part of: ProteinBench CO-LENGTH-500-QUALITY-SCTM: length 500, Quality scTM
- part of: ProteinBench FOLD-ACCURACY-GDT-TS-MEAN: Accuracy GDT-TS (mean)
- part of: ProteinBench FOLD-ACCURACY-GDT-TS-MEDIAN: Accuracy GDT-TS (median)
- part of: ProteinBench FOLD-ACCURACY-LDDT-MEAN: Accuracy lDDT (mean)
- part of: ProteinBench FOLD-ACCURACY-LDDT-MEDIAN: Accuracy lDDT (median)
- part of: ProteinBench FOLD-ACCURACY-RMSD-MEAN: Accuracy RMSD (mean)
- part of: ProteinBench FOLD-ACCURACY-RMSD-MEDIAN: Accuracy RMSD (median)
- part of: ProteinBench FOLD-ACCURACY-TM-SCORE-MEAN: Accuracy TM-score (mean)
- part of: ProteinBench FOLD-ACCURACY-TM-SCORE-MEDIAN: Accuracy TM-score (median)
- part of: ProteinBench FOLD-QUALITY-CA-BREAK-MEAN: Quality CA break (%) (mean)
- part of: ProteinBench FOLD-QUALITY-CA-BREAK-MEDIAN: Quality CA break (%) (median)
- part of: ProteinBench FOLD-QUALITY-CA-CLASH-MEAN: Quality CA clash (%) (mean)
- part of: ProteinBench FOLD-QUALITY-CA-CLASH-MEDIAN: Quality CA clash (%) (median)
- part of: ProteinBench FOLD-QUALITY-PEPBOND-BREAK-MEAN: Quality PepBond break (%) (mean)
- part of: ProteinBench FOLD-QUALITY-PEPBOND-BREAK-MEDIAN: Quality PepBond break (%) (median)
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-100-PLDDT: De novo backbones based sequence design, length 100 pLDDT
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-100-SCTM: De novo backbones based sequence design, length 100 scTM
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-200-PLDDT: De novo backbones based sequence design, length 200 pLDDT
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-200-SCTM: De novo backbones based sequence design, length 200 scTM
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-300-PLDDT: De novo backbones based sequence design, length 300 pLDDT
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-300-SCTM: De novo backbones based sequence design, length 300 scTM
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-400-PLDDT: De novo backbones based sequence design, length 400 pLDDT
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-400-SCTM: De novo backbones based sequence design, length 400 scTM
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-500-PLDDT: De novo backbones based sequence design, length 500 pLDDT
- part of: ProteinBench IF-DE-NOVO-BACKBONES-BASED-SEQUENCE-DESIGN-LENGTH-500-SCTM: De novo backbones based sequence design, length 500 scTM
- part of: ProteinBench IF-FITTING-EVOLUTION-DISTRIBUTION-CAMEO-AAR: Fitting Evolution Distribution, CAMEO AAR
- part of: ProteinBench IF-FITTING-EVOLUTION-DISTRIBUTION-CASP-AAR: Fitting Evolution Distribution, CASP AAR
- part of: ProteinBench SEQ-LENGTH-100-DIVERSITY-MAX-CLUST: length 100, Diversity Max Clust.
- part of: ProteinBench SEQ-LENGTH-100-DIVERSITY-PAIRWISE-TM: length 100, Diversity pairwise TM
- part of: ProteinBench SEQ-LENGTH-100-NOVELTY-MAX-TM: length 100, Novelty Max TM
- part of: ProteinBench SEQ-LENGTH-100-QUALITY-PLDDT: length 100, Quality pLDDT
- part of: ProteinBench SEQ-LENGTH-100-QUALITY-PPL: length 100, Quality ppl
- part of: ProteinBench SEQ-LENGTH-200-DIVERSITY-MAX-CLUST: length 200, Diversity Max Clust.
- part of: ProteinBench SEQ-LENGTH-200-DIVERSITY-PAIRWISE-TM: length 200, Diversity pairwise TM
- part of: ProteinBench SEQ-LENGTH-200-NOVELTY-MAX-TM: length 200, Novelty Max TM
- part of: ProteinBench SEQ-LENGTH-200-QUALITY-PLDDT: length 200, Quality pLDDT
- part of: ProteinBench SEQ-LENGTH-200-QUALITY-PPL: length 200, Quality ppl
- part of: ProteinBench SEQ-LENGTH-300-DIVERSITY-MAX-CLUST: length 300, Diversity Max Clust.
- part of: ProteinBench SEQ-LENGTH-300-DIVERSITY-PAIRWISE-TM: length 300, Diversity pairwise TM
- part of: ProteinBench SEQ-LENGTH-300-NOVELTY-MAX-TM: length 300, Novelty Max TM
- part of: ProteinBench SEQ-LENGTH-300-QUALITY-PLDDT: length 300, Quality pLDDT
- part of: ProteinBench SEQ-LENGTH-300-QUALITY-PPL: length 300, Quality ppl
- part of: ProteinBench SEQ-LENGTH-500-DIVERSITY-MAX-CLUST: length 500, Diversity Max Clust.
- part of: ProteinBench SEQ-LENGTH-500-DIVERSITY-PAIRWISE-TM: length 500, Diversity pairwise TM
- part of: ProteinBench SEQ-LENGTH-500-NOVELTY-MAX-TM: length 500, Novelty Max TM
- part of: ProteinBench SEQ-LENGTH-500-QUALITY-PLDDT: length 500, Quality pLDDT
- part of: ProteinBench SEQ-LENGTH-500-QUALITY-PPL: length 500, Quality ppl