ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction
2023 · PMC10723403.1 · Preprint
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
| Model or configuration | Metric | Value | Dataset | Evaluation |
|---|---|---|---|---|
| ESM-2 | Spearman (mean over assays) | 0.488 unitless | ProteinGym substitution DMS: stability assays | ESM-2: Zero-shot substitution mutation effects: stability |
| ProteinMPNN | Spearman (mean over assays) | 0.566 unitless | ProteinGym substitution DMS: stability assays | ProteinMPNN: Zero-shot substitution mutation effects: stability |
Claims
Statements this source supports, each with where in the source it was checked.
- Reported Mean Spearman rho for ESM-2: Printed value 0.488 (Table A7, ESM-2 (15B) row, Stability column)
- Reported Mean Spearman rho for ProteinMPNN: Printed value 0.566 (Table A7, ProteinMPNN row, Stability column)
Models, benchmarks and other records
Configurations (2)
Tasks (1)
Datasets (1)
Evaluations (2)
Sources (6)
- ProteinGym AMFR assay-level Spearman comparison
- ProteinGym clinical supervised substitutions AUC summary at commit 144fe22
- ProteinGym clinical zero-shot indels AUC summary at commit 144fe22
- ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22
- ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22
- ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
18 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact retrieved at 2026-09-16T10:41:16.517323+00:00 Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact SHA-256 4519641f13271bdd09b166e7d93232f22542489bc52a25b5a1628c3df8badce1 Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10723403/fullTextXML Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Doi 10.1101/2023.12.07.570727 Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Missing metadata: licence: reason unextracted Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Publication status preprint Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-09-15T23:25:00Z Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Scope decision included Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Url https://pmc.ncbi.nlm.nih.gov/articles/PMC10723403/ Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Version PMC10723403.1 Source metadata | ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction No field-specific location recorded Version: PMC10723403.1 | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: discovered
Primary paper retained with its original identifier. Metadata inherited from the literature collection; individual result checks are separate.
Technical metadata and extraction receipts
Stable ID: proteingym-2023
- areas
- proteins-complexes
- url
- https://pmc.ncbi.nlm.nih.gov/articles/PMC10723403/
- version
- PMC10723403.1
- retrieved at
- 2026-09-15T23:25:00Z
- doi
- 10.1101/2023.12.07.570727
- publication status
- preprint
- year
- 2023
- artifact sha256
- 4519641f13271bdd09b166e7d93232f22542489bc52a25b5a1628c3df8badce1
- artifact url
- https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10723403/fullTextXML
- artifact retrieved at
- 2026-09-16T10:41:16.517323+00:00
- scope decision
- included
- missing metadata
- licence: reason: unextracted
Related records
- companion of: ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design
- companion of: ProteinGym AMFR assay-level Spearman comparison
- companion of: ProteinGym clinical supervised substitutions AUC summary at commit 144fe22
- companion of: ProteinGym clinical zero-shot indels AUC summary at commit 144fe22
- companion of: ProteinGym clinical zero-shot substitutions AUC summary at commit 144fe22
- companion of: ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22