Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py
144fe22b07dfaeec2b366f2346203a9838a55b4c
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- ProteinGym against ProteinBench on ESM3 and ProteinMPNN: different tasks: Discrepancy review explained (The outlier and consensus results of discrepancy-fed245c475fd08dc as published at https://benchmarks.rewirebio.io/contested/discrepancy-fed245c475fd08dc/ (release 2026-10-23-968c8720eea6))
- ProteinGym repository against ProteinGym paper on ESM-1v and ESM-2: different ESM-2 sizes: Discrepancy review false-positive (The outlier and consensus results of discrepancy-6e0a1543c1c68a64 as published at https://benchmarks.rewirebio.io/contested/discrepancy-6e0a1543c1c68a64/ (release 2026-10-23-968c8720eea6))
- Relevance of rewire-proteingym-v1-3-dms-substitutions to use-case-protein-stability: Assessed by ProteinGym v1.3 zero-shot DMS substitutions (vep-lit-20261014-source-proteingym-dms-substitutions-spearman-summary-144fe22: Summary_performance_DMS_substitutions_Spearman.csv column Function_Stability; performance_DMS_benchmarks.py lines 336-348)
Models, benchmarks and other records
Benchmarks (1)
Tasks (1)
Protocols (1)
Evaluations (97)
- AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
- CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- CARP (600K): ProteinGym zero-shot DMS substitutions summary (144fe22)
- CARP (640M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- DeepSequence (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)
- DeepSequence (single): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESCOTT: ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM-1b: ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM-1v (single): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM-C (300M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM-C (600M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM-IF1: ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM2 (3B): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM2 (8M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM2 (15B): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM2 (35M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM2 (150M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM2 (650M): ProteinGym zero-shot DMS substitutions summary (144fe22)
- ESM3 open (1.4B): ProteinGym zero-shot DMS substitutions summary (144fe22)
- EVE (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)
- EVE (single): ProteinGym zero-shot DMS substitutions summary (144fe22)
- EVmutation: ProteinGym zero-shot DMS substitutions summary (144fe22)
- GEMME: ProteinGym zero-shot DMS substitutions summary (144fe22)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
13 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 0002bc1b031c02dc2d67fde53da092c8fd301415645e8e83b425b4183570eba2 Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/proteingym/performance_DMS_benchmarks.py Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-09-23T18:39:35.572257+00:00 Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Review artifact data/omics/reviewed/profile-evidence-2026-09-23/artifacts/protocols-proteingym-proteingym-performance_DMS_benchmarks.py.gz Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Url https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/proteingym/performance_DMS_benchmarks.py Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Version 144fe22b07dfaeec2b366f2346203a9838a55b4c Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description Primary source artifact checked for bounded protocol metadata; no model execution or scientific reproduction. Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Search facets None recorded Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Record ID profile-protocol-proteingym-proteingym-performance-dms-benchmarks-py-0002bc1b Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Record type source Source metadata | Protocol metadata evidence: proteingym-proteingym-performance_DMS_benchmarks.py No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: source checked
Primary source artifact checked for bounded protocol metadata; no model execution or scientific reproduction.
Technical metadata and extraction receipts
Stable ID: profile-protocol-proteingym-proteingym-performance-dms-benchmarks-py-0002bc1b
- url
- https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/proteingym/performance_DMS_benchmarks.py
- artifact url
- https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/proteingym/performance_DMS_benchmarks.py
- artifact sha256
- 0002bc1b031c02dc2d67fde53da092c8fd301415645e8e83b425b4183570eba2
- version
- 144fe22b07dfaeec2b366f2346203a9838a55b4c
- retrieved at
- 2026-09-23T18:39:35.572257+00:00
- review artifact
- data/omics/reviewed/profile-evidence-2026-09-23/artifacts/protocols-proteingym-proteingym-performance_DMS_benchmarks.py.gz