ESM-2 (15B) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC: origin
Origin of ESM-2 (15B) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
independent_paper
Correct the origin from 'author_reported' to 'independent_paper' (ProteinGym origin audit, issue #161).
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
7 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Claimed field attributes.origin Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Previous value author_reported Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Source location Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Claimed value independent_paper Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Description Correct the origin from 'author_reported' to 'independent_paper' (ProteinGym origin audit, issue #161). Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Relationship: subject proteingym-evaluation-esm-2-15b-zs-sub-auc Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
| Name ESM-2 (15B) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC: attributes.origin Context-only references | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
Sources and history
Release 2026-11-02-a7f03c9e55a5 · Record review: source checked
1 source record and release history
- ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Original source · Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256
Technical metadata and extraction receipts
Stable ID: metadata-correction-pgorigin-20261014-746a0f55266f6926c9eb
- field
- attributes.origin
- previous value
- author_reported
- value
- independent_paper
- source locator
- Section 4.1 'Baselines': 'We implement a diverse set of 50+ zero-shot baselines'; Section 4.2 'Baselines' and Appendix A.4.2: supervised baselines 'defined in Notin et al. [2023]'; Appendix A.3.3: 'all of the model predictions can be reproduced using this repository'; Appendix A.4.1 per-baseline scoring; reference list
- review
- method: source-hash-verification; automated-source-review; reviewer: claude; reviewer note: Claude (Opus 5.5) agent acting as independent reviewer of PR #168; not the agent that made or applied the judgements; no human review claimed; date: 2026-10-14; method note: Re-retrieved the ProteinGym paper PDF (SHA-256 matches the source record) and read Sections 4.1, 4.2 and 6 and Appendices A.3.3, A.4.1 and A.4.2; checked the developers of the evaluated method against the author metadata of the method paper (Crossref or arXiv) and the ProteinGym author list; checked the subject evaluation's printed value against its Table 2, 3 or 4 cell.; artifact sha256: a3b08cc4a6befd64620cf0f287d78d55a36dc2639955f52c5655f83833c50104; retrieval url: https://papers.nips.cc/paper/2023/file/cac723e5ff29f65e3fcbb0739ae91bee-Paper-Datasets_and_Benchmarks.pdf; note: Accepted in the independent review of 2026-10-14 (docs/reviews/2026-10-14-proteingym-origin-audit.md, section 'Independent review'). ProteinGym paper: the ProteinGym authors scored this baseline and computed the metric; the method is by authors outside ProteinGym (ESM-2: Lin et al. 2023). See docs/reviews/2026-10-14-proteingym-origin-audit.md.