ProteinGym AMFR assay-level Spearman comparison
144fe22b07dfaeec2b366f2346203a9838a55b4c
What this source contributed
Records in this release that cite this source. A record can cite several sources.
Results
Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.
Claims
Statements this source supports, each with where in the source it was checked.
- AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot] family identity: Configuration of AIDO Protein-RAG (Pinned ProteinGym Spearman CSV header: AIDO Protein-RAG (16B))
- AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]: facets.method_types: Facets: method types ["foundation_model"] (Proposer's report, ProteinGym AMFR: protein language and structure-conditioned models fit foundation_model)
- AIDO Protein-RAG (16B) AMFR official snapshot: attributes.origin: Origin unreported (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- AIDO Protein-RAG (16B) AMFR official snapshot: description: Description Official ProteinGym assay-level Spearman for the AMFR assay at commit 144fe22; transcribed, not reproduced. (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- AIDO Protein-RAG: description: Description Retrieval-augmented protein language model from the AIDO.Protein-RAG work, which combines a pretrained protein language model with retrieved multiple sequence alignments; ProteinGym lists AIDO with alignment and structure inputs and scores it as a zero-shot substitution baseline. (uc20260930-source-proteingym-amfr-spearman: Benchmark CSV header (model scored in the zero-shot DMS substitution benchmark); run-doc-proteingym-readme-md-144fe22b: README 'Leaderboard' baseline table row AIDO (MSA & Structure); qa-descriptions-20261010-source-aido-protein-rag: abstract)
- CARP (38M) [ProteinGym 144fe22b snapshot] family identity: Configuration of CARP (Pinned ProteinGym Spearman CSV header: CARP (38M))
- CARP (38M) [ProteinGym 144fe22b snapshot]: facets.method_types: Facets: method types ["foundation_model"] (Proposer's report, ProteinGym AMFR: protein language and structure-conditioned models fit foundation_model)
- CARP (38M) AMFR official snapshot: attributes.origin: Origin unreported (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (38M) AMFR official snapshot: description: Description Official ProteinGym assay-level Spearman for the AMFR assay at commit 144fe22; transcribed, not reproduced. (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (76M) [ProteinGym 144fe22b snapshot] family identity: Configuration of CARP (Pinned ProteinGym Spearman CSV header: CARP (76M))
- CARP (76M) [ProteinGym 144fe22b snapshot]: facets.method_types: Facets: method types ["foundation_model"] (Proposer's report, ProteinGym AMFR: protein language and structure-conditioned models fit foundation_model)
- CARP (76M) AMFR official snapshot: attributes.origin: Origin unreported (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (76M) AMFR official snapshot: description: Description Official ProteinGym assay-level Spearman for the AMFR assay at commit 144fe22; transcribed, not reproduced. (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (600K) [ProteinGym 144fe22b snapshot] family identity: Configuration of CARP (Pinned ProteinGym Spearman CSV header: CARP (600K))
- CARP (600K) [ProteinGym 144fe22b snapshot]: facets.method_types: Facets: method types ["foundation_model"] (Proposer's report, ProteinGym AMFR: protein language and structure-conditioned models fit foundation_model)
- CARP (600K) AMFR official snapshot: attributes.origin: Origin unreported (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (600K) AMFR official snapshot: description: Description Official ProteinGym assay-level Spearman for the AMFR assay at commit 144fe22; transcribed, not reproduced. (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (640M) [ProteinGym 144fe22b snapshot] family identity: Configuration of CARP (Pinned ProteinGym Spearman CSV header: CARP (640M))
- CARP (640M) [ProteinGym 144fe22b snapshot]: facets.method_types: Facets: method types ["foundation_model"] (Proposer's report, ProteinGym AMFR: protein language and structure-conditioned models fit foundation_model)
- CARP (640M) AMFR official snapshot: attributes.origin: Origin unreported (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP (640M) AMFR official snapshot: description: Description Official ProteinGym assay-level Spearman for the AMFR assay at commit 144fe22; transcribed, not reproduced. (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- CARP: description: Description Convolutional protein language model pretrained by masked language modelling on protein sequences, scored in ProteinGym as a zero-shot single-sequence substitution baseline. (uc20260930-source-proteingym-amfr-spearman: Benchmark CSV header (model scored in the zero-shot DMS substitution benchmark); run-doc-proteingym-readme-md-144fe22b: README 'Leaderboard' baseline table (model name, input modalities, reference); qa-descriptions-20261010-source-carp: abstract)
- DeepSequence (ensemble) [ProteinGym 144fe22b snapshot] family identity: Configuration of DeepSequence (Pinned ProteinGym Spearman CSV header: DeepSequence (ensemble))
- DeepSequence (ensemble) AMFR official snapshot: attributes.origin: Origin unreported (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
- DeepSequence (ensemble) AMFR official snapshot: description: Description Official ProteinGym assay-level Spearman for the AMFR assay at commit 144fe22; transcribed, not reproduced. (DMS_substitutions_Spearman_DMS_level.csv: per-model columns with no scorer field; README.md at 144fe22, 'New baselines' (lines 159-170) and Acknowledgements (line 222))
Models, benchmarks and other records
Models (34)
Methods (4)
Configurations (97)
- AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]
- CARP (38M) [ProteinGym 144fe22b snapshot]
- CARP (76M) [ProteinGym 144fe22b snapshot]
- CARP (600K) [ProteinGym 144fe22b snapshot]
- CARP (640M) [ProteinGym 144fe22b snapshot]
- DeepSequence (ensemble) [ProteinGym 144fe22b snapshot]
- DeepSequence (single) [ProteinGym 144fe22b snapshot]
- ESCOTT [ProteinGym 144fe22b snapshot]
- ESM-1b [ProteinGym 144fe22b snapshot]
- ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]
- ESM-1v (single) [ProteinGym 144fe22b snapshot]
- ESM-C (300M) [ProteinGym 144fe22b snapshot]
- ESM-C (600M) [ProteinGym 144fe22b snapshot]
- ESM-IF1 [ProteinGym 144fe22b snapshot]
- ESM2 (3B) [ProteinGym 144fe22b snapshot]
- ESM2 (8M) [ProteinGym 144fe22b snapshot]
- ESM2 (15B) [ProteinGym 144fe22b snapshot]
- ESM2 (35M) [ProteinGym 144fe22b snapshot]
- ESM2 (150M) [ProteinGym 144fe22b snapshot]
- ESM2 (650M) [ProteinGym 144fe22b snapshot]
- ESM3 open (1.4B) [ProteinGym 144fe22b snapshot]
- EVE (ensemble) [ProteinGym 144fe22b snapshot]
- EVE (single) [ProteinGym 144fe22b snapshot]
- EVmutation [ProteinGym 144fe22b snapshot]
- GEMME [ProteinGym 144fe22b snapshot]
Protocols (1)
Dataset subsets (1)
Baselines (2)
Evaluations (97)
- AIDO Protein-RAG (16B) AMFR official snapshot
- CARP (38M) AMFR official snapshot
- CARP (76M) AMFR official snapshot
- CARP (600K) AMFR official snapshot
- CARP (640M) AMFR official snapshot
- DeepSequence (ensemble) AMFR official snapshot
- DeepSequence (single) AMFR official snapshot
- ESCOTT AMFR official snapshot
- ESM-1b AMFR official snapshot
- ESM-1v (ensemble) AMFR official snapshot
- ESM-1v (single) AMFR official snapshot
- ESM-C (300M) AMFR official snapshot
- ESM-C (600M) AMFR official snapshot
- ESM-IF1 AMFR official snapshot
- ESM2 (3B) AMFR official snapshot
- ESM2 (8M) AMFR official snapshot
- ESM2 (15B) AMFR official snapshot
- ESM2 (35M) AMFR official snapshot
- ESM2 (150M) AMFR official snapshot
- ESM2 (650M) AMFR official snapshot
- ESM3 open (1.4B) AMFR official snapshot
- EVE (ensemble) AMFR official snapshot
- EVE (single) AMFR official snapshot
- EVmutation AMFR official snapshot
- GEMME AMFR official snapshot
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
15 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Artifact SHA-256 f432423b87f79ac9778dfac86e3d95be041d246bc618dc0a406b35b0b7466437 Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Artifact url https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/DMS_substitutions_Spearman_DMS_level.csv Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Retrieved at 2026-09-30 Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Review method1 values
| ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Review: scope Only explicitly extracted table rows and protocol methods; no execution. Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Url https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/DMS_substitutions_Spearman_DMS_level.csv Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Version 144fe22b07dfaeec2b366f2346203a9838a55b4c Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Description Primary source retrieved and inspected for this audit. Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Search facets None recorded Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Record ID uc20260930-source-proteingym-amfr-spearman Source metadata | ProteinGym AMFR assay-level Spearman comparison No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c | catalogued No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-21-adc15377fab2 · Record review: source checked
Primary source retrieved and inspected for this audit.
Technical metadata and extraction receipts
Stable ID: uc20260930-source-proteingym-amfr-spearman
- url
- https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/DMS_substitutions_Spearman_DMS_level.csv
- artifact url
- https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/DMS_substitutions_Spearman_DMS_level.csv
- retrieved at
- 2026-09-30
- version
- 144fe22b07dfaeec2b366f2346203a9838a55b4c
- artifact sha256
- f432423b87f79ac9778dfac86e3d95be041d246bc618dc0a406b35b0b7466437
- review
- method: automated-source-review; scope: Only explicitly extracted table rows and protocol methods; no execution.