Source

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22

144fe22b07dfaeec2b366f2346203a9838a55b4c · Project source

What this source contributed

Records in this release that cite this source. A record can cite several sources.

97 evaluations
1,746 results
2 evidence claims
1 use case

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Value	Metric	Model or configuration	Dataset	Evaluation
0.517 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.426 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.522 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.585 unitless	Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.531 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.498 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.518 unitless	Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays))	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.534 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.491 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.587 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.558 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.635 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.527 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.414 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.419 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 3 substitutions: mean over protein and selection-type groups of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.394 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 4 substitutions: mean over protein and selection-type groups of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.414 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 5 or more substitutions: mean over protein and selection-type groups of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.522 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Virus proteins: mean over proteins of assay-level Spearman)	AIDO Protein-RAG (16B) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	AIDO Protein-RAG (16B): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.285 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.268 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.312 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.354 unitless	Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.322 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.206 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.28 unitless	Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays))	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.24 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.218 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.307 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.253 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.314 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.279 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.173 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.106 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 3 substitutions: mean over protein and selection-type groups of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.126 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 4 substitutions: mean over protein and selection-type groups of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.143 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 5 or more substitutions: mean over protein and selection-type groups of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.127 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Virus proteins: mean over proteins of assay-level Spearman)	CARP (38M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (38M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.342 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.282 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.369 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.403 unitless	Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.388 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.254 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.328 unitless	Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays))	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.302 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.271 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.366 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.301 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.376 unitless	Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.336 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)
0.206 unitless	Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman)	CARP (76M) [ProteinGym 144fe22b snapshot]	ProteinGym DMS substitution assays at commit 144fe22 (217 assays)	CARP (76M): ProteinGym zero-shot DMS substitutions summary (144fe22)

Claims

Statements this source supports, each with where in the source it was checked.

Relevance of rewire-proteingym-v1-3-dms-substitutions to use-case-protein-stability: Assessed by ProteinGym v1.3 zero-shot DMS substitutions (vep-lit-20261014-source-proteingym-dms-substitutions-spearman-summary-144fe22: Summary_performance_DMS_substitutions_Spearman.csv column Function_Stability; performance_DMS_benchmarks.py lines 336-348)
Relevance of rewire-proteingym-v1-3-dms-substitutions to use-case-protein-variant-effect-prediction: Assessed by ProteinGym v1.3 zero-shot DMS substitutions (vep-lit-20261014-source-proteingym-dms-substitutions-spearman-summary-144fe22: Summary_performance_DMS_substitutions_Spearman.csv; performance_DMS_benchmarks.py lines 269-348; run-doc-proteingym-readme-md-144fe22b: README line 22 (217 DMS assays) and lines 66-70)

Models, benchmarks and other records

Evaluations (97)

Use cases (1)

Predict missense variant effects in a protein of interest

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

26 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
attributes.artifact_sha256 ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4 Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_sha256` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Artifact url https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.artifact_url` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Licence MIT (repository LICENSE); underlying assay and clinical data carry their own terms Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.licence` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Limitations 2 values Mutable repository: the file is pinned to commit 144fe22b07dfaeec2b366f2346203a9838a55b4c, which was the head of main at retrieval (committed 2026-03-25). The repository does not record which script run, data release or model-score snapshot produced this file; the README says these summary files are what the website hosts. Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.limitations` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Media type text/csv Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.media_type` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Publication status official_project_source Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.publication_status` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieved at 2026-10-14T13:39:32Z Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.retrieved_at` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
attributes.review.artifact_sha256 ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4 Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.artifact_sha256` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review: method note Re-downloaded from the recorded URL on 2026-10-14 and matched the SHA-256; URL, version, licence and limitations checked. Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.method_note` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review method 2 values source-hash-verification ai-assisted-source-review Source metadata	ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 Original source ↗ No field-specific location recorded Version: 144fe22b07dfaeec2b366f2346203a9838a55b4c Retrieved: 2026-10-14T13:39:32Z	catalogued No individual claim review recorded Audit details Field: `attributes.review.method` Source artifact SHA-256: `ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Official ProteinGym leaderboard file: average Spearman over 217 DMS substitution assays for 97 zero-shot models, with function, MSA-depth, taxon and mutational-depth breakdowns and a bootstrap standard error.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: vep-lit-20261014-source-proteingym-dms-substitutions-spearman-summary-144fe22

areas: proteins-complexes
contexts: research
url: https://github.com/OATML-Markslab/ProteinGym/blob/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv
artifact url: https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv
version: 144fe22b07dfaeec2b366f2346203a9838a55b4c
retrieved at: 2026-10-14T13:39:32Z
artifact sha256: ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4
publication status: official_project_source
licence: MIT (repository LICENSE); underlying assay and clinical data carry their own terms
media type: text/csv
source locator: benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv
limitations: Mutable repository: the file is pinned to commit 144fe22b07dfaeec2b366f2346203a9838a55b4c, which was the head of main at retrieval (committed 2026-03-25).; The repository does not record which script run, data release or model-score snapshot produced this file; the README says these summary files are what the website hosts.
review: reviewer: claude; reviewer note: Claude review agent; no human review claimed. Independent Claude (Opus 5.5) review agent, separate from the extraction agent that built this batch. Automated review only. Transcription, not reproduction.; reviewed at: 2026-10-14T15:30:00Z; artifact sha256: ee61a7c09efbfbd8d9a6c9355092c798032a578fa58c6f6256b5da8778555fd4; retrieval url: https://raw.githubusercontent.com/OATML-Markslab/ProteinGym/144fe22b07dfaeec2b366f2346203a9838a55b4c/benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv; method: source-hash-verification; ai-assisted-source-review; method note: Re-downloaded from the recorded URL on 2026-10-14 and matched the SHA-256; URL, version, licence and limitations checked.; note: Hash matches.

Related records

Suggest a correction