ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman

Overview

Supervised substitutions, average over splits, Spearman. Scored with Spearman on ProteinGym substitution DMS assays. Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

Evaluation scope and datasetMetric and reported setup

ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman

spearman (correlation) · Higher values are better.

ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman · ProteinGym substitution DMS assays (ProteinGym split)

Evidence origin: Author-reported evaluation.

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 3, column(Supervised substitutions, average over splits, Spearman)

MSE is better when lower; every other metric here is better when higher.
Zero-shot, supervised and indel benchmarks use different protocols and assay sets, so their figures are not comparable to each other.

Comparison details and limitations

Every method ProteinGym reports on Supervised substitutions, average over splits, Spearman, scored with Spearman on ProteinGym substitution DMS assays.

Author-reported numbers, source checked but not independently reproduced.

Automated source review: 2026-09-18. Numerical source review does not establish independent reproduction.

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Find a tested model or method

Showing 10 of 10 matching rows.

Zoom to observed range

Tested configuration

-1-0.500.51

Reported score

Methods and evaluation design

Procedure, tasks and evaluated configurations

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Benchmarks

ProteinGym

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Relationship: part of discovery-benchmark-proteingym Individual claims	ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Original source ↗ Table 3, column(Supervised substitutions, average over splits, Spearman) Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 Retrieved: 2026-09-17T08:06:28.522924+00:00	source checked automated source review · 2026-09-18 Audit details Primary-source transcription with no human sign-off and no independent reproduction. Field: `links:part_of:discovery-benchmark-proteingym` Claim: proteingym-association-sup-sub-spearman-avg Source artifact SHA-256: `a3b08cc4a6befd64620cf0f287d78d55a36dc2639955f52c5655f83833c50104` Hash scope: Exact retrieved primary paper artifact bytes. Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: source checked

1 source records and release history

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Original source · Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256

Download this release

Technical metadata and extraction receipts

Stable ID: proteingym-task-sup-sub-spearman-avg

areas: proteins-complexes
tasks: Supervised substitutions, average over splits, Spearman
metric: Spearman
metric direction: higher
dataset: ProteinGym substitution DMS assays
protocol: Supervised fitness prediction under three cross-validation schemes: contiguous, modulo and random splits.
source locator: Table 3, column(Supervised substitutions, average over splits, Spearman)
comparison panels: id: proteingym-panel-sup-sub-spearman-avg; title: ProteinGym SUP-SUB-SPEARMAN-AVG: Supervised substitutions, average over splits, Spearman; protocol id: proteingym-task-sup-sub-spearman-avg; dataset id: proteingym-dataset-proteingym-substitution-dms-assays; metric: spearman; unit: correlation; direction: higher; result ids: proteingym-result-ohe-none-sup-sub-spearman-avg-spearman; proteingym-result-ohe-deepsequence-sup-sub-spearman-avg-spearman; proteingym-result-ohe-esm-1v-sup-sub-spearman-avg-spearman; proteingym-result-ohe-msat-sup-sub-spearman-avg-spearman; proteingym-result-ohe-tranception-sup-sub-spearman-avg-spearman; proteingym-result-ohe-trancepteve-sup-sub-spearman-avg-spearman; proteingym-result-embed-esm-1v-sup-sub-spearman-avg-spearman; proteingym-result-embed-msat-sup-sub-spearman-avg-spearman; proteingym-result-embed-tranception-sup-sub-spearman-avg-spearman; proteingym-result-npt-proteinnpt-sup-sub-spearman-avg-spearman; source ids: evidence-expansion-proteingym-a3b08cc4; source locator: Table 3, column(Supervised substitutions, average over splits, Spearman); context: Every method ProteinGym reports on Supervised substitutions, average over splits, Spearman, scored with Spearman on ProteinGym substitution DMS assays.; caveats: Author-reported numbers, source checked but not independently reproduced.; MSE is better when lower; every other metric here is better when higher.; Zero-shot, supervised and indel benchmarks use different protocols and assay sets, so their figures are not comparable to each other.; review: method: automated_source_review; date: 2026-09-18

Related records