ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC

Overview

Zero-shot substitutions, MCC. Scored with MCC on ProteinGym substitution DMS assays. Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

Consult the linked sources for architecture or protocol details. Missing evidence is not evidence of a missing capability.

Results

Each comparison retains its reviewed evaluation scope, dataset and metric. Results are shown without a pooled ranking.

Evaluation scope and datasetMetric and reported setup

ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC

mcc (correlation) · Higher values are better.

ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC · ProteinGym substitution DMS assays (ProteinGym split)

Evidence origin: Author-reported evaluation.

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, column(Zero-shot substitutions, MCC)

MSE is better when lower; every other metric here is better when higher.
Zero-shot, supervised and indel benchmarks use different protocols and assay sets, so their figures are not comparable to each other.

Comparison details and limitations

Every method ProteinGym reports on Zero-shot substitutions, MCC, scored with MCC on ProteinGym substitution DMS assays.

Author-reported numbers, source checked but not independently reproduced.

Automated source review: 2026-09-18. Numerical source review does not establish independent reproduction.

Dots show point estimates. Whiskers show only explicitly defined uncertainty (standard deviation, standard error or a labelled interval); their definitions remain in Table. Unresolved uncertainty is not plotted. Differences do not establish statistical significance.

Find a tested model or method

Showing 12 of 21 matching rows.

Zoom to observed range

Tested configuration

-1-0.500.51

Reported score

Methods and evaluation design

Procedure, tasks and evaluated configurations

Evaluation design

Benchmarks bring together tasks and protocols. A task describes the biological question; a protocol defines a particular test.

Benchmarks

ProteinGym

These source-backed links do not make different protocols or scores interchangeable.

Recorded evaluations

Each evaluation records what was tested and under which conditions.

Explore all linked results

Run instructions

No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.

A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

1 evidence row matching the loaded filters

Claims, original sources and review scope · Release 2026-09-29-06401fd5b220
Property and statement	Original source and location	Review and provenance
Relationship: part of discovery-benchmark-proteingym Individual claims	ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Original source ↗ Table 2, column(Zero-shot substitutions, MCC) Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 Retrieved: 2026-09-17T08:06:28.522924+00:00	source checked automated source review · 2026-09-18 Audit details Primary-source transcription with no human sign-off and no independent reproduction. Field: `links:part_of:discovery-benchmark-proteingym` Claim: proteingym-association-zs-sub-mcc Source artifact SHA-256: `a3b08cc4a6befd64620cf0f287d78d55a36dc2639955f52c5655f83833c50104` Hash scope: Exact retrieved primary paper artifact bytes. Inspected artifact

All evidence (CSV)All evidence (JSONL)How to read the evidence

Sources and history

View linked audit checks and correction history

Release 2026-09-29-06401fd5b220 · Record review: source checked

1 source records and release history

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Original source · Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256

Download this release

Technical metadata and extraction receipts

Stable ID: proteingym-task-zs-sub-mcc

areas: proteins-complexes
tasks: Zero-shot substitutions, MCC
metric: MCC
metric direction: higher
dataset: ProteinGym substitution DMS assays
protocol: Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.
source locator: Table 2, column(Zero-shot substitutions, MCC)
comparison panels: id: proteingym-panel-zs-sub-mcc; title: ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC; protocol id: proteingym-task-zs-sub-mcc; dataset id: proteingym-dataset-proteingym-substitution-dms-assays; metric: mcc; unit: correlation; direction: higher; result ids: proteingym-result-site-independent-zs-sub-mcc-mcc; proteingym-result-wavenet-zs-sub-mcc-mcc; proteingym-result-evmutation-zs-sub-mcc-mcc; proteingym-result-deepsequence-ensemble-zs-sub-mcc-mcc; proteingym-result-eve-ensemble-zs-sub-mcc-mcc; proteingym-result-gemme-zs-sub-mcc-mcc; proteingym-result-unirep-zs-sub-mcc-mcc; proteingym-result-carp-640m-zs-sub-mcc-mcc; proteingym-result-esm-1b-zs-sub-mcc-mcc; proteingym-result-esm-2-15b-zs-sub-mcc-mcc; proteingym-result-rita-xl-zs-sub-mcc-mcc; proteingym-result-esm-1v-ensemble-zs-sub-mcc-mcc; proteingym-result-progen2-xl-zs-sub-mcc-mcc; proteingym-result-vespa-zs-sub-mcc-mcc; proteingym-result-unirep-evotuned-zs-sub-mcc-mcc; proteingym-result-msa-transformer-ensemble-zs-sub-mcc-mcc; proteingym-result-tranception-l-zs-sub-mcc-mcc; proteingym-result-trancepteve-l-zs-sub-mcc-mcc; proteingym-result-esm-if1-zs-sub-mcc-mcc; proteingym-result-mif-st-zs-sub-mcc-mcc; proteingym-result-proteinmpnn-zs-sub-mcc-mcc; source ids: evidence-expansion-proteingym-a3b08cc4; source locator: Table 2, column(Zero-shot substitutions, MCC); context: Every method ProteinGym reports on Zero-shot substitutions, MCC, scored with MCC on ProteinGym substitution DMS assays.; caveats: Author-reported numbers, source checked but not independently reproduced.; MSE is better when lower; every other metric here is better when higher.; Zero-shot, supervised and indel benchmarks use different protocols and assay sets, so their figures are not comparable to each other.; review: method: automated_source_review; date: 2026-09-18

Related records

part of: ProteinGym
subject: ProteinGym ZS-SUB-MCC: part of discovery-benchmark-proteingym
benchmark: CARP (640M) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: DeepSequence (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: ESM-1b on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: ESM-1v (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: ESM-2 (15B) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: ESM-IF1 on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: EVE (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: EVmutation on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: GEMME on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: MIF-ST on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: MSA Transformer (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: ProGen2 XL on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: ProteinMPNN on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: RITA XL on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: Site-Independent on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: TranceptEVE L on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: Tranception L on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: UniRep evotuned on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: UniRep on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: VESPA on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
benchmark: WaveNet on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC