rewirebio.iobenchmarks
Model

ESM-1v

ESM-1v provides protein language models intended for zero-shot variant-effect scoring.

Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models

5 evaluations · 43 results1 configuration with further results

How it worksESM-1v workflow
ESM-1v workflow1. Reference protein sequence. Then: 2. ESM-1v token predictions. Then: 3. Specified variant scoring rule. Then: 4. Variant scoreESM-1v workflow1. Reference protein sequence. Then: 2. ESM-1v token predictions. Then: 3. Specified variant scoring rule. Then: 4. Variant scoreESM-1v workflow1. Reference protein sequence. Then: 2. ESM-1v token predictions. Then: 3. Specified variant scoring rule. Then: 4. Variant score

Conceptual summary of the documented data flow; optional inputs and configured downstream stages must be reported for a reproducible evaluation.

Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Overview

Model type

Protein transformer for variant-effect scoring

Inputs

Protein amino-acid sequence and the specified sequence variation.

Outputs

Token probabilities used to calculate sequence-variant scores.

Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models

limited source coverage · Automated source review, 2026-09-16. All specifications and missing details

Evaluations and results

5 evaluations · 43 results. Different protocols are not a single leaderboard.

Sorted by Spearman (higher is better). The best value in each column is highlighted. Decimals are rounded for display; each value links to the printed value and its source.

Tested configurationProtocol and datasetSpearmanSpearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays))Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, variants with 3 substitutions: mean over protein and selection-type groups of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, variants with 4 substitutions: mean over protein and selection-type groups of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, variants with 5 or more substitutions: mean over protein and selection-type groups of assay-level Spearman)Spearman (ProteinGym zero-shot DMS substitutions, Virus proteins: mean over proteins of assay-level Spearman)AUROCMCCNDCG at 10%Top 10% recall
ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]EvaluationProteinGym AMFR official zero-shot Spearman snapshot, ProteinGym AMFR 4G3O official snapshot cohort0.423 (best in column)Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported
ESM-1v (ensemble)EvaluationProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman, ProteinGym substitution DMS assays (ProteinGym split)0.407Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported
ESM-1v (single) [ProteinGym 144fe22b snapshot]EvaluationProteinGym AMFR official zero-shot Spearman snapshot, ProteinGym AMFR 4G3O official snapshot cohort0.07Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported
ESM-1v (ensemble)EvaluationProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC, ProteinGym substitution DMS assays (ProteinGym split)Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported0.723Not reportedNot reportedNot reported
ESM-1v (ensemble)EvaluationProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC, ProteinGym substitution DMS assays (ProteinGym split)Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported0.32Not reportedNot reported
ESM-1v (ensemble)EvaluationProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%, ProteinGym substitution DMS assays (ProteinGym split)Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported0.749Not reported
ESM-1v (ensemble)EvaluationProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall, ProteinGym substitution DMS assays (ProteinGym split)Not reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reportedNot reported0.211
ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]EvaluationProteinGym v1.3 zero-shot DMS substitutions, ProteinGym DMS substitution assays at commit 144fe22 (217 assays)Not reported0.42 (best in column)0.32 (best in column)0.429 (best in column)0.495 (best in column)0.458 (best in column)0.316 (best in column)0.407 (best in column)0.409 (best in column)0.386 (best in column)0.464 (best in column)0.413 (best in column)0.477 (best in column)0.403 (best in column)0.215 (best in column)0.1730.153 (best in column)0.218 (best in column)0.294 (best in column)Not reportedNot reportedNot reportedNot reported
ESM-1v (single) [ProteinGym 144fe22b snapshot]EvaluationProteinGym v1.3 zero-shot DMS substitutions, ProteinGym DMS substitution assays at commit 144fe22 (217 assays)Not reported0.3960.2680.4050.470.4270.2890.3740.3730.3610.4330.3850.4370.3750.1880.177 (best in column)0.1460.2090.258Not reportedNot reportedNot reportedNot reported
All 43 result rows with coverage, uncertainty and sources
Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: ESM-1v (ensemble)Task: ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
Dataset subset: ProteinGym substitution DMS assays (ProteinGym split)
0.723 AUROC
unitless · higher

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

ESM-1v (ensemble) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC

Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

Aggregation: Not reported

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, AUC)
Configuration: ESM-1v (ensemble)Task: ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
Dataset subset: ProteinGym substitution DMS assays (ProteinGym split)
0.32 MCC
unitless · higher

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

ESM-1v (ensemble) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC

Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

Aggregation: Not reported

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, MCC)
Configuration: ESM-1v (ensemble)Task: ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
Dataset subset: ProteinGym substitution DMS assays (ProteinGym split)
0.749 NDCG at 10%
unitless · higher

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

ESM-1v (ensemble) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%

Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

Aggregation: Not reported

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, NDCG@10%)
Configuration: ESM-1v (ensemble)Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
Dataset subset: ProteinGym substitution DMS assays (ProteinGym split)
0.211 Top 10% recall
fraction · higher

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

ESM-1v (ensemble) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall

Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

Aggregation: Not reported

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, top 10% recall)
Configuration: ESM-1v (ensemble)Task: ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman
Dataset subset: ProteinGym substitution DMS assays (ProteinGym split)
0.407 Spearman
unitless · higher

Uncertainty: Not yet extracted

Coverage: Not reported scored / Not reported eligible

Independent external evaluation · Source checked
Methods, coverage and source

ESM-1v (ensemble) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman

Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes.

Aggregation: Not reported

ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-1v (ensemble)), column(Zero-shot substitutions, Spearman)
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym AMFR official zero-shot Spearman snapshot
Dataset subset: ProteinGym AMFR 4G3O official snapshot cohort
0.423 Spearman
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble) AMFR official snapshot

Not reported

Aggregation: Not reported

ProteinGym AMFR assay-level Spearman comparison · DMS_substitutions_Spearman_DMS_level.csv line 15, DMS ID=AMFR_HUMAN_Tsuboyama_2023_4G3O, column=ESM-1v (ensemble)
Configuration: ESM-1v (single) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym AMFR official zero-shot Spearman snapshot
Dataset subset: ProteinGym AMFR 4G3O official snapshot cohort
0.07 Spearman
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (single) AMFR official snapshot

Not reported

Aggregation: Not reported

ProteinGym AMFR assay-level Spearman comparison · DMS_substitutions_Spearman_DMS_level.csv line 15, DMS ID=AMFR_HUMAN_Tsuboyama_2023_4G3O, column=ESM-1v (single)
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.407 Spearman (ProteinGym zero-shot DMS substitutions, mean over selection types of the per-type mean over proteins of assay-level Spearman (all 217 assays))
unitless · higher

Uncertainty: SE 0.013. From compute_bootstrap_standard_error_functional_categories in performance_DMS_benchmarks.py at 144fe22 (default 10,000 resamples, lines 95-111 and 310-315); not an absolute standard error of the model's own average.

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Average_Spearman'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.403 Spearman (ProteinGym zero-shot DMS substitutions, variants with 1 substitution: mean over protein and selection-type groups of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Depth_1'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.215 Spearman (ProteinGym zero-shot DMS substitutions, variants with 2 substitutions: mean over protein and selection-type groups of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Depth_2'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.173 Spearman (ProteinGym zero-shot DMS substitutions, variants with 3 substitutions: mean over protein and selection-type groups of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Depth_3'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.153 Spearman (ProteinGym zero-shot DMS substitutions, variants with 4 substitutions: mean over protein and selection-type groups of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Depth_4'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.218 Spearman (ProteinGym zero-shot DMS substitutions, variants with 5 or more substitutions: mean over protein and selection-type groups of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Depth_5+'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.42 Spearman (ProteinGym zero-shot DMS substitutions, Activity assays: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Function_Activity'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.32 Spearman (ProteinGym zero-shot DMS substitutions, Binding assays: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Function_Binding'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.429 Spearman (ProteinGym zero-shot DMS substitutions, Expression assays: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Function_Expression'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.386 Spearman (ProteinGym zero-shot DMS substitutions, Organismal fitness assays: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Function_OrganismalFitness'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.477 Spearman (ProteinGym zero-shot DMS substitutions, Stability assays: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Function_Stability'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.495 Spearman (ProteinGym zero-shot DMS substitutions, High MSA depth proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'High_MSA_depth'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.316 Spearman (ProteinGym zero-shot DMS substitutions, Low MSA depth proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Low_MSA_depth'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.409 Spearman (ProteinGym zero-shot DMS substitutions, Medium MSA depth proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Medium_MSA_depth'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.458 Spearman (ProteinGym zero-shot DMS substitutions, Human proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Taxa_Human'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.464 Spearman (ProteinGym zero-shot DMS substitutions, Other eukaryote proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Taxa_Other_Eukaryote'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.413 Spearman (ProteinGym zero-shot DMS substitutions, Prokaryote proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Taxa_Prokaryote'
Configuration: ESM-1v (ensemble) [ProteinGym 144fe22b snapshot]Protocol: ProteinGym v1.3 zero-shot DMS substitutions
Dataset subset: ProteinGym DMS substitution assays at commit 144fe22 (217 assays)
0.294 Spearman (ProteinGym zero-shot DMS substitutions, Virus proteins: mean over proteins of assay-level Spearman)
unitless · higher

Uncertainty: Not reported by the source

Coverage: Not reported scored / Not reported eligible

Evaluation origin not reported · Source checked
Methods, coverage and source

ESM-1v (ensemble): ProteinGym zero-shot DMS substitutions summary (144fe22)

ProteinGym zero-shot DMS substitutions, Spearman summary

Aggregation: Not reported

ProteinGym zero-shot DMS substitutions Spearman summary at commit 144fe22 · benchmarks/DMS_zero_shot/substitutions/Spearman/Summary_performance_DMS_substitutions_Spearman.csv line 48, row 'ESM-1v (ensemble)', column 'Taxa_Virus'

Source checking is not independent reproduction. Release 2026-10-22-1b7cf027e4f6.

Further results on linked records

One linked record has results that are not included above, because no reviewed claim yet ties the link to this model. Each record keeps its own settings and evaluations, and its scores are on its own page.

Related configurations, pipelines and services

These configurations, services and pipelines use this model within their own configurations. Their results, where available, are not assigned to the underlying model.

Use this model

How it works, versions and access

Versions and evaluated configurations

How it works

How it works

Transformer architecture shared with ESM-1b, trained on UniRef90; multiple released model instances support the variant-scoring workflow. The documented inputs are protein amino-acid sequence and the specified sequence variation. The output consists of token probabilities used to calculate sequence-variant scores.

Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Versions and reproducibility

esm1v_t33_650M_UR90S_1 through _5. The applicable input limits require configuration-specific checking.

Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Strengths, limitations and unresolved questions

Strengths and limitations

Profile review details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Stable record: discovery-model-esm-1v

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Inputs, outputs and configuration
PropertyDescription and evidence
Model typeProtein transformer for variant-effect scoring
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
ArchitectureTransformer architecture shared with ESM-1b, trained on UniRef90; multiple released model instances support the variant-scoring workflow.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
InputsProtein amino-acid sequence and the specified sequence variation.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
OutputsToken probabilities used to calculate sequence-variant scores.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Parameters650 million per model; 33 transformer layers.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Known versionsesm1v_t33_650M_UR90S_1 through _5.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Training dataUniRef90/S 2020_03, as reported in the pretrained-model table.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Training cutoffThe official model table identifies the UniRef90/S2020_03 training release; this is a corpus version rather than a verified latest-sequence deposition date.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Context limitsThe inspected official esm1v_t33_650M_UR90S_1 config reserves 1,026 absolute positions. This includes model positions and is not a claim of 1,026-residue validated biological context.
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Weights licenceSeparate checkpoint-distribution terms are not stated in the inspected release documentation and licence material. The source-code licence alone is not recorded as an explicit weight grant. · Not reported in inspected sources
Sources (3)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json; facebookresearch/esm: LICENSE · README.md: Main models, Zero-shot variant prediction and Pre-trained Models; LICENSE: licence text
AccessOfficial project documentation and implementation: https://github.com/facebookresearch/esm
Sources (2)facebookresearch/esm: README.md; facebook/esm1v_t33_650M_UR90S_1: config.json · README.md: Main models, Zero-shot variant prediction and Pre-trained Models
Code licenceMIT
Sourcesfacebookresearch/esm: LICENSE · LICENSE: licence text
Applicable tests and references

Applicability is distinct from a completed evaluation.

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

40 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statementOriginal source and locationReview and provenance
Diagram caption
Conceptual summary of the documented data flow; optional inputs and configured downstream stages must be reported for a reproducible evaluation.
Individual claims
facebook/esm1v_t33_650M_UR90S_1: config.json

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 8bfdb1892536cc77bd0760b9c25ddced2cd0b4c8
Retrieved: 2026-09-16T20:12:20.912868+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 6c4b576e2e73ad85fc51a4fa75d4e350532b773e04551307a74643c3fc3c5d59

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram caption
Conceptual summary of the documented data flow; optional inputs and configured downstream stages must be reported for a reproducible evaluation.
Individual claims
facebookresearch/esm: README.md

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac
Retrieved: 2026-09-16T19:46:19.090082+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.diagram.caption

Source artifact SHA-256: 8b273c21a322fc9473d1b68d0dd40c8166ab2f89e4a190aa26ca87251b97cba9

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram steps
  • Reference protein sequence
  • ESM-1v token predictions
  • Specified variant scoring rule
  • Variant score
Individual claims
facebook/esm1v_t33_650M_UR90S_1: config.json

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 8bfdb1892536cc77bd0760b9c25ddced2cd0b4c8
Retrieved: 2026-09-16T20:12:20.912868+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 6c4b576e2e73ad85fc51a4fa75d4e350532b773e04551307a74643c3fc3c5d59

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram steps
  • Reference protein sequence
  • ESM-1v token predictions
  • Specified variant scoring rule
  • Variant score
Individual claims
facebookresearch/esm: README.md

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac
Retrieved: 2026-09-16T19:46:19.090082+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.diagram.steps

Source artifact SHA-256: 8b273c21a322fc9473d1b68d0dd40c8166ab2f89e4a190aa26ca87251b97cba9

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram title
ESM-1v workflow
Individual claims
facebook/esm1v_t33_650M_UR90S_1: config.json

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 8bfdb1892536cc77bd0760b9c25ddced2cd0b4c8
Retrieved: 2026-09-16T20:12:20.912868+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.diagram.title

Source artifact SHA-256: 6c4b576e2e73ad85fc51a4fa75d4e350532b773e04551307a74643c3fc3c5d59

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Diagram title
ESM-1v workflow
Individual claims
facebookresearch/esm: README.md

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac
Retrieved: 2026-09-16T19:46:19.090082+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.diagram.title

Source artifact SHA-256: 8b273c21a322fc9473d1b68d0dd40c8166ab2f89e4a190aa26ca87251b97cba9

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Model type
Protein transformer for variant-effect scoring
Individual claims
facebook/esm1v_t33_650M_UR90S_1: config.json

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 8bfdb1892536cc77bd0760b9c25ddced2cd0b4c8
Retrieved: 2026-09-16T20:12:20.912868+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: 6c4b576e2e73ad85fc51a4fa75d4e350532b773e04551307a74643c3fc3c5d59

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Model type
Protein transformer for variant-effect scoring
Individual claims
facebookresearch/esm: README.md

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac
Retrieved: 2026-09-16T19:46:19.090082+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.facts.0.value

Source artifact SHA-256: 8b273c21a322fc9473d1b68d0dd40c8166ab2f89e4a190aa26ca87251b97cba9

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Architecture
Transformer architecture shared with ESM-1b, trained on UniRef90; multiple released model instances support the variant-scoring workflow.
Individual claims
facebook/esm1v_t33_650M_UR90S_1: config.json

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 8bfdb1892536cc77bd0760b9c25ddced2cd0b4c8
Retrieved: 2026-09-16T20:12:20.912868+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: 6c4b576e2e73ad85fc51a4fa75d4e350532b773e04551307a74643c3fc3c5d59

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

Architecture
Transformer architecture shared with ESM-1b, trained on UniRef90; multiple released model instances support the variant-scoring workflow.
Individual claims
facebookresearch/esm: README.md

Original source ↗

README.md: Main models, Zero-shot variant prediction and Pre-trained Models

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 2b369911bb5b4b0dda914521b9475cad1656b2ac
Retrieved: 2026-09-16T19:46:19.090082+00:00

source checked

automated source review · 2026-09-16

Audit details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Field: attributes.profile.facts.1.value

Source artifact SHA-256: 8b273c21a322fc9473d1b68d0dd40c8166ab2f89e4a190aa26ca87251b97cba9

Hash scope: SHA-256 of retrieved original artifact bytes

Format: original_artifact

Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: discovered

4 source records and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: discovery-model-esm-1v

areas
protein-function
access
official_source_linked
candidate benchmark ids
discovery-benchmark-proteingym
entity level
family
reported name
ESM-1v
entity classification
review date: 2026-09-17; rationale: The cited profile describes a named learned biological predictor or representation model/family. Preserve this identity separately from task-specific fitting, individual checkpoints, pipelines and hosted access.; source ids: evidence-official-29d4b229a3aa426a6dfb; evidence-official-137a3a936ffa4673b3d3; source locator: README.md: Main models, Zero-shot variant prediction and Pre-trained Models; ambiguities: None recorded
missing metadata
version: reason: unextracted
Related records

Suggest a correction