Model type
Masked-token protein transformer encoder
Configuration
ESM-2 (15B)
ESM-2 is a family of protein sequence encoders that produce representations for downstream protein analyses.
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.jsonHow it worksESM-2 workflow
Conceptual summary of the documented data flow; optional inputs and configured downstream stages must be reported for a reproducible evaluation.
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.jsonOverview
Inputs
Single amino-acid sequences.
Outputs
Residue embeddings, sequence representations and masked-token predictions.
Access
Official project documentation and implementation: https://github.com/facebookresearch/esm
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.jsonlimited source coverage · Automated source review, 2026-09-23. All specifications and missing details
Evaluations and results
1 evaluation · 5 results. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: ESM-2 (15B) | Task: ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) | 0.72 auc fraction · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 (15B) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Aggregation: Not reported ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, AUC) |
| Configuration: ESM-2 (15B) | Task: ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) | 0.314 mcc correlation · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 (15B) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Aggregation: Not reported ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, MCC) |
| Configuration: ESM-2 (15B) | Task: ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10% Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) | 0.759 ndcg fraction · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 (15B) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10% Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Aggregation: Not reported ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, NDCG@10%) |
| Configuration: ESM-2 (15B) | Task: ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) | 0.208 recall fraction · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 (15B) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Aggregation: Not reported ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, top 10% recall) |
| Configuration: ESM-2 (15B) | Task: ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman Dataset subset: ProteinGym substitution DMS assays (ProteinGym split) | 0.401 spearman correlation · higher Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceESM-2 (15B) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman Zero-shot scoring of substitution assays, averaged over assays with the correction the paper describes. Aggregation: Not reported ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Table 2, row(ESM-2 (15B)), column(Zero-shot substitutions, Spearman) |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Use this model
How it works, versions and access
Related profile: ESM-2. This page retains the exact record and its evaluation context.
This configuration
Fitness prediction model evaluated by the ProteinGym authors under their harness.
- record
- ESM-2 (15B)
- configuration
- Not reported
- entity type
- Configuration
How it works
How it works
ESM-2 tokenizes an amino-acid sequence and uses a transformer encoder trained to recover masked residues. Self-attention lets each residue representation depend on its sequence context. The released model returns token probabilities and embeddings; a specified pooling rule, task head or complete folding pipeline is needed for a particular biological prediction.
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.jsonVersions and reproducibility
ESM-2 checkpoint identifiers encode layer count, parameter scale and training-data tag. The checked esm2_t33_650M_UR50D configuration lists max_position_embeddings=1,026. This configuration field includes model positions and is not a claim of training or validated inference on 1,026 amino acids.
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.jsonStrengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
- Released checkpoints span several sizes and can be used without constructing a multiple sequence alignment.
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json
Limitations and conditions
- A general embedding is not a directly measured function or structure. Fine-tuning, pooling and downstream heads remain part of each evaluated configuration.
Sources (3)
facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json
Profile review details
Follow-up review of Reference checkpoint, Context limits, Training data release, Training cutoff. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.
Stable record: discovery-model-esm-2Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-23. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Model type | Masked-token protein transformer encoderSources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Architecture | Masked-token protein transformer encoder; the checked 650M checkpoint has 33 layers, hidden width 1,280, 20 attention heads and rotary positional encoding.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Inputs | Single amino-acid sequences.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Outputs | Residue embeddings, sequence representations and masked-token predictions.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Parameters | Released scales: 8M, 35M, 150M, 650M, 3B and 15B.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Known versions | ESM-2 checkpoint identifiers encode layer count, parameter scale and training-data tag.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Training data | UniRef50 clusters with UniRef90 sampling; the pretrained-model table labels UR50/D 2021_04.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Training cutoff | The inspected official table establishes an April 2021 UniRef release, but no separate latest-deposited-sequence date. Keep the corpus release distinct from a temporal leakage guarantee. · Not reported in inspected sourcesSourcesfacebookresearch/esm: README.md · README: Pre-trained Models and Pre-training Dataset Split |
| Context limits | The official extraction script truncates to 1,022 residues by default; --truncation_seq_length is configurable. This workflow default is not a universal architecture or validated biological-context limit.Sourcesesm2 extract: primary artifact · create_parser: --truncation_seq_length; run: get_batch_converter |
| Weights licence | The official facebook/esm2_t33_650M_UR50D model card declares MIT; this is the inspected checkpoint, not a licence inference from source code.Sources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Access | Official project documentation and implementation: https://github.com/facebookresearch/esmSources (3)facebookresearch/esm: README.md; facebook/esm2_t33_650M_UR50D: README.md; facebook/esm2_t33_650M_UR50D: config.json · ESM README: Pre-trained Models and Main models; official facebook/esm2_t33_650M_UR50D README licence metadata and config.json |
| Code licence | MITSourcesfacebookresearch/esm: LICENSE · LICENSE: licence text |
| Reference checkpoint | Example release: facebook/esm2_t33_650M_UR50D at 08e4846e537177426273712802403f7ba8261b6c. Its model.safetensors has registry-reported SHA-256 a08adabb949fa67ad3c14b509d04fd60368b35007b0095e3358f81200c4f4db0. This identifies a downloadable 650M checkpoint, not every ESM-2 evaluation.Sourcesesm2 release: primary artifact · sha; siblings[model.safetensors].lfs.sha256 |
| Training data release | The official pretrained-model table labels ESM-2 training data UR50/D 2021_04.Sourcesfacebookresearch/esm: README.md · README: Pre-trained Models table, ESM-2 rows |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
1 evidence row matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Relationship: family discovery-model-esm-2 Individual claims | ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design Section 4.1 Zero-shot benchmarks; Table 2 named model row Version: Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256 | source checked automated source review · 2026-09-23 Audit detailsSource review establishes this relationship only. Exact evaluated configurations and original numerical review status remain unchanged. The primary table and baseline descriptions identify the evaluated family. ESM-1v remains an ensemble; ESM-2 remains 15B; inverse-folding conditions remain distinct. Field: Claim: model-evaluation-identity-d005adff79d693934315 Source artifact SHA-256: Hash scope: Exact retrieved primary paper artifact bytes. |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: source checked
1 source records and release history
- ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design · Original source · Primary full-text snapshot retrieved 2026-09-17; exact bytes pinned by SHA-256
Technical metadata and extraction receipts
Stable ID: proteingym-method-esm-2-15b
- areas
- proteins-complexes
- source locator
- Table 2, row(ESM-2 (15B))
- missing metadata
- checkpoint revision: unreported; parameters: unextracted
Related records
- family: ESM-2
- subject: ESM-2 (15B): family ESM-2
- model: ESM-2 (15B) on ProteinGym ZS-SUB-AUC: Zero-shot substitutions, AUC
- model: ESM-2 (15B) on ProteinGym ZS-SUB-MCC: Zero-shot substitutions, MCC
- model: ESM-2 (15B) on ProteinGym ZS-SUB-NDCG: Zero-shot substitutions, NDCG@10%
- model: ESM-2 (15B) on ProteinGym ZS-SUB-RECALL: Zero-shot substitutions, top 10% recall
- model: ESM-2 (15B) on ProteinGym ZS-SUB-SPEARMAN: Zero-shot substitutions, Spearman