Relevance of Mega-scale dataset stability ranking (Chu et al. 2024) to "Assess methods for protein stability experiments"
- Use case
- Assess methods for protein stability experiments
- Assessed by
- Mega-scale dataset stability ranking, Spearman correlation (Chu et al. 2024 Table 1)
- Relevance
- Direct evidence
- Endpoint
- Spearman correlation for Rosetta, MUPro, RaSP, ELASPIC-2 and two ESM-2 checkpoints on Mega-scale dataset
Held-out protein domains from the cDNA display proteolysis set directly measure how well each method ranks substitutions by folding stability.
6 reviewed evaluations
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
21 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group chu2024-transfer Context-only references | Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title Transfer from mega-scale domains to larger proteins (Chu et al. 2024) Context-only references | Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints2 values
| Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint Spearman correlation for Rosetta, MUPro, RaSP, ELASPIC-2 and two ESM-2 checkpoints on Mega-scale dataset Context-only references | Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Excluded evaluations1 values
| Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:protein-stability-20261009-protocol-chu2024-mega-scale Context-only references | Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric spearman-correlation Context-only references | Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations6 values
| Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records15 values
| Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-21-adc15377fab2 · Record review: source checked
1 source record and release history
- Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset · Original source · PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Technical metadata and extraction receipts
Stable ID: use-case-mapping-protein-stability-20261009-chu2024-mega-scale
- field
- links:assessed_by:protein-stability-20261009-protocol-chu2024-mega-scale
- value
- protein-stability-20261009-protocol-chu2024-mega-scale
- relevance
- direct
- endpoint
- Spearman correlation for Rosetta, MUPro, RaSP, ELASPIC-2 and two ESM-2 checkpoints on Mega-scale dataset
- rationale
- Held-out protein domains from the cDNA display proteolysis set directly measure how well each method ranks substitutions by folding stability.
- constraints
- Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; datasets, splits, structures and predictor versions differ between sources.
- limitations
- ESM therm was fine-tuned by the authors of this paper; the other methods were run by them as comparators.; Only Spearman correlation is printed; no RMSE, calibration or uncertainty.; The larger-protein benchmark was limited to six ProteinGym datasets and BglB because of Rosetta's compute cost (Methods P25).; ESM therm is excluded from this comparison: it was scored on point and multi-point variants, the other methods on point mutations only (Table 1 caption).; Metrics are aggregated over test-set-only domains of 40-72 residues, the same assay the model was trained on.; The paper does not say who produced the comparator scores on the mega-scale test-set-only domains.
- revision
- 1
- reason
- Add primary-source stability predictor comparison evidence from the protein stability use-case pass (2026-10-09).
- citation locators
- source id: protein-stability-20261009-source-chu2024; locator: Table 1 row 'Mega-scale dataset'; Table 2; Results
- source locator
- protein-stability-20261009-source-chu2024: Table 1 row 'Mega-scale dataset'; Table 2; Results
- comparison group
- chu2024-transfer
- comparison title
- Transfer from mega-scale domains to larger proteins (Chu et al. 2024)
- headline metric
- spearman-correlation
- stratum label
- Mega-scale held-out domains
- stratum order
- 1
- excluded evaluations
- id: protein-stability-20261009-eval-chu2024-esm-therm-mega-scale; reason: Table 1 caption: every other method is scored on point mutations only, while ESM therm on the mega-scale set is also scored on multi-point variants, so its row is not matched. Excluded from this comparison at independent review 2026-10-09; its result stays recorded.
- reviewed evaluations
- protein-stability-20261009-eval-chu2024-elaspic-2-mega-scale; protein-stability-20261009-eval-chu2024-esm-2-35m-mega-scale; protein-stability-20261009-eval-chu2024-esm-2-3b-mega-scale; protein-stability-20261009-eval-chu2024-mupro-mega-scale; protein-stability-20261009-eval-chu2024-rasp-mega-scale; protein-stability-20261009-eval-chu2024-rosetta-mega-scale
- review
- method: automated-source-review; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-09T23:59:00Z; note: Independent review of endpoint fit, relevance grade, limitations, exclusions and grouping against the use case and the source; every value of the listed evaluations was checked against the source table. No execution, reproduction, human scientific review or clinical validation.
- pins
- record id: protein-stability-20261009-eval-chu2024-elaspic-2-mega-scale; fields: status; links; attributes.origin; attributes.comparison; sha256: e25c5cbcf061bdf37a2889e7c7c57241c73ec7fb45c720abc70efd0dee392016; record id: protein-stability-20261009-eval-chu2024-esm-2-35m-mega-scale; fields: status; links; attributes.origin; attributes.comparison; sha256: 772c3670218407282e9243eda52d83c288add1f7ec1057e8b0c89fa94e9fa865; record id: protein-stability-20261009-eval-chu2024-esm-2-3b-mega-scale; fields: status; links; attributes.origin; attributes.comparison; sha256: 7703a5454b4dd581fb085c92c54c90a71b62e8f4cd8661f8113292b572351db7; record id: protein-stability-20261009-eval-chu2024-mupro-mega-scale; fields: status; links; attributes.origin; attributes.comparison; sha256: 63fb394b916db56e3cc8f2e567676b0373c055f2026d5e346a33f8faa51c293f; record id: protein-stability-20261009-eval-chu2024-rasp-mega-scale; fields: status; links; attributes.origin; attributes.comparison; sha256: f2d117ca5287f983651434e41eaef003d1ded77a6dd0e9538a724a45108ce860; record id: protein-stability-20261009-eval-chu2024-rosetta-mega-scale; fields: status; links; attributes.origin; attributes.comparison; sha256: d3fa3b024ea49abc67c2dc09b4de5e40328e880164275d8458f8d1ce76a23009; record id: protein-stability-20261009-protocol-chu2024-mega-scale; fields: status; name; description; facets; source_ids; links; attributes; sha256: c804d78c9b5f79cf44f4d8a64676edb8f2283c505026cabbbc27550c8c10b6f0; record id: protein-stability-20261009-result-chu2024-elaspic-2-mega-scale-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a8ca3493739c96fe78d2bbe8de27e56d2be0acceaecfaa608ec132732189a8d0; record id: protein-stability-20261009-result-chu2024-esm-2-35m-mega-scale-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 377ec36ac90c4520ddf018495ac0320ba5dd4f64aa1507a0ccec9230c688f6ab; record id: protein-stability-20261009-result-chu2024-esm-2-3b-mega-scale-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 1d8638baa92c85d45dcec1e409d6410265c8e7cf083755798a972ecc420cb20e; record id: protein-stability-20261009-result-chu2024-mupro-mega-scale-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7ee99a22eed951928dfb6e29fe3976f6fe4ff9f7306a5a00b6cd39d94628ac68; record id: protein-stability-20261009-result-chu2024-rasp-mega-scale-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a8ca3493739c96fe78d2bbe8de27e56d2be0acceaecfaa608ec132732189a8d0; record id: protein-stability-20261009-result-chu2024-rosetta-mega-scale-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f6bbc25f6760c82aba00ad40db1869002ccd85a00ef60fc81e150df4f383ca41; record id: protein-stability-20261009-source-chu2024; fields: status; attributes.artifact_sha256; sha256: e959594a702e1cfb9147c235f9a31116ada6106293e06b6e23185a14a95803b0; record id: use-case-protein-stability; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 2dd1d0e139a17e51ae72cc5945196ffbb366fe4daa4c20c4efd950346a30a54f