rewirebio.iobenchmarks
Evidence claim

Relevance of Acetyltransferase dataset stability ranking (Chu et al. 2024) to "Assess methods for protein stability experiments"

Use case
Assess methods for protein stability experiments
Assessed by
Acetyltransferase dataset stability ranking, Spearman correlation (Chu et al. 2024 Table 1)
Relevance
Proxy evidence
Endpoint
Spearman correlation for Rosetta, MUPro, RaSP, ELASPIC-2, ESM therm and two ESM-2 checkpoints on Acetyltransferase dataset

Measures ranking against chemical stability in a single larger protein, which is a proxy for folding stability and shows how methods transfer beyond small domains.

7 reviewed evaluations

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

20 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-11-21-adc15377fab2
Property and statementOriginal source and locationReview and provenance
Citation locations
1 values
  • source id
    protein-stability-20261009-source-chu2024
    locator
    Table 1 row 'Acetyltransferase dataset'; Table 2; Results
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.citation_locators

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison group
chu2024-transfer
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_group

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Comparison title
Transfer from mega-scale domains to larger proteins (Chu et al. 2024)
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.comparison_title

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Constraints
2 values
  • Inspect every linked evaluation's source locator before citing a result.
  • Do not combine this mapping's evaluations with any other protocol's results; datasets, splits, structures and predictor versions differ between sources.
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.constraints

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Endpoint
Spearman correlation for Rosetta, MUPro, RaSP, ELASPIC-2, ESM therm and two ESM-2 checkpoints on Acetyltransferase dataset
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.endpoint

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Claimed field
links:assessed_by:protein-stability-20261009-protocol-chu2024-acetyltransferase
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.field

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Headline metric
spearman-correlation
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.headline_metric

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Limitations
6 values
  • ESM therm was fine-tuned by the authors of this paper; the other methods were run by them as comparators.
  • Only Spearman correlation is printed; no RMSE, calibration or uncertainty.
  • The larger-protein benchmark was limited to six ProteinGym datasets and BglB because of Rosetta's compute cost (Methods P25).
  • Single protein (Gentamicin 3-N-acetyltransferase, 177 residues); the measured quantity is chemical stability, not ddG of folding.
  • ESM therm was fine-tuned on domains of at most 72 residues, so this is an out-of-range test for it.
  • The Results text gives ranges that do not match Table 1 (ELASPIC-2 0.42 to 0.58, Rosetta 0.33 to 0.48); Table 1 values are used.
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.limitations

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Pinned records
17 values
  • record id
    protein-stability-20261009-eval-chu2024-elaspic-2-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    b2c675be8f50703aabaf255673f86622dea338d5d4aef757d852fa3e313631f0
  • record id
    protein-stability-20261009-eval-chu2024-esm-2-35m-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    d07005f60ef0f453a953c8d5bafe995cb36c4b175236d9a65073fd30ee40a5ff
  • record id
    protein-stability-20261009-eval-chu2024-esm-2-3b-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    04050a9d49c5acfb02562aa454f9911953a9a5c1e2f059d66e82ca1fc9dcfb2d
  • record id
    protein-stability-20261009-eval-chu2024-esm-therm-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    68bcbefc902d11eef01c84eefc2ecea2256d04776f30eb36db87a76f29f45bd7
  • record id
    protein-stability-20261009-eval-chu2024-mupro-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    16eb1130857d6c532b315fbfafbd0cebc72751938c9eaaf6140eb55b2a9a9935
  • record id
    protein-stability-20261009-eval-chu2024-rasp-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    153c9e447078afc278f7d6c0620e081121f754caee5085d857578b93d42d0ba0
  • record id
    protein-stability-20261009-eval-chu2024-rosetta-acetyltransferase
    fields
    • status
    • links
    • attributes.origin
    • attributes.comparison
    sha256
    2899d7570eb2a06cafb1d3c902ef018696edfd97e6ee8fc22e1e74b030518bfe
  • record id
    protein-stability-20261009-protocol-chu2024-acetyltransferase
    fields
    • status
    • name
    • description
    • facets
    • source_ids
    • links
    • attributes
    sha256
    086f44d6f3de79524d260be1ed91e10941e1390fc90263e01c34c2896a9acf70
  • record id
    protein-stability-20261009-result-chu2024-elaspic-2-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9a00615dc158b0b5aa54186b51c2ce16bcc364ea3653047be74e51430bae8478
  • record id
    protein-stability-20261009-result-chu2024-esm-2-35m-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    63a6825f29de8f0eea31ff5fafa07f4300151d80298a1bc6c9ff9abfda35fc1f
  • record id
    protein-stability-20261009-result-chu2024-esm-2-3b-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    5ed1ba85e49c53caf5d2a7bab9a089c245894d27e8a62b4522e7926c47e7ae11
  • record id
    protein-stability-20261009-result-chu2024-esm-therm-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    0c4e695b948383dd34535b150e64e22d6229f3d96f637ab0e5df5bbda1f8c4e0
  • record id
    protein-stability-20261009-result-chu2024-mupro-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    f752ce68294276961ab3d1fde864a4b3658f1390db4957a27b7f4d99fb99ab4f
  • record id
    protein-stability-20261009-result-chu2024-rasp-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    995d1581d08731e098672ad03ef9722f2340ff7d9a5819296ba792b5de957a47
  • record id
    protein-stability-20261009-result-chu2024-rosetta-acetyltransferase-spearman-correlation
    fields
    • status
    • attributes.metric
    • attributes.metric_qualifier
    • attributes.printed_value
    • attributes.numeric_value
    sha256
    9c33552b95a6c060d631b773fe7e5661dd1d5fd633984d483cede0e1b64e6ce3
  • record id
    protein-stability-20261009-source-chu2024
    fields
    • status
    • attributes.artifact_sha256
    sha256
    e959594a702e1cfb9147c235f9a31116ada6106293e06b6e23185a14a95803b0
  • record id
    use-case-protein-stability
    fields
    • status
    • name
    • description
    • facets
    • attributes.question
    • attributes.decision
    • attributes.inputs
    • attributes.output
    • attributes.exclusions
    • attributes.setting
    • attributes.clinical_scope
    • attributes.intended_users
    sha256
    2dd1d0e139a17e51ae72cc5945196ffbb366fe4daa4c20c4efd950346a30a54f
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.pins

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Rationale
Measures ranking against chemical stability in a single larger protein, which is a proxy for folding stability and shows how methods transfer beyond small domains.
Context-only references
Protein stability prediction by fine-tuning a protein language model on a mega-scale dataset

Original source ↗

protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results

Version: PLOS Computational Biology 20(7):e1012248, published 2024-07-22; PMC11293664 full-text XML
Retrieved: 2026-10-09T21:01:08Z

not individually reviewed

No individual claim review recorded

Audit details

Field: attributes.rationale

Source artifact SHA-256: bb9830cbd6bab1b3e0ddbb52edb2a9afd4b06e427dbfeb19e6a07f843ebd914f

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

How to read the evidence

Sources and history

Release 2026-11-21-adc15377fab2 · Record review: source checked

1 source record and release historyDownload this release (gzip)
Technical metadata and extraction receipts

Stable ID: use-case-mapping-protein-stability-20261009-chu2024-acetyltransferase

field
links:assessed_by:protein-stability-20261009-protocol-chu2024-acetyltransferase
value
protein-stability-20261009-protocol-chu2024-acetyltransferase
relevance
proxy
endpoint
Spearman correlation for Rosetta, MUPro, RaSP, ELASPIC-2, ESM therm and two ESM-2 checkpoints on Acetyltransferase dataset
rationale
Measures ranking against chemical stability in a single larger protein, which is a proxy for folding stability and shows how methods transfer beyond small domains.
constraints
Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; datasets, splits, structures and predictor versions differ between sources.
limitations
ESM therm was fine-tuned by the authors of this paper; the other methods were run by them as comparators.; Only Spearman correlation is printed; no RMSE, calibration or uncertainty.; The larger-protein benchmark was limited to six ProteinGym datasets and BglB because of Rosetta's compute cost (Methods P25).; Single protein (Gentamicin 3-N-acetyltransferase, 177 residues); the measured quantity is chemical stability, not ddG of folding.; ESM therm was fine-tuned on domains of at most 72 residues, so this is an out-of-range test for it.; The Results text gives ranges that do not match Table 1 (ELASPIC-2 0.42 to 0.58, Rosetta 0.33 to 0.48); Table 1 values are used.
revision
1
reason
Add primary-source stability predictor comparison evidence from the protein stability use-case pass (2026-10-09).
citation locators
source id: protein-stability-20261009-source-chu2024; locator: Table 1 row 'Acetyltransferase dataset'; Table 2; Results
source locator
protein-stability-20261009-source-chu2024: Table 1 row 'Acetyltransferase dataset'; Table 2; Results
comparison group
chu2024-transfer
comparison title
Transfer from mega-scale domains to larger proteins (Chu et al. 2024)
headline metric
spearman-correlation
stratum label
Gentamicin 3-N-acetyltransferase (chemical stability)
stratum order
4
reviewed evaluations
protein-stability-20261009-eval-chu2024-elaspic-2-acetyltransferase; protein-stability-20261009-eval-chu2024-esm-2-35m-acetyltransferase; protein-stability-20261009-eval-chu2024-esm-2-3b-acetyltransferase; protein-stability-20261009-eval-chu2024-esm-therm-acetyltransferase; protein-stability-20261009-eval-chu2024-mupro-acetyltransferase; protein-stability-20261009-eval-chu2024-rasp-acetyltransferase; protein-stability-20261009-eval-chu2024-rosetta-acetyltransferase
review
method: automated-source-review; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-09T23:59:00Z; note: Independent review of endpoint fit, relevance grade, limitations, exclusions and grouping against the use case and the source; every value of the listed evaluations was checked against the source table. No execution, reproduction, human scientific review or clinical validation.
pins
record id: protein-stability-20261009-eval-chu2024-elaspic-2-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: b2c675be8f50703aabaf255673f86622dea338d5d4aef757d852fa3e313631f0; record id: protein-stability-20261009-eval-chu2024-esm-2-35m-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: d07005f60ef0f453a953c8d5bafe995cb36c4b175236d9a65073fd30ee40a5ff; record id: protein-stability-20261009-eval-chu2024-esm-2-3b-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: 04050a9d49c5acfb02562aa454f9911953a9a5c1e2f059d66e82ca1fc9dcfb2d; record id: protein-stability-20261009-eval-chu2024-esm-therm-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: 68bcbefc902d11eef01c84eefc2ecea2256d04776f30eb36db87a76f29f45bd7; record id: protein-stability-20261009-eval-chu2024-mupro-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: 16eb1130857d6c532b315fbfafbd0cebc72751938c9eaaf6140eb55b2a9a9935; record id: protein-stability-20261009-eval-chu2024-rasp-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: 153c9e447078afc278f7d6c0620e081121f754caee5085d857578b93d42d0ba0; record id: protein-stability-20261009-eval-chu2024-rosetta-acetyltransferase; fields: status; links; attributes.origin; attributes.comparison; sha256: 2899d7570eb2a06cafb1d3c902ef018696edfd97e6ee8fc22e1e74b030518bfe; record id: protein-stability-20261009-protocol-chu2024-acetyltransferase; fields: status; name; description; facets; source_ids; links; attributes; sha256: 086f44d6f3de79524d260be1ed91e10941e1390fc90263e01c34c2896a9acf70; record id: protein-stability-20261009-result-chu2024-elaspic-2-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9a00615dc158b0b5aa54186b51c2ce16bcc364ea3653047be74e51430bae8478; record id: protein-stability-20261009-result-chu2024-esm-2-35m-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 63a6825f29de8f0eea31ff5fafa07f4300151d80298a1bc6c9ff9abfda35fc1f; record id: protein-stability-20261009-result-chu2024-esm-2-3b-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5ed1ba85e49c53caf5d2a7bab9a089c245894d27e8a62b4522e7926c47e7ae11; record id: protein-stability-20261009-result-chu2024-esm-therm-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0c4e695b948383dd34535b150e64e22d6229f3d96f637ab0e5df5bbda1f8c4e0; record id: protein-stability-20261009-result-chu2024-mupro-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f752ce68294276961ab3d1fde864a4b3658f1390db4957a27b7f4d99fb99ab4f; record id: protein-stability-20261009-result-chu2024-rasp-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 995d1581d08731e098672ad03ef9722f2340ff7d9a5819296ba792b5de957a47; record id: protein-stability-20261009-result-chu2024-rosetta-acetyltransferase-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 9c33552b95a6c060d631b773fe7e5661dd1d5fd633984d483cede0e1b64e6ce3; record id: protein-stability-20261009-source-chu2024; fields: status; attributes.artifact_sha256; sha256: e959594a702e1cfb9147c235f9a31116ada6106293e06b6e23185a14a95803b0; record id: use-case-protein-stability; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 2dd1d0e139a17e51ae72cc5945196ffbb366fe4daa4c20c4efd950346a30a54f
Related records

Suggest a correction