Relevance of Fireprot homologue-free split ddG prediction (Dieckhaus et al. 2024) to "Assess methods for protein stability experiments"
- Use case
- Assess methods for protein stability experiments
- Assessed by
- Fireprot homologue-free split ddG prediction (Dieckhaus et al. 2024 Table 2)
- Relevance
- Direct evidence
- Endpoint
- RMSE and correlation of predicted ddG on the Fireprot homologue-free split for the ThermoMPNN comparison set
Held-out proteins with no homologue in the training sets directly test generalisation to unseen proteins, the validation step the use case asks about.
6 reviewed evaluations
- FoldX on Fireprot homologue-free split (Dieckhaus et al. 2024)
- PROSTATA on Fireprot homologue-free split (Dieckhaus et al. 2024)
- ProteinMPNN on Fireprot homologue-free split (Dieckhaus et al. 2024)
- RaSP on Fireprot homologue-free split (Dieckhaus et al. 2024)
- Rosetta on Fireprot homologue-free split (Dieckhaus et al. 2024)
- ThermoMPNN on Fireprot homologue-free split (Dieckhaus et al. 2024)
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
21 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
Citation locations1 values
| Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison group dieckhaus2024-thermompnn Context-only references | Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Comparison title ThermoMPNN comparison sets (Dieckhaus et al. 2024) Context-only references | Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Constraints2 values
| Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Endpoint RMSE and correlation of predicted ddG on the Fireprot homologue-free split for the ThermoMPNN comparison set Context-only references | Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Excluded evaluations6 values
| Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Claimed field links:assessed_by:protein-stability-20261009-protocol-dieckhaus2024-fireprot Context-only references | Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Headline metric pearson-correlation Context-only references | Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Limitations4 values
| Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Pinned records27 values
| Transfer learning to leverage larger datasets for improved prediction of protein stability changes protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets' Version: PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML | not individually reviewed No individual claim review recorded Audit detailsField: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
Release 2026-11-21-adc15377fab2 · Record review: source checked
1 source record and release history
- Transfer learning to leverage larger datasets for improved prediction of protein stability changes · Original source · PNAS 121(6):e2314853121, published 2024-01-29; PMC10861915 full-text XML
Technical metadata and extraction receipts
Stable ID: use-case-mapping-protein-stability-20261009-dieckhaus2024-fireprot
- field
- links:assessed_by:protein-stability-20261009-protocol-dieckhaus2024-fireprot
- value
- protein-stability-20261009-protocol-dieckhaus2024-fireprot
- relevance
- direct
- endpoint
- RMSE and correlation of predicted ddG on the Fireprot homologue-free split for the ThermoMPNN comparison set
- rationale
- Held-out proteins with no homologue in the training sets directly test generalisation to unseen proteins, the validation step the use case asks about.
- constraints
- Inspect every linked evaluation's source locator before citing a result.; Do not combine this mapping's evaluations with any other protocol's results; datasets, splits, structures and predictor versions differ between sources.
- limitations
- Developer comparison (ThermoMPNN authors).; Six methods marked * in Table 2 (ACDC-NN, ACDC-NN-Seq, ThermoNet, MAESTRO, mCSM, MUPRO) are excluded because their training sets contain close homologues of Fireprot proteins.; The homologue-free split has 89 proteins and 2,578 mutations; the three Fireprot proteins with more than 250 measurements were kept in training.; No uncertainty is printed.
- revision
- 1
- reason
- Add primary-source stability predictor comparison evidence from the protein stability use-case pass (2026-10-09).
- citation locators
- source id: protein-stability-20261009-source-dieckhaus2024; locator: Table 2 and footnotes; Methods 'Datasets'
- source locator
- protein-stability-20261009-source-dieckhaus2024: Table 2 and footnotes; Methods 'Datasets'
- comparison group
- dieckhaus2024-thermompnn
- comparison title
- ThermoMPNN comparison sets (Dieckhaus et al. 2024)
- headline metric
- pearson-correlation
- stratum label
- Fireprot homologue-free split
- stratum order
- 2
- excluded evaluations
- id: protein-stability-20261009-eval-dieckhaus2024-acdc-nn-fireprot; reason: Marked * in Table 2: the score may be inflated because this method's training set contains close homologues (over 25% identity) of Fireprot proteins, so it is not a homologue-free result. Excluded at independent review 2026-10-09.; id: protein-stability-20261009-eval-dieckhaus2024-acdc-nn-seq-fireprot; reason: Marked * in Table 2: the score may be inflated because this method's training set contains close homologues (over 25% identity) of Fireprot proteins, so it is not a homologue-free result. Excluded at independent review 2026-10-09.; id: protein-stability-20261009-eval-dieckhaus2024-thermonet-fireprot; reason: Marked * in Table 2: the score may be inflated because this method's training set contains close homologues (over 25% identity) of Fireprot proteins, so it is not a homologue-free result. Excluded at independent review 2026-10-09.; id: protein-stability-20261009-eval-dieckhaus2024-maestro-fireprot; reason: Marked * in Table 2: the score may be inflated because this method's training set contains close homologues (over 25% identity) of Fireprot proteins, so it is not a homologue-free result. Excluded at independent review 2026-10-09.; id: protein-stability-20261009-eval-dieckhaus2024-mcsm-fireprot; reason: Marked * in Table 2: the score may be inflated because this method's training set contains close homologues (over 25% identity) of Fireprot proteins, so it is not a homologue-free result. Excluded at independent review 2026-10-09.; id: protein-stability-20261009-eval-dieckhaus2024-mupro-fireprot; reason: Marked * in Table 2: the score may be inflated because this method's training set contains close homologues (over 25% identity) of Fireprot proteins, so it is not a homologue-free result. Excluded at independent review 2026-10-09.
- reviewed evaluations
- protein-stability-20261009-eval-dieckhaus2024-foldx-fireprot; protein-stability-20261009-eval-dieckhaus2024-prostata-fireprot; protein-stability-20261009-eval-dieckhaus2024-proteinmpnn-fireprot; protein-stability-20261009-eval-dieckhaus2024-rasp-fireprot; protein-stability-20261009-eval-dieckhaus2024-rosetta-fireprot; protein-stability-20261009-eval-dieckhaus2024-thermompnn-fireprot
- review
- method: automated-source-review; reviewer: claude; reviewer note: Separate Claude review agent, independent of the extractor; no human review claimed; reviewed at: 2026-10-09T23:59:00Z; note: Independent review of endpoint fit, relevance grade, limitations, exclusions and grouping against the use case and the source; every value of the listed evaluations was checked against the source table. No execution, reproduction, human scientific review or clinical validation.
- pins
- record id: protein-stability-20261009-eval-dieckhaus2024-foldx-fireprot; fields: status; links; attributes.origin; attributes.comparison; sha256: a793377719bc111d867a1de29a6920103c2ea8405272c122f33d079c0deb2082; record id: protein-stability-20261009-eval-dieckhaus2024-prostata-fireprot; fields: status; links; attributes.origin; attributes.comparison; sha256: 34a0f1dedfbf6183a87904b72d40492f29908152f75ab58b3e6c56052f3ace71; record id: protein-stability-20261009-eval-dieckhaus2024-proteinmpnn-fireprot; fields: status; links; attributes.origin; attributes.comparison; sha256: fa3afb401fa9f7cf4991c26033be8f72c501d8185c70b20a26a8049e757fee89; record id: protein-stability-20261009-eval-dieckhaus2024-rasp-fireprot; fields: status; links; attributes.origin; attributes.comparison; sha256: 99b42cec37ed9e85d78161690e1155101f24043ef134d40451e1512672cfd4c8; record id: protein-stability-20261009-eval-dieckhaus2024-rosetta-fireprot; fields: status; links; attributes.origin; attributes.comparison; sha256: b5dd6b20c407088cbe3dd3df460ee69cec32c43c857bf37c914be3ede1d1fa7e; record id: protein-stability-20261009-eval-dieckhaus2024-thermompnn-fireprot; fields: status; links; attributes.origin; attributes.comparison; sha256: 1c8b377cdf943d6f6922392fe930cd4e609c30c32aa70282552da67db06ffd34; record id: protein-stability-20261009-protocol-dieckhaus2024-fireprot; fields: status; name; description; facets; source_ids; links; attributes; sha256: ddea3cd3d29a67284856641292c8e8ab31a55e2d95a6da8d7705b6a917576770; record id: protein-stability-20261009-result-dieckhaus2024-foldx-fireprot-pearson-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: f5c3195a37385fd00954045a93179fa48eb2c438264aa07eff7eddc5bf982c60; record id: protein-stability-20261009-result-dieckhaus2024-foldx-fireprot-root-mean-squared-error; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 0284e6594ac8bfb05264bd5a31e2d9599e990a66a79d68a2e41489a87c5313a6; record id: protein-stability-20261009-result-dieckhaus2024-foldx-fireprot-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 7eefc04214727327bb90658098d89651306f68f69563239d79c2e56bacfb8fa5; record id: protein-stability-20261009-result-dieckhaus2024-prostata-fireprot-pearson-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 773fa9532e32113189ce70cd2a45fcf9d65365338d155a50f0b2055d15abb941; record id: protein-stability-20261009-result-dieckhaus2024-prostata-fireprot-root-mean-squared-error; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: ab11e7be8fc6fed7479edf0bc4b3ce76e0930260cdd507bafa922fc0006c2629; record id: protein-stability-20261009-result-dieckhaus2024-prostata-fireprot-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: e4ec7aa36b7cb1327a03c3422c7d2501374ef7721372bbb01783e3439597f547; record id: protein-stability-20261009-result-dieckhaus2024-proteinmpnn-fireprot-pearson-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 5faa1170f6e426f1ec4faca0981a46caf4ab85e15d27954b7b3617482811452a; record id: protein-stability-20261009-result-dieckhaus2024-proteinmpnn-fireprot-root-mean-squared-error; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 70dae993f057deabe1c9010c5ba5faacb4741307e0bc60a923aeb2b0eeca933d; record id: protein-stability-20261009-result-dieckhaus2024-proteinmpnn-fireprot-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3ff17d88efcf8a3f34ad04aa06440dab9c9f68800b2d6ff1283ad0752d45412a; record id: protein-stability-20261009-result-dieckhaus2024-rasp-fireprot-pearson-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6b67789f98c86027b541df7aa6380172011408a2cc9094d9814dce907619c56b; record id: protein-stability-20261009-result-dieckhaus2024-rasp-fireprot-root-mean-squared-error; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 2e493207b6fbede781ce1c0b46a58634be3926b087c83a899b7dc1cc6311335e; record id: protein-stability-20261009-result-dieckhaus2024-rasp-fireprot-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: de5f73babce6334b51da411c2709eee7ad0833407f37320188f1e2540407bd75; record id: protein-stability-20261009-result-dieckhaus2024-rosetta-fireprot-pearson-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3e147b6c0afd80ea297b05a9c1e13b1da1097873a24148162f6eaf8857189647; record id: protein-stability-20261009-result-dieckhaus2024-rosetta-fireprot-root-mean-squared-error; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: a6674d1ddd0e6904eb061c2dfa6e72db0c4bb021862aa6c5cdfdc8d9ada4aa81; record id: protein-stability-20261009-result-dieckhaus2024-rosetta-fireprot-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 3a8763393c18774fad452a1ddcecd2b333c74248253fd2f5723bdf5782db2bb7; record id: protein-stability-20261009-result-dieckhaus2024-thermompnn-fireprot-pearson-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 35130b150b8a0515bc065da760e1929d46e28e2f9ceff5cd709082b7a11c2c1b; record id: protein-stability-20261009-result-dieckhaus2024-thermompnn-fireprot-root-mean-squared-error; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: bca2d5de5864c66777fe4811f9ae310ee48941eda4a25d9ac4593ee354643404; record id: protein-stability-20261009-result-dieckhaus2024-thermompnn-fireprot-spearman-correlation; fields: status; attributes.metric; attributes.metric_qualifier; attributes.printed_value; attributes.numeric_value; sha256: 6494b7ccdbccacfe2c12d5c0825e624a90b1e7f13d8e0fb5dd66f59780a22633; record id: protein-stability-20261009-source-dieckhaus2024; fields: status; attributes.artifact_sha256; sha256: e4c02837eab159fbef7943f8346e38196625d3a62702c961e9e1e54d1d61464e; record id: use-case-protein-stability; fields: status; name; description; facets; attributes.question; attributes.decision; attributes.inputs; attributes.output; attributes.exclusions; attributes.setting; attributes.clinical_scope; attributes.intended_users; sha256: 2dd1d0e139a17e51ae72cc5945196ffbb366fe4daa4c20c4efd950346a30a54f