Relevance of DeepProtein Stability (rho) to "Assess methods for protein stability experiments"

Stable ID: use-case-mapping-lit-protein-20261011-deepprotein-stability-rho

field

links:assessed_by:lit-protein-20261011-deepprotein-protocol-stability-rho

value

lit-protein-20261011-deepprotein-protocol-stability-rho

relevance

proxy

endpoint

Rank correlation between predicted and measured stability on the TAPE stability test split (53571/2512/12851 train/valid/test), mean over five seeds

rationale

Proxy, not direct: the use case needs per-method correlation and error on held-out or dissimilar proteins with a stated construct and endpoint, and this protocol gives correlation on one supervised split of designed mini-protein stability measurements. It says nothing about antisymmetry on reverse variants, about transfer to larger proteins, or about ddG in kcal/mol.

constraints

Supervised split: every model is trained on the task's training set, so these values are not comparable with zero-shot stability rankings.; Spearman rank correlation on one supervised split; not comparable with zero-shot stability rankings or with ddG values in kcal/mol.; Compare only within this protocol and this split.; Inspect every linked evaluation's source locator before citing a result.

limitations

Author-reported: the DeepProtein authors ran every baseline and also introduce DeepProt-T5, which tops several tasks.; Graphormer ran out of memory on this task, so its cell is recorded with a missing value reason rather than a score.; One dataset and one split; no held-out-protein or homology-split stability evaluation.; The TAPE stability truth set is proteolysis-derived stability of designed mini-proteins, so it is biased towards small designed domains and away from the larger natural proteins the use case asks about.

citation locators

source id: lit-protein-20261011-deepprotein-source-shen2025; locator: Table 2, column group 'Stability', column 'rho'

source locator

Table 2, column group 'Stability', column 'rho'

headline metric

spearman-correlation

reviewed evaluations

lit-protein-20261011-deepprotein-eval-stability-rho-attentivefp; lit-protein-20261011-deepprotein-eval-stability-rho-chemllm-7b; lit-protein-20261011-deepprotein-eval-stability-rho-cnn; lit-protein-20261011-deepprotein-eval-stability-rho-cnn-rnn; lit-protein-20261011-deepprotein-eval-stability-rho-deep-prot-t5; lit-protein-20261011-deepprotein-eval-stability-rho-esm-1b; lit-protein-20261011-deepprotein-eval-stability-rho-esm-2-650m; lit-protein-20261011-deepprotein-eval-stability-rho-gat; lit-protein-20261011-deepprotein-eval-stability-rho-gcn; lit-protein-20261011-deepprotein-eval-stability-rho-graphormer; lit-protein-20261011-deepprotein-eval-stability-rho-llasmol-mistral-7b; lit-protein-20261011-deepprotein-eval-stability-rho-mpnn; lit-protein-20261011-deepprotein-eval-stability-rho-neuralfp; lit-protein-20261011-deepprotein-eval-stability-rho-pagtn; lit-protein-20261011-deepprotein-eval-stability-rho-prot-t5-xl; lit-protein-20261011-deepprotein-eval-stability-rho-protbert; lit-protein-20261011-deepprotein-eval-stability-rho-transformer

reason

Not shown: the source carries an evidence concern noting that Figure 3's caption calls the regression metric 'Spearman (Pearson) Coefficient' while the Evaluation metrics section defines it as Spearman. A source with evidence concerns withholds every judgement that cites it, so this judgement and the use-case summary edit that would describe it are held until the concern is cleared.

• subject: Assess methods for protein stability experiments