Source

DeepProtein: deep learning library and benchmark for protein sequence learning

Bioinformatics · 2025 · Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) · Peer-reviewed

DOI: 10.1093/bioinformatics/btaf165 Original source

What this source contributed

Records in this release that cite this source. A record can cite several sources.

16 models
18 configurations
1 benchmark
60 protocols
17 datasets
436 evaluations
436 results
1 evidence claim

Results

Values as the source printed them. Compare results only under compatible protocols, datasets and metrics.

Value	Metric	Model or configuration	Dataset	Evaluation
0.262 ± 0.007 unit-unreported	MAE (mean absolute error)	AttentiveFP (DeepProtein benchmark)	beta-lactamase (PEER)	AttentiveFP on DeepProtein beta-lactamase (PEER) (MAE ↓)
0.058±0.011 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	AttentiveFP (DeepProtein benchmark)	beta-lactamase (PEER)	AttentiveFP on DeepProtein beta-lactamase (PEER) (ρ ↑)
80.08 ± 0.39 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	AttentiveFP (DeepProtein benchmark)	Binary localization (PEER)	AttentiveFP on DeepProtein Binary localization (PEER) (Averaged F1 ↑)
80.58±0.30 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	AttentiveFP (DeepProtein benchmark)	Binary localization (PEER)	AttentiveFP on DeepProtein Binary localization (PEER) (PR-AUC ↑)
1.285 ± 0.001 unit-unreported	MAE (mean absolute error)	AttentiveFP (DeepProtein benchmark)	Fluorescence (TAPE)	AttentiveFP on DeepProtein Fluorescence (TAPE) (MAE ↓)
0.180±0.006 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	AttentiveFP (DeepProtein benchmark)	Fluorescence (TAPE)	AttentiveFP on DeepProtein Fluorescence (TAPE) (ρ ↑)
39.21 ± 0.66 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	AttentiveFP (DeepProtein benchmark)	Solubility (PEER)	AttentiveFP on DeepProtein Solubility (PEER) (Averaged F1 ↑)
60.56 ± 0.74 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	AttentiveFP (DeepProtein benchmark)	Solubility (PEER)	AttentiveFP on DeepProtein Solubility (PEER) (PR-AUC ↑)
0.763 ± 0.003 unit-unreported	MAE (mean absolute error)	AttentiveFP (DeepProtein benchmark)	Stability (TAPE)	AttentiveFP on DeepProtein Stability (TAPE) (MAE ↓)
0.013±0.004 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	AttentiveFP (DeepProtein benchmark)	Stability (TAPE)	AttentiveFP on DeepProtein Stability (TAPE) (ρ ↑)
42.38±1.25 percent	Accuracy (accuracy, printed on a 0 to 100 scale)	AttentiveFP (DeepProtein benchmark)	Subcellular localization (PEER)	AttentiveFP on DeepProtein Subcellular localization (PEER) (Acc ↑)
23.50 ± 1.34 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	AttentiveFP (DeepProtein benchmark)	Subcellular localization (PEER)	AttentiveFP on DeepProtein Subcellular localization (PEER) (Averaged F1 ↑)
111.76 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	beta-lactamase (PEER)	ChemLLM-7B on DeepProtein beta-lactamase (PEER) (MAE ↓)
0.017 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	ChemLLM-7B (DeepProtein benchmark)	beta-lactamase (PEER)	ChemLLM-7B on DeepProtein beta-lactamase (PEER) (ρ ↑)
1.43 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Binary localization (PEER)	ChemLLM-7B on DeepProtein Binary localization (PEER) (Averaged F1 ↑)
56.32 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Binary localization (PEER)	ChemLLM-7B on DeepProtein Binary localization (PEER) (PR-AUC ↑)
20.0503 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	CRISPR-Leenay repair outcome	ChemLLM-7B on DeepProtein CRISPR-Leenay repair outcome (MAE ↓)
0.061 unitless	R² (coefficient of determination)	ChemLLM-7B (DeepProtein benchmark)	CRISPR-Leenay repair outcome	ChemLLM-7B on DeepProtein CRISPR-Leenay repair outcome (R2 ↑)
19.878 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	Fluorescence (TAPE)	ChemLLM-7B on DeepProtein Fluorescence (TAPE) (MAE ↓)
−0.019 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	ChemLLM-7B (DeepProtein benchmark)	Fluorescence (TAPE)	ChemLLM-7B on DeepProtein Fluorescence (TAPE) (ρ ↑)
0.64 percent	Accuracy (accuracy, printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Fold classification (TAPE)	ChemLLM-7B on DeepProtein Fold classification (TAPE) (Acc ↑)
0.04 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Fold classification (TAPE)	ChemLLM-7B on DeepProtein Fold classification (TAPE) (Averaged F1 ↑)
15.49 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Human PPI (PEER)	ChemLLM-7B on DeepProtein Human PPI (PEER) (Averaged F1 ↑)
47.15 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Human PPI (PEER)	ChemLLM-7B on DeepProtein Human PPI (PEER) (PR-AUC ↑)
28.771 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	PPI affinity (PEER)	ChemLLM-7B on DeepProtein PPI affinity (PEER) (MAE ↓)
0.082 unitless	R² (coefficient of determination)	ChemLLM-7B (DeepProtein benchmark)	PPI affinity (PEER)	ChemLLM-7B on DeepProtein PPI affinity (PEER) (R2 ↑)
32.33 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	SAbDab-Chen antibody developability	ChemLLM-7B on DeepProtein SAbDab-Chen antibody developability (MAE ↓)
−0.006 unitless	R² (coefficient of determination)	ChemLLM-7B (DeepProtein benchmark)	SAbDab-Chen antibody developability	ChemLLM-7B on DeepProtein SAbDab-Chen antibody developability (R2 ↑)
1.64 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Solubility (PEER)	ChemLLM-7B on DeepProtein Solubility (PEER) (Averaged F1 ↑)
50.65 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Solubility (PEER)	ChemLLM-7B on DeepProtein Solubility (PEER) (PR-AUC ↑)
46273.897 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	Stability (TAPE)	ChemLLM-7B on DeepProtein Stability (TAPE) (MAE ↓)
−0.187 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	ChemLLM-7B (DeepProtein benchmark)	Stability (TAPE)	ChemLLM-7B on DeepProtein Stability (TAPE) (ρ ↑)
6.24 percent	Accuracy (accuracy, printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Subcellular localization (PEER)	ChemLLM-7B on DeepProtein Subcellular localization (PEER) (Acc ↑)
0.59 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Subcellular localization (PEER)	ChemLLM-7B on DeepProtein Subcellular localization (PEER) (Averaged F1 ↑)
45.957 unit-unreported	MAE (mean absolute error)	ChemLLM-7B (DeepProtein benchmark)	TAP antibody developability	ChemLLM-7B on DeepProtein TAP antibody developability (MAE ↓)
0.099 unitless	R² (coefficient of determination)	ChemLLM-7B (DeepProtein benchmark)	TAP antibody developability	ChemLLM-7B on DeepProtein TAP antibody developability (R2 ↑)
15.45 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Yeast PPI (PEER)	ChemLLM-7B on DeepProtein Yeast PPI (PEER) (Averaged F1 ↑)
53.65 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	ChemLLM-7B (DeepProtein benchmark)	Yeast PPI (PEER)	ChemLLM-7B on DeepProtein Yeast PPI (PEER) (PR-AUC ↑)
0.152 ± 0.001 unit-unreported	MAE (mean absolute error)	CNN (DeepProtein benchmark)	beta-lactamase (PEER)	CNN on DeepProtein beta-lactamase (PEER) (MAE ↓)
0.721 ± 0.020 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	CNN (DeepProtein benchmark)	beta-lactamase (PEER)	CNN on DeepProtein beta-lactamase (PEER) (ρ ↑)
87.74 ± 0.08 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	CNN (DeepProtein benchmark)	Binary localization (PEER)	CNN on DeepProtein Binary localization (PEER) (Averaged F1 ↑)
90.88±0.31 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	CNN (DeepProtein benchmark)	Binary localization (PEER)	CNN on DeepProtein Binary localization (PEER) (PR-AUC ↑)
0.0745 ± 0.0005 unit-unreported	MAE (mean absolute error)	CNN (DeepProtein benchmark)	CRISPR-Leenay repair outcome	CNN on DeepProtein CRISPR-Leenay repair outcome (MAE ↓)
0.781 ± 0.009 unitless	R² (coefficient of determination)	CNN (DeepProtein benchmark)	CRISPR-Leenay repair outcome	CNN on DeepProtein CRISPR-Leenay repair outcome (R2 ↑)
0.194 ± 0.001 unit-unreported	MAE (mean absolute error)	CNN (DeepProtein benchmark)	Fluorescence (TAPE)	CNN on DeepProtein Fluorescence (TAPE) (MAE ↓)
0.680 ± 0.001 unitless	Spearman (Spearman rank correlation, as defined in the article's Evaluation metrics section; supervised split)	CNN (DeepProtein benchmark)	Fluorescence (TAPE)	CNN on DeepProtein Fluorescence (TAPE) (ρ ↑)
8.01 ± 0.44 percent	Accuracy (accuracy, printed on a 0 to 100 scale)	CNN (DeepProtein benchmark)	Fold classification (TAPE)	CNN on DeepProtein Fold classification (TAPE) (Acc ↑)
0.47 ± 0.11 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	CNN (DeepProtein benchmark)	Fold classification (TAPE)	CNN on DeepProtein Fold classification (TAPE) (Averaged F1 ↑)
69.65 ± 1.63 percent	Macro F1 (averaged macro F1 (Figure 3 caption: 'averaged macro F1'), printed on a 0 to 100 scale)	CNN (DeepProtein benchmark)	Human PPI (PEER)	CNN on DeepProtein Human PPI (PEER) (Averaged F1 ↑)
70.37±1.22 percent	AUPRC (area under the precision-recall curve, printed on a 0 to 100 scale)	CNN (DeepProtein benchmark)	Human PPI (PEER)	CNN on DeepProtein Human PPI (PEER) (PR-AUC ↑)

Claims

Statements this source supports, each with where in the source it was checked.

Relevance of DeepProtein Stability (rho) to "Assess methods for protein stability experiments": Assessed by DeepProtein Stability (TAPE): ρ ↑ (Table 2, column group 'Stability', column 'rho')

Models, benchmarks and other records

Models (16)

Configurations (18)

Benchmarks (1)

DeepProtein benchmark

Protocols (60)

Datasets (17)

Evaluations (436)

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

26 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-10-22-1b7cf027e4f6
Property and statement	Original source and location	Review and provenance
attributes.artifact_sha256 b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.artifact_sha256` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Artifact url https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12553333/fullTextXML Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.artifact_url` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Doi 10.1093/bioinformatics/btaf165 Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.doi` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Evidence concerns 1 values source id lit-protein-20261011-deepprotein-source-shen2025 message Figure 3's caption calls the regression metric 'Spearman (Pearson) Coefficient' while the Evaluation metrics section defines rho as Spearman rank correlation and R2 separately. The section's definition is taken as authoritative and the rho columns are recorded as Spearman; the caption wording is noted so a reader of the figure is not surprised. source locator Tables 2, 4 and 6 column headings 'rho' and 'R2'; Figure 3 caption artifact sha256 b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f review date 2026-10-11 Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.evidence_concerns` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Licence CC BY 4.0 Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.licence` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Limitations 4 values Tables 2 to 8. Table 1 (a feature comparison with earlier benchmark libraries), Table 9 (default hyperparameters) and the task-instruction table hold no measurements and were not extracted. The regression correlation columns are printed as rho. The Evaluation metrics section defines rho as Spearman rank correlation; Figure 3's caption calls it 'Spearman (Pearson) Coefficient'. The section's definition is used. Cells printed as '/' are not measurements: the Table 7 footnote says structure-based and pretrained protein language models were disregarded for residue-level and secondary structure prediction. They are left out, not recorded as zero. Bold, underline and the '**' significance marks are kept in printed_value where they are part of the printed cell; the article's own statistical tests were not re-run. Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.limitations` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Media type application/xml Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.media_type` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Publication status peer_reviewed Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.publication_status` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Retrieved at 2026-10-11T21:30:00Z Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.retrieved_at` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact
Review: date 2026-10-11 Source metadata	DeepProtein: deep learning library and benchmark for protein sequence learning Original source ↗ No field-specific location recorded Version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text) Retrieved: 2026-10-11T21:30:00Z	catalogued No individual claim review recorded Source has a recorded evidence concern. Consult its source page before using the claim. Audit details Field: `attributes.review.date` Source artifact SHA-256: `b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f` Hash scope: Hash scope not separately documented; inspect source record Inspected artifact

How to read the evidence

Sources and history

Release 2026-10-22-1b7cf027e4f6 · Record review: source checked

Primary source retrieved and hashed for the 2026-10-11 protein and structure literature pass.

Release history

No supporting source is linked yet.

Read original source

Download this release (gzip)

Technical metadata and extraction receipts

Stable ID: lit-protein-20261011-deepprotein-source-shen2025

areas: protein-function
contexts: research
url: https://pmc.ncbi.nlm.nih.gov/articles/PMC12553333/
artifact url: https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12553333/fullTextXML
doi: 10.1093/bioinformatics/btaf165
version: Version of record, Bioinformatics 2025 (PMC12553333 JATS full text)
retrieved at: 2026-10-11T21:30:00Z
artifact sha256: b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f
publication status: peer_reviewed
licence: CC BY 4.0
media type: application/xml
venue: Bioinformatics
year: 2025
limitations: Tables 2 to 8. Table 1 (a feature comparison with earlier benchmark libraries), Table 9 (default hyperparameters) and the task-instruction table hold no measurements and were not extracted.; The regression correlation columns are printed as rho. The Evaluation metrics section defines rho as Spearman rank correlation; Figure 3's caption calls it 'Spearman (Pearson) Coefficient'. The section's definition is used.; Cells printed as '/' are not measurements: the Table 7 footnote says structure-based and pretrained protein language models were disregarded for residue-level and secondary structure prediction. They are left out, not recorded as zero.; Bold, underline and the '**' significance marks are kept in printed_value where they are part of the printed cell; the article's own statistical tests were not re-run.
evidence concerns: source id: lit-protein-20261011-deepprotein-source-shen2025; message: Figure 3's caption calls the regression metric 'Spearman (Pearson) Coefficient' while the Evaluation metrics section defines rho as Spearman rank correlation and R2 separately. The section's definition is taken as authoritative and the rho columns are recorded as Spearman; the caption wording is noted so a reader of the figure is not surprised.; source locator: Tables 2, 4 and 6 column headings 'rho' and 'R2'; Figure 3 caption; artifact sha256: b586921363170806d9f332afb37762ea61f464da6067a3ef0ab8853281964b7f; review date: 2026-10-11
review: method: source-hash-verification; method note: Artifact re-retrieved from Europe PMC and SHA-256 matched.; reviewer: claude; reviewer note: Claude review agent reviewer-ps-1, independent of the extractor lit-protein-structure; no human review claimed; date: 2026-10-11

Related records

Suggest a correction