Datasets
CPF, SH3 and TRX family collections from UniProtKB/TrEMBL, with additional fungal SH3 sequences from NCBI.
Task
protein function annotation
Protein-function annotation is evaluated within selected domain-containing protein families.
Sources
Scaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passagesOverview
Metrics
The source defines a class-weighted macro F1 variant and weighted accuracy; these should not be assumed identical to ordinary macro F1.
Allowed inputs
Protein sequence representations.
Sources
Scaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passagesEvaluation procedure diagram
How it worksComputational evaluation flow
Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol.
Sources
Scaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passageslimited source coverage · Automated source review, 2026-09-16. All specifications and missing details
Results
Results are available, but no reviewed comparison panel is linked in this release.
All evaluations
1 evaluation · 1 result. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Pipeline: SPIN + ESM2-35M | Task: protein function annotation Dataset: TRX | 0.796 F1 macro-weighted fraction · unknown Uncertainty: Not reported Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceSPIN + ESM2-35M: protein function annotation frozen ESM2-35M backbone in SPIN Aggregation: Not reported Scaling the profile of life by function with SPIN · Table 1, ESM2-35M Test row, F1_m-w column |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Methods and evaluation design
Procedure, tasks and evaluated configurations
How it works
Evaluation methodology
CPF, SH3 and TRX family collections from UniProtKB/TrEMBL, with additional fungal SH3 sequences from NCBI. The source defines a class-weighted macro F1 variant and weighted accuracy; these should not be assumed identical to ordinary macro F1. Classical SVM/gradient-boosting/random-forest/KNN and neural CNN/LSTM/Transformer references; protein-model backbone comparisons form a separate TRX ablation. Sequences are clustered to reduce high similarity before subsequent filtering. Independent training seeds quantify initialization variation; the domain-span bootstrap is a separate uncertainty analysis.
Sources
Scaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passagesRecorded evaluations
Each evaluation records what was tested and under which conditions.
Run instructions
No runnable recipe has been reviewed for this task. Dataset access, model requirements, licences and compute requirements must be checked against its sources before execution.
A task describes a biological question. Choose a linked protocol to obtain concrete split and scoring instructions.
Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
No source-reviewed explanatory claims are recorded here yet.
Limitations and conditions
- The 80/10/10 within-family split and repeated random initializations do not establish sequence-homology exclusion. Domain membership alone does not define the functional classes.
Sources
Scaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages
Profile review details
Relevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged.
Stable record: reported-task-c4a578065f44b2Specifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Datasets | CPF, SH3 and TRX family collections from UniProtKB/TrEMBL, with additional fungal SH3 sequences from NCBI.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Splits | Each protein-family dataset has 80% training, 10% validation and 10% test data in Table 2. Three random initializations repeat training; the TRX training-size experiment keeps the validation and test sets fixed. These percentages do not establish a homology-disjoint split.SourcesScaling the profile of life by function with SPIN · Table 2 header/footnote; §2.3 Family-specific model tuning; §3.3 and Table 3 |
| Metrics | The source defines a class-weighted macro F1 variant and weighted accuracy; these should not be assumed identical to ordinary macro F1.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Baselines | Classical SVM/gradient-boosting/random-forest/KNN and neural CNN/LSTM/Transformer references; protein-model backbone comparisons form a separate TRX ablation.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Leakage controls | Sequences are clustered to reduce high similarity before subsequent filtering.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Uncertainty | Independent training seeds quantify initialization variation; the domain-span bootstrap is a separate uncertainty analysis.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Entity type | Paper-specific computational evaluation protocol.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Organisms | Protein families CPF, SH3 and TRX; fungal SH3 sequences form an additional collection.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Assays | UniProtKB/TrEMBL family/function annotations.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Allowed inputs | Protein sequence representations.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
| Adaptation | ESM2-35M fine-tuning with the Domain Span prediction module; the frozen-backbone configuration is a separate ablation.SourcesScaling the profile of life by function with SPIN · Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Papers and result coverage
Last literature check: 2026-09-17. Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
| Paper or primary resource | Version | Reference |
|---|---|---|
| Scaling the profile of life by function with SPIN | journal full text in PMC | Read source |
Historical gaps recorded on 2026-09-17
The catalogue now holds 1 result rows for this benchmark. A note below about pending extraction describes the state on 2026-09-17 and may since have been answered by a later batch. The result rows and their sources are the current record.
- complete numerical transcription and independent cell review: Full primary artifact and table inventory preserved; no new numeric row is published from this audit alone.
- exact checkpoint hashes and per-method scored denominators: Table labels alone do not establish these fields; do not infer checkpoint or scored count from model name or dataset size.
Search and extraction details
primary comparison tables located
Searches
- Scaling the profile of life by function with SPIN 10.1093/bioadv/vbag064
Evidence locations
- Table 1; XML table vbag064-T1
- Table 2; XML table vbag064-T2
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
17 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Diagram caption Conceptual summary of the cited evaluation; exact task configuration and source version remain part of the protocol. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Diagram steps
| Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Diagram title Computational evaluation flow Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Datasets CPF, SH3 and TRX family collections from UniProtKB/TrEMBL, with additional fungal SH3 sequences from NCBI. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Splits Each protein-family dataset has 80% training, 10% validation and 10% test data in Table 2. Three random initializations repeat training; the TRX training-size experiment keeps the validation and test sets fixed. These percentages do not establish a homology-disjoint split. Individual claims | Scaling the profile of life by function with SPIN Table 2 header/footnote; §2.3 Family-specific model tuning; §3.3 and Table 3 Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Adaptation ESM2-35M fine-tuning with the Domain Span prediction module; the frozen-backbone configuration is a separate ablation. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Metrics The source defines a class-weighted macro F1 variant and weighted accuracy; these should not be assumed identical to ordinary macro F1. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Baselines Classical SVM/gradient-boosting/random-forest/KNN and neural CNN/LSTM/Transformer references; protein-model backbone comparisons form a separate TRX ablation. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Leakage controls Sequences are clustered to reduce high similarity before subsequent filtering. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
| Uncertainty Independent training seeds quantify initialization variation; the domain-span bootstrap is a separate uncertainty analysis. Individual claims | Scaling the profile of life by function with SPIN Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages Version: journal full text in PMC | source checked automated source review · 2026-09-16 Audit detailsRelevant full-paper computational evaluation sections, tables/captions and cited supplementary task passages were reviewed. Reporting omissions are scoped to the inspected sources. Original numerical results are unchanged. Field: Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: needs review
2 source records and release history
- Scaling the profile of life by function with SPIN · Original source · journal full text in PMC
- Scaling the profile of life by function with SPIN · Original source · journal full text in PMC
Technical metadata and extraction receipts
Stable ID: reported-task-c4a578065f44b2
- areas
- proteins-complexes
- tasks
- protein function annotation
- entity level
- task
- version
- Not reported
- task
- protein function annotation
- scope note
- Paper-specific evaluation task; protocol completeness requires further extraction.
- benchmark research
- review date: 2026-09-17; status: primary_comparison_tables_located; primary sources: evidence-expansion-spin-protein-function-2026-9701843e; inspected locators: Table 1; XML table vbag064-T1; Table 2; XML table vbag064-T2; searched queries: Scaling the profile of life by function with SPIN 10.1093/bioadv/vbag064; gaps: complete numerical transcription and independent cell review: Full primary artifact and table inventory preserved; no new numeric row is published from this audit alone.; exact checkpoint hashes and per-method scored denominators: Table labels alone do not establish these fields; do not infer checkpoint or scored count from model name or dataset size.; claim scope: Dated primary-source discovery and protocol/table screening. Source checking does not mean experimental reproduction. Only separately extracted and independently reviewed numeric batches are publishable.
- historical missing metadata
- protocol version: not_reported_in_legacy_extract
- metadata review scope
- historical_missing_metadata preserves the original discovery state. Current descriptive evidence and missingness are recorded in profile.facts; numerical-result review is separate.
- legacy kinds
- benchmark
- entity classification
- review date: 2026-09-17; rationale: This source-scoped record identifies the biological prediction task and holds its paper context. Preserve the existing task identity; exact split, model adaptation and scoring remain in linked evaluations or separate protocol records.; source ids: spin-protein-function-2026; source locator: Methods §§2.6, 2.9; cached text lines 38–40, 48–50; comparative evaluation and ablation passages; Results embedding comparison and supervised-classifier comparison; uncertainty/repeat-run/statistical-comparison passages; ambiguities: A paper- or suite-specific task may constrain some inputs or metrics; that alone does not make it interchangeable with a complete versioned protocol. No protocol equivalence is inferred.; Some legacy profile Entity type facts use the generic phrase computational evaluation protocol. That boilerplate is not sufficient to establish a single fixed protocol identity or to merge this task with another protocol record.
Related records
- dataset: TRX
- benchmark: SPIN + ESM2-35M: protein function annotation